When solving a pattern classification problem, it is common to apply a feature extraction method as a pre-processing step, not only to reduce the computation complexity but also to obtain better classification performance by reducing the amount of irrelevant and redundant information in the data. In this study, we investigate a novel schema for linear feature extraction in classification problems. The method we have proposed is based on clustering technique to realize feature extraction. It focuses in identifying and transforming redundant information in the data. A new similarity measure-based trend analysis is devised to identify those features. The simulation results on face recognition show that the proposed method gives better or competitive results when compared to conventional unsupervised methods like PCA and ICA.