We propose an unsupervised fuzzy rule-based system to learn structure preserving data projection. Although, the framework is quite general and any structure preserving measure can be used, we use Sammon’s stress, an extensively used objective function for dimensionality reduction. Unlike Sammon’s method, it can predict the projection for new test points. To extract fuzzy rules, we perform fuzzy c-means clustering on the input data and translate the clusters to the antecedent parts of the rules. Initially, we set the consequent parameters of the rules with random values. We estimate the parameters of the rule base minimizing the Sammon’s stress error function using gradient descent. We explore both Mamdani-Assilian and Takagi-Sugeno type fuzzy rule-based systems. An additional advantage of the proposed system over a neural network based generalization of the Sammon’s method is that the proposed system can reject the test data that are far from the training data used to design the system. We use both synthetic as well as real-world datasets to validate the proposed scheme.