Convolutional neural network has unique superiority in images processing, it can effectively extract features and reduce data dimensions by convolution and pooling. But it takes the “violent segmentation” method in the process of pooling. This method cannot guarantee the final selected pixel value can be a good representative of the partial features, neither average pooling nor max pooling. Therefore, three pooling methods based on super-pixel segmentation are proposed, called “super-pixel average pooling”, “super-pixel max pooling” and “super-pixel smooth pooling”. Firstly, the super-pixel segmentation is performed on the feature images, and then the value of the point which has the smoothest gradient in each super pixel is selected to represent the feature of the local area. Compared to the violent segmentation pooling operation, this method exhibits more stable characterization ability, it can retain image features perfectly while reduce the data dimensions. Experiments show that the improved convolutional neural network achieved better results than normal algorithm in the standard data sets.