In this paper, a hierarchical structure based convolutional neural network is proposed to provide the ability for robust information processing. The weight sharing ability of convolutional neural networks (CNNs) is considered as a level of hierarchy in these networks. Weight sharing reduces the number of free parameters and improves the generalization ability. In the proposed structure, a small CNN which is used for feature extractor is shared between the whole input image pixels. A scalable architecture for implementing extensive CNNs is resulted using a smaller and modularized trainable network to solve a large and complicated task. The proposed structure causes less training time, fewer numbers of parameters and higher test data accuracy. The recognition accuracy for recognizing unseen data shows improvement in generalization. Also presented are application examples for face recognition. The comprehensive experiments completed on ORL, Yale and JAFFE face databases show improved classification rates and reduced training time and network parameters.