In this paper, we aim to improve the convolutional deep learning by means of proposed statistical feature image descriptor and Laplacian of Gaussian filtering. We propose a statistical feature image descriptor (SFID) that is composed of spatial and temporal parts for video content classification using convolutional deep neural network. We apply the proposed descriptor to multi-view action unit classification. The SFID is a statistical representation of the raw image based upon K-abstraction levels. It is capable of addressing the fixing input array size of the deep learning model. Further, it eliminates redundancy in representation of the content; hence, it reduces computation cost. The proposed SFID can be in spatial and/or temporal form. The temporal form is particularly important in video content classification. We added a new layer of Laplacian of Gaussian filter (LoG) right before fully connected layer into the regular deep convolutional neural network (DCNN) structure. The parameters of the LoG are adaptively calculated using the Gaussian mixture models. The classification results are compared with regular DCNN, SVM models, and KNN together with feature descriptors of SIFT and SURF. The results show that the proposed feature descriptor and introducing a LoG filter layer give promising performance for deep learning.