In recent years, deep learning methods have been widely used for the classification of hyperspectral images (HSIs). However, the training of deep models is very time-consuming. In addition, the rare labeled samples of remote sensing images also limit the classification performance of deep models. In this letter, a simple deep learning model, a rotation-based deep forest (RBDF), is proposed for the classification of HSIs. Specifically, the output probability of each layer is used as the supplement feature of the next layer. The rotation forest is used to increase the discriminative power of spectral features and neighboring pixels are used to introduce spatial information. The RBDF consumes much less training time than traditional deep models. Experimental results based on three HSIs demonstrate that the proposed method achieves the state-of-the-art classification performance. In addition, the RBDF obtains satisfied classification results with very few training samples.