In this paper, we propose a novel ensemble margin based semi-supervised random forest (EMRF) algorithm for the classification of the hyperspectral image with limited training data. The proposed method tries to improve the effectiveness of the ensemble model via adaptively labeling the unlabeled instances with high classification probability then adding them into the training set. The classification probability of a training instance is reflected by the unsupervised margin value of this instance. The higher ensemble margin of an instance, the higher probability the instance being classified correctly and added into to the training set in the next iteration.