The use of electroencephalogram (EEG) data for sleep scoring is critical for the detection and treatment of sleep disorders. However, accurately classifying different sleep stages based on raw EEG signals is difficult due to variability in sleep patterns’ temporal and frequency scales, as well as similarities between sleep stages. We present a supervised contrastive learning model for feature extraction that minimizes intra-class distances while maximizing inter-class distances in this work. In addition, to increase classification performance, we employ ensemble learning multi classifiers such as Random Forest, Logistic Regression, and AdaBoost. Label bias and unstable model performance during repetitive training, on the other hand, are persistent issues in sleep scoring. To address these concerns, we propose a selective batch sampling strategy and self-knowledge distillation to improve model stability during training and extract learning features that are resistant to label bias. Overall, our proposed model provides an interpretable and adaptive representation of EEG data that may be used to accurately classify sleep stages.