Mislabeled training data is a challenge to face in order to build a robust classifier whether it is an ensemble or not. This work handles the mislabeling problem by exploiting four different ensemble margins for identifying, then eliminating or correcting the mislabeled training data. Our approach is based on class noise ordering and relies on the margin values of misclassified data. The effectiveness of our ordering-based class noise removal and correction methods is demonstrated in performing image classification. A comparative analysis is conducted with respect to the majority vote filter, a reference ensemble-based class noise filter.