In this paper we describe the utilisation of the diversity of different feature representations for speaker, emotion and language verification. The underlying principle behind the method is that some features are better at discriminating some classes and other features for other classes. Studies are done on four features and their combinations. An information theoretic procedure is described which can be used to determine the feature most suitable for a given class. Based on this, we propose the method of feature-switching, which uses different features for verifying different classes. Experiments show that feature-switching improves the accuracy for all three tasks.