Markov state models (MSMs), which model conformational dynamics as a network of transitions between metastable states, have been increasingly used to model the thermodynamics and kinetics of biomolecules. In considering perturbations to molecular dynamics induced by sequence mutations, chemical modifications, or changes in external conditions, it is important to assess how transition rates change, independent of changes in metastable state definitions. Here, we present a surprisal metric to quantify the difference in metastable state transitions for two closely related MSMs, taking into account the statistical uncertainty in observed transition counts. We show that the surprisal is a relative entropy metric closely related to the Jensen–Shannon divergence between two MSMs, which can be used to identify conformational states most affected by perturbations. As examples, we apply the surprisal metric to a two-dimensional lattice model of a protein hairpin with mutations to hydrophobic residues, all-atom simulations of the Fs peptide α-helix with a salt-bridge mutation, and a comparison of protein G β-hairpin with its trpzip4 variant. Moreover, we show that surprisal-based adaptive sampling is an efficient strategy to reduce the statistical uncertainty in the Jensen–Shannon divergence, which could be a useful strategy for molecular simulation-based ab initio design.