Feature matching is critical in analyzing remote sensing images, aiming to find the optimal mapping between correspondences. Regularization technology is essential to ensure the well-posedness of feature matching. However, current regularization-based methods scarcely consider the geometry structure of the image, which is beneficial for estimating the mapping, especially when the image pairs have a large view or scale change and local distortion. In this article, we introduce manifold regularization to overcome this limit and formulate feature matching as a unified semisupervised latent variable mixture model for both rigid and nonrigid transformations. Especially, we apply a Bayesian model with latent variables indicating whether matches in the putative correspondences are outliers or inliers. Moreover, we employ all the feature points, only part of which have correct matches, to express the intrinsic structure, which is preserved by manifold regularization. Finally, we combine manifold regularization with three different transformation models (e.g., rigid, affine, and thin-plate spline) to estimate the corresponding mappings. Experimental results on four remote sensing image datasets demonstrate that our method can significantly outperform the state of the art.