Improvement of the breeding efficiencies of heterotic crops adaptive to different conditions can mitigate the food shortage crisis due to overpopulation and climate change. To date, diverse molecular markers have been used to guide field phenotypic selection, whereas accurate predictions of complex heterotic traits are rarely reported. Here, we present a practical metabolome-based strategy for predicting yield heterosis in rice. The dissection of population structure based on untargeted metabolite profiles as the initial critical step in multivariate modeling performed better than the screening of predictive variables. Then the assessment of each predictive variable’s contribution to predictive models according to all latent factors was more precise than the conventional first one. Metabolites belonging to specific pathways were closely associated with yield heterosis, and the up-regulation of galactose metabolism promoted robust yield heterosis in hybrids under different growth conditions. Our study demonstrates that metabolome-based predictive models with correctly dissected population structure and screened predictive variables can facilitate accurate predictions of yield heterosis and have great potential for establishing molecular marker–based precision breeding programs.