Supercritical CO₂ (sCO₂) Brayton cycle is a promising technology for coal-fired power generation with high efficiency and compact equipment size. However, its sophisticated construct and high-temperature waste heat rejection require a systematic design to maximize its performance. Herein, we develop a combined sCO₂ Brayton cycle-organic Rankine cycle (ORC) design for coal-fired power plant. A novel glass-box model that considers the specific designs of sCO₂ boiler, recuperators, coolers, and turbomachinery is formulated to optimize the power plant. A high-accuracy artificial neural network model is also developed to estimate the system’s pressure drop to reduce model complexity. As a result, the glass-box model is reformulated into a grey-box model. The model is applied to three different combined cycles’ design problem to evaluate their performance. Result shows that the grey-box model saves more than 50% of CPU time. With the turbine inlet at 620 °C/25 MPa and the main compressor inlet at 35 °C/7.38 MPa, the proposed combined cycle reaches a thermal efficiency of 45.73%, thereby achieving a 2.75 percentage point improvement compared with the standalone design. Sensitivity analysis is also carried out to evaluate the effects of ORC working fluid, flue gas temperature at the cooling wall outlet, main compressor inlet pressure, and evaporating temperature of ORC, on the system’s performance.