Optimal control of systems with complex nonlinear behaviour such as steady state multiplicity results in a nonlinear optimization problem that needs to be solved online at each sample time. We present an approach based on simulation, function approximation and evolutionary improvement aimed towards simplifying online optimization. Closed loop data from a suboptimal control law, such as MPC based on successive linearization, are used to obtain an approximation of the ‘cost‐to‐go’ function, which is subsequently improved through iterations of the Bellman equation. Using this offline‐computed cost approximation, an infinite horizon problem is converted to an equivalent single stage problem—substantially reducing the computational burden. This approach is tested on continuous culture of microbes growing on a nutrient medium containing two substrates that exhibits steady state multiplicity. Extrapolation of the cost‐to‐go function approximator can lead to deterioration of online performance. Some remedies to prevent such problems caused by extrapolation are proposed. Copyright © 2003 John Wiley & Sons, Ltd.