We study the performance of composite pulses in the presence of time-varying control noise on a single qubit. These protocols, originally devised only to correct for static, systematic errors, are shown to be robust to time-dependent non-Markovian noise in the control field up to frequencies as high as of the Rabi frequency. Our study combines a generalized filter-function approach with asymptotic dc-limit calculations to give a simple analytic framework for error analysis applied to a number of composite-pulse sequences relevant to nuclear magnetic resonance as well as quantum information experiments. Results include examination of recently introduced concatenated composite pulses and dynamically corrected gates, demonstrating equivalent first-order suppression of time-dependent fluctuations in amplitude and/or detuning, as appropriate for the sequence in question. Our analytic results agree well with numerical simulations for realistic noise spectra with a roll-off to , providing independent validation of our theoretical insights.