Autoregression (AR) is a tool commonly used to understand and predict time series data. Traditionally the excitation noise is modelled as a Gaussian. However, real-world data may not be Gaussian in nature, and it is known that Gaussian models are adversely affected by the presence of outliers. We introduce a Bayesian AR model in which the excitation noise is assumed to be Student-t distributed. Variational Bayesian approximations to the posterior distributions of the model parameters are used to overcome the intractable integrations inherent in the Bayesian model. Independent automatic relevance determination (ARD) priors over each of the AR coefficients are used to estimate the model order. Using synthetic data, we show that the Student-t model performs well against both Gaussian and leptokurtic data, in terms of parameter estimation (including the model order) and is much more robust to outliers than either Gaussian or finite mixtures of Gaussian models. We apply the model to strongly leptokurtic EEG signals and show that the Student-t model makes more accurate one-step-ahead predictions than the Gaussian model and provides more consistent estimates of the AR coefficients over simultaneously recorded EEG channels.