We present an approach for real-time tracking of a non-rigid target with a moving pan-tilt-zoom (PTZ) camera. The tracking of the object and control of the camera is handled by one computer in real time. The main contribution of the paper is method for target representation, localisation and detection, which takes into account both foreground and background properties, and is more discriminative than the common colour histogram based back-projection. A Bayesian hypothesis test is used to decide whether each pixel is occupied by the target or not. We show that this target representation is suitable for use with a Continuously Adaptive Mean Shift (CAMSHIFT) tracker. Experiments show that this leads to a tracking system that is efficient and accurate enough to guide a PTZ camera to follow a moving target in real time, despite the presence of background clutter and partial occlusion.