Detection of moving vehicles in wide area motion imagery (WAMI) is increasingly important, with promising applications in surveillance, traffic scene understanding and public service applications such as emergency evacuation and policy security. However, the large camera motion, along with low contrast between vehicles and backgrounds, makes detection a challenging task. In this paper, we propose a novel moving vehicle detection approach by embedding the scene context, which is a road network estimated online. A two-step framework is used in the work. First, with an initial vehicle detection, trajectories are achieved by vehicle tracking. Then, the road network is extracted and used to reduce false detections. Quantitative evaluation demonstrates that the proposed contextual model remarkably improves the detection performance.