With a significant increase in air traffic over the years, the frequency of potential conflicts between aircraft has increased, which in turn has proportionally increased the workload of Air Traffic Control Operators (ATCOs). This has increased the importance of the role of automation tools in assisting the ATCOs. Deep Reinforcement Learning (RL) is a very promising way to address the problem of resolving conflicts in air traffic and find optimum alternate flight paths. Existing research on using RL in air traffic conflict resolution is skewed towards aircraft speed control which is easier to obtain optimum solutions from compared to control of other aircraft parameters. However depending upon airspace sector size, this may not always be feasible. To address this research gap, the current research proposes a deep reinforcement learning architecture for air traffic conflict resolution in a structured airspace, using heading vector control. Results from the architecture shows that the ownship aircraft achieves safe separation of 5 nautical miles for 100% of the conflicts in the designed scenarios. Further analysis reveals that the RL agent achieves promising results in the presence of surrounding aircraft as well as in the event of a potential secondary conflict. The RL agent is capable of guiding the ownship back to its original path once conflicts are resolved. The results also demonstrate that our model is able to provide safe and efficient resolution to the generated air traffic conflicts under uncertainties associated with the speed and cross-track deviation of the intruder aircraft.