Dual-energy CT (DECT) was introduced to address the inability of conventional single-energy computed tomography (SECT) to distinguish materials with similar absorbances but different elemental compositions. However, material decomposition algorithms based purely on the physics of the underlying attenuation process have several limitations, leading to low signal-to-noise ratio (SNR) in the derived material-specific images. To overcome these, we trained a convolutional neural network (CNN) to develop a framework to reconstruct non-contrast SECT images from DECT scans. We show that the traditional physics-based decomposition algorithms do not bring to bear the full information content of the image data. A CNN that leverages the underlying physics of the DECT image generation process as well as the anatomic information gleaned via training with actual images can generate higher fidelity processed DECT images.