The rendition of medical images influences the accuracy and precision of quantifications. Image variations or biases make measuring imaging biomarkers challenging. The objective of this paper is to reduce the variability of computed tomography (CT) quantifications for radiomics and biomarkers using physics-based deep neural networks (DNNs). The proposed framework used the knowledge of ground truth through a virtual imaging trial (VIT) methodology to harmonize the different renditions of CT scans across variations in reconstruction kernel and dose. This harmonization was done by developing a generative adversarial network (GAN) model, informed by pixel values and patient-based modulation transfer function (MTF) estimates. To train the network, the VIT platform was used to acquire CT images from a set of forty computational human models (XCAT). Models included varying levels of pulmonary diseases including lung nodules and emphysema. The patient models were imaged with a validated CT simulator (DukeSim) modeling a commercial CT scanner operating across a range of dose (20-100 mAs) and reconstruction kernels (15 from smooth to sharp). The harmonized virtual images were evaluated in three different ways: 1) visual assessment of the images, 2) bias and variation in density-based biomarkers, and 3) bias and variation in morphological-based biomarkers. The harmonization improved the bias and variability of the test set images yielding a structural similarity index of 95±1%, a normalized mean squared error of 10.2±1.5%, and a peak signal-to-noise ratio of 31.8±1.5 dB compared to variations in these metrics when no harmonization was applied (87±9%, 14.2±4%, and 29.6±2, respectively). Emphysema-based imaging biomarkers of LAA-950 (-1.5±1.8%), Perc15 (13.65±9.3 HU), and Lung mass (0.1±0.3 g) had more precise and consistent quantifications compared to values when images were not harmonized (9.9±12%, -36.0±53 HU, and 0.2±0.4g, respectively). These results suggest that the method can be promising to improve consistency in multi-center studies when reliable and consistent quantification of data from multiple systems are required.