We incorporate dielectric indium tin oxide nanocrystals into the hot-spot of gold nanogap-antennas and perform third harmonic spectroscopy on these hybrid nanostructure arrays. The combined system shows a 2-fold increase of the radiated third harmonic intensity when compared to bare gold antennas. In order to identify the origin of the enhanced nonlinear response we perform finite element simulations of the nanostructures, which are in excellent agreement with our measurements. We find that the third harmonic signal enhancement is mainly related to changes in the linear optical properties of the plasmonic antenna resonances when the ITO nanocrystals are incorporated. Furthermore, the dominant source of the third harmonic is found to be located in the gold volume of the plasmonic antennas.