The concept of synthetic dimensions in photonics provides a versatile platform in exploring multi-dimensional physics. Many of these physics are characterized by band structures in more than one dimensions. Existing efforts on band structure measurements in the photonic synthetic frequency dimension however are limited to either one-dimensional Brillouin zones or one-dimensional subsets of multi-dimensional Brillouin zones. Here we theoretically propose and experimentally demonstrate a method to fully measure multi-dimensional band structures in the synthetic frequency dimension. We use a single photonic resonator under dynamical modulation to create a multi-dimensional synthetic frequency lattice. We show that the band structure of such a lattice over the entire multi-dimensional Brillouin zone can be measured by introducing a gauge potential into the lattice Hamiltonian. Using this method, we perform experimental measurements of two-dimensional band structures of a Hermitian and a non-Hermitian Hamiltonian. The measurements reveal some of the general properties of point-gap topology of the non-Hermitian Hamiltonian in more than one dimensions. Our results demonstrate experimental capabilities to fully characterize high-dimensional physical phenomena in the photonic synthetic frequency dimension.