Breast cancer is the most common cancer among women worldwide and ranks second in terms of overall cancer deaths. One of the difficulties associated with treating breast cancer is that it is a heterogeneous disease with variations in benign and pathologic tissue composition, which contributes to disease development, progression, and treatment response. Many of these phenotypes are uncharacterized and their presence is difficult to detect, in part due to the sparsity of methods to correlate information between the cellular microscale and the whole-breast macroscale. Quantitative multiscale imaging of the breast is an emerging field concerned with the development of imaging technology that can characterize anatomic, functional, and molecular information across different resolutions and fields of view. It involves a diverse collection of imaging modalities, which touch large sections of the breast imaging research community. Prospective studies have shown promising results, but there are several challenges, ranging from basic physics and engineering to data processing and quantification, that must be met to bring the field to maturity. This paper presents some of the challenges that investigators face, reviews currently used multiscale imaging methods for preclinical imaging, and discusses the potential of these methods for clinical breast imaging.