A systemic analysis of the tumor-immune interactions within the heterogeneous tumor microenvironment is of particular importance for understanding the antitumor immune response. We used multiplexed immunofluorescence to elucidate cellular spatial interactions and T-cell infiltrations in metastatic melanoma tumor microenvironment. We developed two novel computational approaches that enable infiltration clustering and single cell analysis—cell aggregate algorithm and cell neighborhood analysis algorithm—to reveal and to compare the spatial distribution of various immune cells relative to tumor cell in sub-anatomic tumor microenvironment areas. We showed that the heterogeneous tumor human leukocyte antigen-1 expressions differently affect the magnitude of cytotoxic T-cell infiltration and the distributions of CD20+ B cells and CD4+FOXP3+ regulatory T cells within and outside of T-cell infiltrated tumor areas. In a cohort of 166 stage III melanoma samples, high tumor human leukocyte antigen-1 expression is required but not sufficient for high T-cell infiltration, with significantly improved overall survival. Our results demonstrate that tumor cells with heterogeneous properties are associated with differential but predictable distributions of immune cells within heterogeneous tumor microenvironment with various biological features and impacts on clinical outcomes. It establishes tools necessary for systematic analysis of the tumor microenvironment, allowing the elucidation of the “homogeneous patterns” within the heterogeneous tumor microenvironment.