Pioneering investigations referring to citrus flavor have been intensively conducted. However, the characteristic flavor difference between sweet orange and mandarin has not been defined. In this study, sensory analysis illustrated the crucial role of aroma in the differentiation between orange flavor and mandarin flavor. To study aroma, Valencia orange and LB8–9 mandarin were used. Their most aroma-active compounds were preliminarily identified by aroma extract dilution analysis (AEDA). Quantitation of key volatiles followed by calculation of odor activity values (OAVs) further detected potent components (OAV ≥ 1) impacting the overall aromatic profile of orange/mandarin. Follow-up aroma profile analysis revealed that ethyl butanoate, ethyl 2-methylbutanoate, octanal, decanal, and acetaldehyde were essential for orange-like aroma, whereas linalool, octanal, α-pinene, limonene, and (E,E)-2,4-decadienal were considered key components for mandarin-like aroma. Furthermore, an unreleased mandarin hybrid producing fruit with orange-like flavor was used to validate the identification of characteristic volatiles in orange-like aroma.