Habitual declines in sleep duration and increased rates of obesity are public health concerns worldwide. Accumulating evidence suggests a prominent link between reduced sleep duration and weight gain. Our cross-sectional study investigated the relationship between sleep duration and body fat distribution in US adults. We extracted data for 5151 participants (2575 men and 2576 women) aged 18–59 years from the US National Health and Nutrition Examination Survey 2011–2012 and 2013–2014. Weekday or workday night-time sleep duration was estimated using an in-home interview questionnaire. Dual-energy x-ray absorptiometry scans were used to determine regional body fat mass (arms, legs, trunk [android and gynoid], and abdominal [subcutaneous and visceral]). Multiple linear regression and restricted cubic spline analyses were performed after adjusting for several demographic, anthropometric, and nutritional covariates. There was a significant negative association between sleep duration and visceral fat mass overall (β: −12.139, P < 0.001) and by sex (men: β: −10.096, P < 0.001; women: β: −11.545, P = 0.038), after adjusting for age, ethnicity, body mass index, total body fat mass, daily energy and alcohol intake, sleep quality and sleep disorder status. Sleep duration and visceral fat appeared to plateau at ≥ 8 h of daily sleep. Sleep duration is negatively associated with visceral fat mass accumulation during adulthood with possibly no benefits beyond 8 h of sleep per day. Mechanistic and prospective studies are required to confirm the effect of sleep duration on visceral adiposity and determine its causes.