The current emphasis on comparative effectiveness research will provide practicing physicians with increasing volumes of observational evidence about preventive care. However, numerous highly publicized observational studies of the effect of prevention on health outcomes have reported exaggerated relationships that were later contradicted by randomized controlled trials. A growing body of research has identified sources of bias in observational studies that are related to patient behaviors or underlying patient characteristics, known as the healthy user effect, the healthy adherer effect, confounding by functional status or cognitive impairment, and confounding by selective prescribing. In this manuscript we briefly review observational studies of prevention that have appeared to reach incorrect conclusions. We then describe potential sources of bias in these studies and discuss study designs, analytical methods, and sensitivity analyses that may mitigate bias or increase confidence in the results reported. More careful consideration of these sources of bias and study designs by providers can enhance evidence-based decision-making.