To study cross-sectional and longitudinal associations between objectively assessed neighborhood walkability, walking difficulties, and participation in leisure activities among older people. Self-reported 2 km walking difficulty (intact, modifications, difficulties) at baseline and participating in organized group, outdoor recreation and cultural activities at baseline and follow-up were studied in community-dwelling persons (N = 848) aged 75–90. A walkability index, calculated using a geographic information system, was categorized into tertiles (lowest, middle, highest). Residence in the highest walkability areas was associated with higher participation in cultural activities and lower participation in outdoor recreation, while the latter was most frequently reported by residents in the lowest walkability areas. Those reporting no difficulties were more likely than those reporting difficulties to participate in all studied activities. Residence in the middle or highest walkability areas predicted higher participation in cultural activities at follow-up. Older persons activity profiles associate with neighborhood walkability and walking difficulties.