Black grouse (Tetrao tetrix) in Central Europe have undergone a severe contraction of their range in recent decades with only a few small isolated remaining populations. Here we compare genetic diversity of two contemporary isolated populations (Sallandse Heuvelrug, Netherlands and Lüneburger Heide, Germany) with historical samples from the same region collected within the last one hundred years. We use markers with both putatively neutral and functional variation to test whether the present small and highly fragmented populations hold lower genetic diversity compared to the former larger population. For this we applied three different types of genetic markers: nine microsatellites and 21 single nucleotide polymorphisms (SNPs), both sets which have been found to be neutral, and two functional major histocompatibility complex (MHC) genes for which there is evidence they are under selection. The contemporary small isolated populations displayed lower neutral genetic diversity compared to the corresponding historical samples. Furthermore, samples from Denmark showed that this now extinct population displayed lower genetic variation in the period immediately prior to the local extinction. Population structure was more pronounced among contemporary populations compared to historical populations for microsatellites and SNPs. This effect was not as distinct for MHC which is consistent with the possibility that MHC has been subjected to balancing selection in the past, a process which maintains genetic variation and may minimize population structure for such markers. Genetic differentiation among the present populations highlights the strong effects of population decline on the genetic structure of natural populations, which can be ultimately attributed to habitat loss following anthropogenic land use changes.