In higher eukaryotes, histone methylation is involved in maintaining cellular identity during somatic development. As most nucleosomes are replaced by protamines during spermatogenesis, it is unclear whether histone modifications function in paternal transmission of epigenetic information. Here we show that two modifications important for Trithorax- and Polycomb-mediated gene regulation have methylation-specific distributions at regulatory regions in human spermatozoa. Histone H3 Lys4 dimethylation (H3K4me2) marks genes that are relevant in spermatogenesis and cellular homeostasis. In contrast, histone H3 Lys27 trimethylation (H3K27me3) marks developmental regulators in sperm, as in somatic cells. However, nucleosomes are only moderately retained at regulatory regions in human sperm. Nonetheless, genes with extensive H3K27me3 coverage around transcriptional start sites in particular tend not to be expressed during male and female gametogenesis or in preimplantation embryos. Promoters of orthologous genes are similarly modified in mouse spermatozoa. These data are compatible with a role for Polycomb in repressing somatic determinants across generations, potentially in a variegating manner.