The enormous chemical diversity and strain variability of prokaryotic protein glycosylation makes their large-scale exploration exceptionally challenging. Therefore, despite the universal relevance of protein glycosylation across all domains of life, the understanding of their biological significance and the evolutionary forces shaping oligosaccharide structures remains highly limited. Here, we report on a newly established mass binning glycoproteomics approach that establishes the chemical identity of the carbohydrate components and performs untargeted exploration of prokaryotic oligosaccharides from large-scale proteomics data directly. We demonstrate our approach by exploring an enrichment culture of the globally relevant anaerobic ammonium-oxidizing bacterium Ca. Kuenenia stuttgartiensis. By doing so we resolve a remarkable array of oligosaccharides, which are produced by two seemingly unrelated biosynthetic routes, and which modify the same surface-layer protein simultaneously. More intriguingly, the investigated strain also accomplished modulation of highly specialized sugars, supposedly in response to its energy metabolism—the anaerobic oxidation of ammonium—which depends on the acquisition of substrates of opposite charges. Ultimately, we provide a systematic approach for the compositional exploration of prokaryotic protein glycosylation, and reveal a remarkable example for the evolution of complex oligosaccharides in bacteria.