We study the persistence of a geometrically frustrated local order inside partially crystallized packings of equal-sized spheres. Measurements by x-ray tomography reveal previously unseen grain scale rearrangements occurring inside large three-dimensional packings as they crystallize. Three successive structural transitions are detected by a statistical description of the local volume fluctuations. These compaction regimes are related to the disappearance of densely packed tetrahedral patterns of beads. Amorphous packings of monodisperse spheres are saturated with these tetrahedral clusters at Bernal’s limiting density (). But, no periodic lattice can be built upon these patterns; they are geometrically frustrated and are thus condemned to vanish while the crystallization occurs. Remarkably, crystallization-induced grain rearrangements can be interpreted in terms of the evolution of key topological features of these aggregates.