The dynamical processes associated with electric field manipulation of the polarization in a ferroelectric remain largely unknown but fundamentally determine the speed and functionality of ferroelectric materials and devices. Here we apply subpicosecond duration, single-cycle terahertz pulses as an ultrafast electric field bias to prototypical ferroelectric thin films with the atomic-scale response probed by femtosecond x-ray-scattering techniques. We show that electric fields applied perpendicular to the ferroelectric polarization drive large-amplitude displacements of the titanium atoms along the ferroelectric polarization axis, comparable to that of the built-in displacements associated with the intrinsic polarization and incoherent across unit cells. This effect is associated with a dynamic rotation of the ferroelectric polarization switching on and then off on picosecond time scales. These transient polarization modulations are followed by long-lived vibrational heating effects driven by resonant excitation of the ferroelectric soft mode, as reflected in changes in the -axis tetragonality. The ultrafast structural characterization described here enables a direct comparison with first-principles-based molecular-dynamics simulations, with good agreement obtained.