It is demonstrated that a single, uniform long-period fiber grating (LPFG) working in the linear regime inherently behaves as an ultrafast optical temporal differentiator. Specifically, we show that the output temporal waveform in the core mode of a LPFG providing full energy coupling into the cladding mode is proportional to the first derivative of the optical temporal signal (e.g., optical pulse) launched at the input of the LPFG. Moreover, a LPFG providing full energy recoupling back from the cladding mode into the core mode inherently implements second-order temporal differentiation. Our numerical results have confirmed the feasibility of this simple, all-fiber approach to processing optical signals with temporal features in the picosecond and subpicosecond ranges.