The use of high-energy (> 8 MeV) LINear ACcelerators (LINACs) for medical cancer treatments causes the photoproduction of secondary neutrons, whose unwanted dose to the patient has to be calculated. The characterization of the neutron spectra is necessary to allow the dosimetric evaluation of the neutron beam contamination. The neutron spectrum in a hospital environment is usually measured with integrating detectors such as bubble dosimeters, Thermo Luminescent Dosimeters (TLDs) or Bonner Spheres, which integrate the information over a time interval and an energy one. This paper presents the development of a neutron spectrometer based on the Time of Flight (ToF) technique in order to perform a real time characterization of the neutron contamination. The detector measures the neutron spectrum exploiting the fact that the LINAC beams are pulsed and arranged in bunches with a rate of 100–300 Hz depending on the beam type and energy. The detector consists of boron loaded scintillating fibers readout by a MultiAnode PhotoMultiplier Tube (MAPMT). A detailed description of the detector and the acquisition system together with the results in terms of ToF spectra and number of neutrons with a Varian Clinac iX are presented.