In this paper, we propose a novel method for extracting fiducial points (FPs) of electrocardiogram (ECG) signals. We propose the use of multi hidden Markov model (MultiHMM) as opposed to the traditional use of Classic HMM. In the MultiHMM method, each segment of an ECG beat is represented by a separate ergodic continuous density HMM. Each HMM has different state number and is trained separately. In the test step, the log-likelihood of two consecutive HMMs is compared and a path is estimated, which shows the correspondence of each part of the ECG signal to the HMM with the maximum log-likelihood. Fiducial points are estimated from the obtained path. For performance evaluation, the Physionet QT database and a Swine ECG database are used and the proposed method is compared with the Classic HMM and a method based on partially collapsed Gibbs sampler (PCGS). In our evaluation using the QT database, we also compare the results with low-pass differentiation, hybrid feature extraction algorithm, a method based on the wavelet transform and three HMM-based approaches. For the Swine database, the root mean square error (RMSE) values, across all FPs for MultiHMM, Classic HMM and PCGS methods are 13, 21 and 40 ms, respectively and the MultiHMM exhibits smaller error variability than other methods. For the QT database, RMSE values for MultiHMM, Classic HMM, Wavelet and PCGS methods are 10, 17, 26 and 38 ms, respectively. Our results demonstrate that our proposed MultiHMM approach outperforms other benchmark methods that exist in the literature; therefore can be used in practical ECG fiducial point extraction.