In this paper, we propose a new type of adaptive fuzzy inference system with a view to achieve improved performance for forecasting nonlinear time series data by dynamically adapting the fuzzy rules with arrival of new data. The structure of the fuzzy model utilized in the proposed system is developed based on the log-likelihood value of each data vector generated by a trained Hidden Markov Model. As part of its adaptation process, our system checks and computes the parameter values and generates new fuzzy rules as required, in response to new observations for obtaining better performance. In addition, it can also identify the most appropriate fuzzy rule in the system that covers the new data; and thus requires to adapt the parameters of the corresponding rule only, while keeping the rest of the model unchanged. This intelligent adaptive behavior enables our adaptive fuzzy inference system (FIS) to outperform standard FISs. We evaluate the performance of the proposed approach for forecasting stock price indices. The experimental results demonstrate that our approach can predict a number of stock indices, e.g., Dow Jones Industrial (DJI) index, NASDAQ index, Standard and Poor500 (S&P500) index and few other indices from UK (FTSE100), Germany (DAX) , Australia (AORD) and Japan (NIKKEI) stock markets, accurately compared with other existing computational and statistical methods.