The number of zero-day attacks has been exponentially increasing due to a variety of heterogeneous Internet of Things (IoT) protocols. Many centralized-based techniques have been introduced to identify malicious activities in IoT environments. However, these techniques have suffered including a high latency to satisfy IoT requirements. This paper proposes a fog-based attack detection (FBAD) framework using an ensemble of online sequential extreme learning machine (EOS-ELM) for efficiently detecting malicious activities. We indicate a high-level view of the efficient proposed framework for deploying the distributed attack detection technique in fog computing due to high accuracy and low latency, as it is closer to the IoT devices at the network edge. Furthermore, we compare the performance of our framework with other existing approaches including ELM and OS-ELM. The results of the experiment demonstrate that distributed architecture outperforms centralized architecture in terms of the detection time and classification accuracy.