The Internet of Things (IoT) has seen significant growth in recent years, impacting various sectors such as smart cities, healthcare, and transportation. However, IoT networks face significant security challenges, particularly from botnets that perform DDoS attacks. Traditional centralized intrusion detection systems struggle with the large traffic volumes in IoT environments. This study proposes a decentralized approach using a fog computing layer with a reptile group intelligence algorithm to reduce network traffic size, followed by analysis in the cloud layer using Apache Spark architecture. Key network traffic features are selected using a chameleon optimization algorithm and a principal component reduction method. Multi-layer artificial neural networks are employed for traffic analysis in the fog layer. Experiments on the NSL-KDD dataset indicate that the proposed method achieves up to 99.65% accuracy in intrusion detection. Additionally, the model outperforms other deep and combined learning methods, such as Bi-LSTM, CNN-BiLSTM, SVM-RBF, and SAE-SVM-RBF, in attack detection. Implementation of decision tree, random forest, and support vector machine algorithms in the cloud layer also demonstrates high accuracy rates of 96.27%, 98.34%, and 96.12%, respectively.