Anomaly-based DoS detection and prevention in SIP networks by modeling SIP normal traffic

Due to the various features of Voice over Internet Protocol (VoIP), this technology has attracted the attention of many users in comparison with the traditional telephony system. However, with the growth of this technology, the security issues and protection of its users against different kinds of threats have been raised as an essential requirement. Session Initiation Protocol is a predominant protocol to initiate and terminate multimedia sessions in VoIP networks that provide simplicity and text-based features. Despite its mentioned advantages, these features impose several vulnerabilities on VoIP networks. Denial of Service attack, as one of the most common attacks against VoIP networks, is also a noted security issue in the Internet Protocol platforms. In such attacks, the attacker tries to prevent service from authorized users by consuming server resources. These attacks can be launched by sending out-of-sequence messages, malformed messages, and flooding different kinds of messages. In this study, a new anomaly-based method is presented for detection and prevention of these attacks. Normal traffic of a VoIP network is modeled by making a finite state machine, which is used for attack detection besides other proposed modules. Furthermore, a whitelist method is implemented using Bloom data structure for attack prevention. The proposed method is completely implemented and tested using different test scenarios. The obtained results show that by using proposed method, attacks can be detected more accurately with lower false ratios and delay in comparison with the existing methods.