Discriminating DDoS attack traffic from flash crowd through packet arrival patterns

Current DDoS attacks are carried out by attack tools, worms and botnets using different packet-transmission strategies and various forms of attack packets to beat defense systems. These problems lead to defense systems requiring various detection methods in order to identify attacks. Moreover, DDoS attacks can mix their traffics during flash crowds. By doing this, the complex defense system cannot detect the attack traffic in time. In this paper, we propose a behavior based detection that can discriminate DDoS attack traffic from traffic generated by real users. By using Pearson's correlation coefficient, our comparable detection methods can extract the repeatable features of the packet arrivals. The extensive simulations were tested for the accuracy of detection. We then performed experiments with several datasets and our results affirm that the proposed method can differentiate traffic of an attack source from legitimate traffic with a quick response. We also discuss approaches to improve our proposed methods at the conclusion of this paper.

CALD : surviving various application-layer DDoS attacks that mimic flash crowd

Distributed denial of service (DDoS) attack is a continuous critical threat to the Internet. Derived from the low layers, new application-layer-based DDoS attacks utilizing legitimate HTTP requests to overwhelm victim resources are more undetectable. The case may be more serious when suchattacks mimic or occur during the flash crowd event of a popular Website. In this paper, we present the design and implementation of CALD, an architectural extension to protect Web servers against various DDoS attacks that masquerade as flash crowds. CALD provides real-time detection using mess tests but is different from other systems that use resembling methods. First, CALD uses a front-end sensor to monitor thetraffic that may contain various DDoS attacks or flash crowds. Intense pulse in the traffic means possible existence of anomalies because this is the basic property of DDoS attacks and flash crowds. Once abnormal traffic is identified, the sensor sends ATTENTION signal to activate the attack detection module. Second, CALD dynamically records the average frequency of each source IP and check the total mess extent. Theoretically, the mess extent of DDoS attacks is larger than the one of flash crowds. Thus, with some parameters from the attack detection module, the filter is capable of letting the legitimate requests through but the attack traffic stopped. Third, CALD may divide the security modules away from the Web servers. As a result, it keeps maximum performance on the kernel web services, regardless of the harassment from DDoS. In the experiments, the records from www.sina.com and www.taobao.com have proved the value of CALD.

Fool me if you can: mimicking attacks and anti-attacks in cyberspace

Botnets have become major engines for malicious activities in cyberspace nowadays. To sustain their botnets and disguise their malicious actions, botnet owners are mimicking legitimate cyber behavior to fly under the radar. This poses a critical challenge in anomaly detection. In this paper, we use web browsing on popular web sites as an example to tackle this problem. First of all, we establish a semi-Markov model for browsing behavior. Based on this model, we find that it is impossible to detect mimicking attacks based on statistics if the number of active bots of the attacking botnet is sufficiently large (no less than the number of active legitimate users). However, we also find it is hard for botnet owners to satisfy the condition to carry out a mimicking attack most of the time. With this new finding, we conclude that mimicking attacks can be discriminated from genuine flash crowds using second order statistical metrics. We define a new fine correntropy metrics and show its effectiveness compared to others. Our real world data set experiments and simulations confirm our theoretical claims. Furthermore, the findings can be widely applied to similar situations in other research fields.

Distinguishing DDoS attacks from flash crowds using probability metrics

Both Flash crowds and DDoS (Distributed Denial-of-Service) attacks have very similar properties in terms of internet traffic, however Flash crowds are legitimate flows and DDoS attacks are illegitimate flows, and DDoS attacks have been a serious threat to internet security and stability. In this paper we propose a set of novel methods using probability metrics to distinguish DDoS attacks from Flash crowds effectively, and our simulations show that the proposed methods work well. In particular, these mathods can not only distinguish DDoS attacks from Flash crowds clearly, but also can distinguish the anomaly flow being DDoS attacks flow or being Flash crowd flow from Normal network flow effectively. Furthermore, we show our proposed hybrid probability metrics can greatly reduce both false positive and false negative rates in detection.

Distributed denial of service (DDoS) detection by traffic pattern analysis

In this paper, we propose a behavior-based detection that can discriminate Distributed Denial of Service (DDoS) attack traffic from legitimated traffic regardless to various types of the attack packets and methods. Current DDoS attacks are carried out by attack tools, worms and botnets using different packet-transmission rates and packet forms to beat defense systems. These various attack strategies lead to defense systems requiring various detection methods in order to identify the attacks. Moreover, DDoS attacks can craft the traffics like flash crowd events and fly under the radar through the victim. We notice that DDoS attacks have features of repeatable patterns which are different from legitimate flash crowd traffics. In this paper, we propose a comparable detection methods based on the Pearson’s correlation coefficient. Our methods can extract the repeatable features from the packet arrivals in the DDoS traffics but not in flash crowd traffics. The extensive simulations were tested for the optimization of the detection methods. We then performed experiments with several datasets and our results affirm that the proposed methods can differentiate DDoS attacks from legitimate traffics.

Time correlated anomaly detection based on inferences

Anomaly detection techniques are used to find the presence of anomalous activities in a network by comparing traffic data activities against a "normal" baseline. Although it has several advantages which include detection of "zero-day" attacks, the question surrounding absolute definition of systems deviations from its "normal" behaviour is important to reduce the number of false positives in the system. This study proposes a novel multi-agent network-based framework known as Statistical model for Correlation and Detection (SCoDe), an anomaly detection framework that looks for timecorrelated anomalies by leveraging statistical properties of a large network, monitoring the rate of events occurrence based on their intensity. SCoDe is an instantaneous learning-based anomaly detector, practically shifting away from the conventional technique of having a training phase prior to detection. It does acquire its training using the improved extension of Exponential Weighted Moving Average (EWMA) which is proposed in this study. SCoDe does not require any previous knowledge of the network traffic, or network administrators chosen reference window as normal but effectively builds upon the statistical properties from different attributes of the network traffic, to correlate undesirable deviations in order to identify abnormal patterns. The approach is generic as it can be easily modified to fit particular types of problems, with a predefined attribute, and it is highly robust because of the proposed statistical approach. The proposed framework was targeted to detect attacks that increase the number of activities on the network server, examples which include Distributed Denial of Service (DDoS) and, flood and flash-crowd events. This paper provides a mathematical foundation for SCoDe, describing the specific implementation and testing of the approach based on a network log file generated from the cyber range simulation experiment of the industrial partner of this project.

Detection and defense of application-layer DDoS attacks in backbone web traffic

Web servers are usually located in a well-organized data center where these servers connect with the outside Internet directly through backbones. Meanwhile, the application-layer distributed denials of service (AL-DDoS) attacks are critical threats to the Internet, particularly to those business web servers. Currently, there are some methods designed to handle the AL-DDoS attacks, but most of them cannot be used in heavy backbones. In this paper, we propose a new method to detect AL-DDoS attacks. Our work distinguishes itself from previous methods by considering AL-DDoS attack detection in heavy backbone traffic. Besides, the detection of AL-DDoS attacks is easily misled by flash crowd traffic. In order to overcome this problem, our proposed method constructs a Real-time Frequency Vector (RFV) and real-timely characterizes the traffic as a set of models. By examining the entropy of AL-DDoS attacks and flash crowds, these models can be used to recognize the real AL-DDoS attacks. We integrate the above detection principles into a modularized defense architecture, which consists of a head-end sensor, a detection module and a traffic filter. With a swift AL-DDoS detection speed, the filter is capable of letting the legitimate requests through but the attack traffic is stopped. In the experiment, we adopt certain episodes of real traffic from Sina and Taobao to evaluate our AL-DDoS detection method and architecture. Compared with previous methods, the results show that our approach is very effective in defending AL-DDoS attacks at backbones. © 2013 Elsevier B.V. All rights reserved.

Parametric differences between a real-world distributed denial-of-service attack and a flash event

Distributed Denial-of-Service (DDoS) attacks continue to be one of the most pernicious threats to the delivery of services over the Internet. Not only are DDoS attacks present in many guises, they are also continuously evolving as new vulnerabilities are exploited. Hence accurate detection of these attacks still remains a challenging problem and a necessity for ensuring high-end network security. An intrinsic challenge in addressing this problem is to effectively distinguish these Denial-of-Service attacks from similar looking Flash Events (FEs) created by legitimate clients. A considerable overlap between the general characteristics of FEs and DDoS attacks makes it difficult to precisely separate these two classes of Internet activity. In this paper we propose parameters which can be used to explicitly distinguish FEs from DDoS attacks and analyse two real-world publicly available datasets to validate our proposal. Our analysis shows that even though FEs appear very similar to DDoS attacks, there are several subtle dissimilarities which can be exploited to separate these two classes of events.

Detecting distributed Denial-of-Service attacks and Flash Events

This thesis investigates and develops techniques for accurately detecting Internet-based Distributed Denial-of-Service (DDoS) Attacks where an adversary harnesses the power of thousands of compromised machines to disrupt the normal operations of a Web-service provider, resulting in significant down-time and financial losses. This thesis also develops methods to differentiate these attacks from similar-looking benign surges in web-traffic known as Flash Events (FEs). This thesis also addresses an intrinsic challenge in research associated with DDoS attacks, namely, the extreme scarcity of public domain datasets (due to legal and privacy issues) by developing techniques to realistically emulate DDoS attack and FE traffic.

Discriminating DDoS flows from flash crowds using information distance

Discriminating DDoS flooding attacks from flash crowds poses a tough challenge for the network security community. Because of the vulnerability of the original design of the Internet, attackers can easily mimic the patterns of legitimate network traffic to fly under the radar. The existing fingerprint or feature based algorithms are incapable to detect new attack strategies. In this paper, we aim to differentiate DDoS attack flows from flash crowds. We are motivated by the following fact: the attack flows are generated by the same prebuilt program (attack tools), however, flash crowds come from randomly distributed users all over the Internet. Therefore, the flow similarity among DDoS attack flows is much stronger than that among flash crowds. We employ abstract distance metrics, the Jeffrey distance, the Sibson distance, and the Hellinger distance to measure the similarity among flows to achieve our goal. We compared the three metrics and found that the Sibson distance is the most suitable one for our purpose. We apply our algorithm to the real datasets and the results indicate that the proposed algorithm can differentiate them with an accuracy around 65%.

Discriminating DDoS attacks from flash crowds using flow correlation coefficient

Distributed Denial of Service (DDoS) attack is a critical threat to the Internet, and botnets are usually the engines behind them. Sophisticated botmasters attempt to disable detectors by mimicking the traffic patterns of flash crowds. This poses a critical challenge to those who defend against DDoS attacks. In our deep study of the size and organization of current botnets, we found that the current attack flows are usually more similar to each other compared to the flows of flash crowds. Based on this, we proposed a discrimination algorithm using the flow correlation coefficient as a similarity metric among suspicious flows. We formulated the problem, and presented theoretical proofs for the feasibility of the proposed discrimination method in theory. Our extensive experiments confirmed the theoretical analysis and demonstrated the effectiveness of the proposed method in practice.

DDoS attack detection at local area networks using information theoretical metrics

DDoS attacks are one of the major threats to Internet services. Sophisticated hackers are mimicking the features of legitimate network events, such as flash crowds, to fly under the radar. This poses great challenges to detect DDoS attacks. In this paper, we propose an attack feature independent DDoS flooding attack detection method at local area networks. We employ flow entropy on local area network routers to supervise the network traffic and raise potential DDoS flooding attack alarms when the flow entropy drops significantly in a short period of time. Furthermore, information distance is employed to differentiate DDoS attacks from flash crowds. In general, the attack traffic of one DDoS flooding attack session is generated by many bots from one botnet, and all of these bots are executing the same attack program. As a result, the similarity among attack traffic should higher than that among flash crowds, which are generated by many random users. Mathematical models have been established for the proposed detection strategies. Analysis based on the models indicates that the proposed methods can raise the alarm for potential DDoS flooding attacks and can differentiate DDoS flooding attacks from flash crowds with conditions. The extensive experiments and simulations confirmed the effectiveness of our proposed detection strategies.