A novel semi-supervised approach for network traffic clustering

Network traffic classification is an essential component for network management and security systems. To address the limitations of traditional port-based and payload-based methods, recent studies have been focusing on alternative approaches. One promising direction is applying machine learning techniques to classify traffic flows based on packet and flow level statistics. In particular, previous papers have illustrated that clustering can achieve high accuracy and discover unknown application classes. In this work, we present a novel semi-supervised learning method using constrained clustering algorithms. The motivation is that in network domain a lot of background information is available in addition to the data instances themselves. For example, we might know that flow ƒ1 and ƒ2 are using the same application protocol because they are visiting the same host address at the same port simultaneously. In this case, ƒ1 and ƒ2 shall be grouped into the same cluster ideally. Therefore, we describe these correlations in the form of pair-wise must-link constraints and incorporate them in the process of clustering. We have applied three constrained variants of the K-Means algorithm, which perform hard or soft constraint satisfaction and metric learning from constraints. A number of real-world traffic traces have been used to show the availability of constraints and to test the proposed approach. The experimental results indicate that by incorporating constraints in the course of clustering, the overall accuracy and cluster purity can be significantly improved.

Reliable aggregation on network traffic for web based knowledge discovery

The web is a rich resource for information discovery, as a result web mining is a hot topic. However, a reliable mining result depends on the reliability of the data set. For every single second, the web generate huge amount of data, such as web page requests, file transportation. The data reflect human behavior in the cyber space and therefore valuable for our analysis in various disciplines, e.g. social science, network security. How to deposit the data is a challenge. An usual strategy is to save the abstract of the data, such as using aggregation functions to preserve the features of the original data with much smaller space. A key problem, however is that such information can be distorted by the presence of illegitimate traffic, e.g. botnet recruitment scanning, DDoS attack traffic, etc. An important consideration in web related knowledge discovery then is the robustness of the aggregation method , which in turn may be affected by the reliability of network traffic data. In this chapter, we first present the methods of aggregation functions, and then we employe information distances to filter out anomaly data as a preparation for web data mining.

Hierarchical summarization techniques for network traffic

In today’s high speed networks it is becoming increasingly challenging for network managers to understand the nature of the traffic that is carried in their network. A major problem for traffic analysis in this context is how to extract a concise yet accurate summary of the relevant aggregate traffic flows that are present in network traces. In this paper, we present two summarization techniques to minimize the size of the traffic flow report that is generated by a hierarchical cluster analysis tool. By analyzing the accuracy and compaction gain of our approach on a standard benchmark dataset, we demonstrate that our approach achieves more accurate summaries than those of an existing tool that is based on frequent itemset mining.

Number plate survey methodology for long-distance traffic flows

The lack of comprehensive data on transport operations is a long- standing problem in transport research. Information on road transport in particular has proved difficult to obtain. This Paper documents a study which was aimed at developing and testing a technique to estimate long-distance passenger and freight movements based on direct observation of vehicle movements.

Predicted packet padding for anonymous web browsing against traffic analysis attacks

Anonymous communication has become a hot research topic in order to meet the increasing demand for web privacy protection. However, there are few such systems which can provide high level anonymity for web browsing. The reason is the current dominant dummy packet padding method for anonymization against traffic analysis attacks. This method inherits huge delay and bandwidth waste, which inhibits its use for web browsing. In this paper, we propose a predicted packet padding strategy to replace the dummy packet padding method for anonymous web browsing systems. The proposed strategy mitigates delay and bandwidth waste significantly on average. We formulated the traffic analysis attack and defense problem, and defined a metric, cost coefficient of anonymization (CCA), to measure the performance of anonymization. We thoroughly analyzed the problem with the characteristics of web browsing and concluded that the proposed strategy is better than the current dummy packet padding strategy in theory. We have conducted extensive experiments on two real world data sets, and the results confirmed the advantage of the proposed method.

Guest editorial : In Journal of networks, v.7 n.3

Networking of computing devices has been going through rapid evolution and thus continuing to be an ever expanding area of importance in recent years. New technologies, protocols, services and usage patterns have contributed to the major research interests in this area of computer science. The current special issue is an effort to bring forward some of these interesting developments that are being pursued by researchers at present in different parts of the globe. Our objective is to provide the readership with some insight into the latest innovations in computer networking through this. This Special Issue presents selected papers from the thirteenth conference of the series (ICCIT 2010) held during December 23-25, 2010 at the Ahsanullah University of Science and Technology. The first ICCIT was held in Dhaka, Bangladesh, in 1998. Since then the conference has grown to be one of the largest computer and IT related research conferences in the South Asian region, with participation of academics and researchers from many countries around the world. Starting in 2008 the proceedings of ICCIT are included in IEEExplore. In 2010, a total of 410 full papers were submitted to the conference of which 136 were accepted after reviews conducted by an international program committee comprising 81 members from 16 countries. This was tantamount to an acceptance rate of 33%. From these 136 papers, 14 highly ranked manuscripts were invited for this Special Issue. The authors were advised to enhance their papers significantly and submit them to undergo review for suitability of inclusion into this publication. Of those, eight papers survived the review process and have been selected for inclusion in this Special Issue. The authors of these papers represent academic and/or research institutions from Australia, Bangladesh, Japan, Korea and USA. These papers address issues concerning different domains of networks namely, optical fiber communication, wireless and interconnection networks, issues related to networking hardware and software and network mobility. The paper titled “Virtualization in Wireless Sensor Network: Challenges and Opportunities” argues in favor of bringing in different heterogeneous sensors under a common virtual framework so that the issues like flexibility, diversity, management and security can be handled practically. The authors Md. Motaharul Islam and Eui-Num Huh propose an architecture for sensor virtualization. They also present the current status and the challenges and opportunities for further research on the topic. The manuscript “Effect of Polarization Mode Dispersion on the BER Performance of Optical CDMA” deals with impact of polarization mode dispersion on the bit error rate performance of direct sequence optical code division multiple access. The authors, Md. Jahedul Islam and Md. Rafiqul Islam present an analytical approach toward determining the impact of different performance parameters. The authors show that the bit error rate performance improves significantly by the third order polarization mode dispersion than its first or second order counterparts. The authors Md. Shohrab Hossain, Mohammed Atiquzzaman and William Ivancic of the paper “Cost and Efficiency Analysis of NEMO Protocol Entities” present an analytical model for estimating the cost incurred by major mobility entities of a NEMO. The authors define a new metric for cost calculation in the process. Both the newly developed metric and the analytical model are likely to be useful to network engineers in estimating the resource requirement at the key entities while designing such a network. The article titled “A Highly Flexible LDPC Decoder using Hierarchical Quasi-Cyclic Matrix with Layered Permutation” deals with Low Density Parity Check decoders. The authors, Vikram Arkalgud Chandrasetty and Syed Mahfuzul Aziz propose a novel multi-level structured hierarchical matrix approach for generating codes of different lengths flexibly depending upon the requirement of the application. The manuscript “Analysis of Performance Limitations in Fiber Bragg Grating Based Optical Add-Drop Multiplexer due to Crosstalk” has been contributed by M. Mahiuddin and M. S. Islam. The paper proposes a new method of handling crosstalk with a fiber Bragg grating based optical add drop multiplexer (OADM). The authors show with an analytical model that different parameters improve using their proposed OADM. The paper “High Performance Hierarchical Torus Network Under Adverse Traffic Patterns” addresses issues related to hierarchical torus network (HTN) under adverse traffic patterns. The authors, M.M. Hafizur Rahman, Yukinori Sato, and Yasushi Inoguchi observe that dynamic communication performance of an HTN under adverse traffic conditions has not yet been addressed. The authors evaluate the performance of HTN for comparison with some other relevant networks. It is interesting to see that HTN outperforms these counterparts in terms of throughput and data transfer under adverse traffic. The manuscript titled “Dynamic Communication Performance Enhancement in Hierarchical Torus Network by Selection Algorithm” has been contributed by M.M. Hafizur Rahman, Yukinori Sato, and Yasushi Inoguchi. The authors introduce three simple adapting routing algorithms for efficient use of physical links and virtual channels in hierarchical torus network. The authors show that their approaches yield better performance for such networks. The final title “An Optimization Technique for Improved VoIP Performance over Wireless LAN” has been contributed by five authors, namely, Tamal Chakraborty, Atri Mukhopadhyay, Suman Bhunia, Iti Saha Misra and Salil K. Sanyal. The authors propose an optimization technique for configuring the parameters of the access points. In addition, they come up with an optimization mechanism in order to tune the threshold of active queue management system appropriately. Put together, the mechanisms improve the VoIP performance significantly under congestion. Finally, the Guest Editors would like to express their sincere gratitude to the 15 reviewers besides the guest editors themselves (Khalid M. Awan, Mukaddim Pathan, Ben Townsend, Morshed Chowdhury, Iftekhar Ahmad, Gour Karmakar, Shivali Goel, Hairulnizam Mahdin, Abdullah A Yusuf, Kashif Sattar, A.K.M. Azad, F. Rahman, Bahman Javadi, Abdelrahman Desoky, Lenin Mehedy) from several countries (Australia, Bangladesh, Japan, Pakistan, UK and USA) who have given immensely to this process. They have responded to the Guest Editors in the shortest possible time and dedicated their valuable time to ensure that the Special Issue contains high-quality papers with significant novelty and contributions.

Network and traffic engineering in emerging distributed computing applications

This book focuses on network management and traffic engineering for Internet and distributed computing technologies, as well as present emerging technology trends and advanced platform

Internet traffic clustering with constraints

Due to the limitations of the traditional port-based and payload-based traffic classification approaches, the past decade has seen extensive work on utilizing machine learning techniques to classify network traffic based on packet and flow level features. In particular, previous studies have shown that the unsupervised clustering approach is both accurate and capable of discovering previously unknown application classes. In this paper, we explore the utility of side information in the process of traffic clustering. Specifically, we focus on the flow correlation information that can be efficiently extracted from packet headers and expressed as instance-level constraints, which indicate that particular sets of flows are using the same application and thus should be put into the same cluster. To incorporate the constraints, we propose a modified constrained K-Means algorithm. A variety of real-world traffic traces are used to show that the constraints are widely available. The experimental results indicate that the constrained approach not only improves the quality of the resulted clusters, but also speeds up the convergence of the clustering process.

Semi-supervised and compound classification of network traffic

This paper presents a new semi-supervised method to effectively improve traffic classification performance when few supervised training data are available. Existing semi supervised methods label a large proportion of testing flows as unknown flows due to limited supervised information, which severely affects the classification performance. To address this problem, we propose to incorporate flow correlation into both training and testing stages. At the training stage, we make use of flow correlation to extend the supervised data set by automatically labeling unlabeled flows according to their correlation to the pre-labeled flows. Consequently, the traffic classifier has better performance due to the extended size and quality of the supervised data sets. At the testing stage, the correlated flows are identified and classified jointly by combining their individual predictions, so as to further boost the classification accuracy. The empirical study on the real-world network traffic shows that the proposed method outperforms the state-of-the-art flow statistical feature based classification methods.

Classification of correlated internet traffic flows

A critical problem for Internet traffic classification is how to obtain a high-performance statistical feature based classifier using a small set of training data. The solutions to this problem are essential to deal with the encrypted applications and the new emerging applications. In this paper, we propose a new Naive Bayes (NB) based classification scheme to tackle this problem, which utilizes two recent research findings, feature discretization and flow correlation. A new bag-of-flow (BoF) model is firstly introduced to describe the correlated flows and it leads to a new BoF-based traffic classification problem. We cast the BoF-based traffic classification as a specific classifier combination problem and theoretically analyze the classification benefit from flow aggregation. A number of combination methods are also formulated and used to aggregate the NB predictions of the correlated flows. Finally, we carry out a number of experiments on a large scale real-world network dataset. The experimental results show that the proposed scheme can achieve significantly higher classification accuracy and much faster classification speed with comparison to the state-of-the-art traffic classification methods.

Direct versus search engine traffic : an innovative approach to demand analysis in the property market

Purpose – The application of “Google” econometrics (Geco) has evolved rapidly in recent years and can be applied in various fields of research. Based on accepted theories in existing economic literature, this paper seeks to contribute to the innovative use of research on Google search query data to provide a new innovative to property research.

Design/methodology/approach – In this study, existing data from Google Insights for Search (GI4S) is extended into a new potential source of consumer sentiment data based on visits to a commonly-used UK online real-estate agent platform (Rightmove.co.uk). In order to contribute to knowledge about the use of Geco's black box, namely the unknown sampling population and the specific search queries influencing the variables, the GI4S series are compared to direct web navigation.

Findings – The main finding from this study is that GI4S data produce immediate real-time results with a high level of reliability in explaining the future volume of transactions and house prices in comparison to the direct website data. Furthermore, the results reveal that the number of visits to Rightmove.co.uk is driven by GI4S data and vice versa, and indeed without a contemporaneous relationship.

Originality/value – This study contributes to the new emerging and innovative field of research involving search engine data. It also contributes to the knowledge base about the increasing use of online consumer data in economic research in property markets.

Congestion and optimal immigration policy

This paper presents a model to explain the stylized fact that many countries have a low ratio of migrants in their population while some countries have a high ratio of migrants. Immigration improves the income of the domestic residents, but migrants also increase the congestion of public services. If migrants are unskilled and therefore pay low taxes, and the government does not limit access to these services, then the welfare of the domestic residents decreases with the number of migrants. Visa auctions can lower the cost of immigration control and substitute legal migrants for illegal migrants. If the government decides to limit the access of migrants to public services, immigration control becomes unnecessary and the optimal number of migrants can be very large.

Effects of exposure to traffic noise on health

Traffic noise causes adverse effects on the health and quality of life of individuals and communities exposed to it, including annoyance, sleep disturbance, decreased performance at school/work, stress, hypertension, and ischemic heart disease. In Australia there are few standards or policies addressing noise in urban environments, with many discrepancies in noise level thresholds when comparing states and regions. Currently Victoria has a day-to-night threshold for noise levels well above accepted levels in Europe, and there is no standard for the late night period. A better understanding of the health impacts of noise in the Australian context is vital for informing development and implementation of policy and legislation for road traffic noise management. This paper reviews the evidence base and policies related to traffic noise in urban areas, and presents a case study of noise mapping and assessing population health impacts (eg. sleep disturbance), in Geelong,Vcitoria,Australia.

Distributed denial of service (DDoS) detection by traffic pattern analysis

In this paper, we propose a behavior-based detection that can discriminate Distributed Denial of Service (DDoS) attack traffic from legitimated traffic regardless to various types of the attack packets and methods. Current DDoS attacks are carried out by attack tools, worms and botnets using different packet-transmission rates and packet forms to beat defense systems. These various attack strategies lead to defense systems requiring various detection methods in order to identify the attacks. Moreover, DDoS attacks can craft the traffics like flash crowd events and fly under the radar through the victim. We notice that DDoS attacks have features of repeatable patterns which are different from legitimate flash crowd traffics. In this paper, we propose a comparable detection methods based on the Pearson’s correlation coefficient. Our methods can extract the repeatable features from the packet arrivals in the DDoS traffics but not in flash crowd traffics. The extensive simulations were tested for the optimization of the detection methods. We then performed experiments with several datasets and our results affirm that the proposed methods can differentiate DDoS attacks from legitimate traffics.