Network traffic clustering using random forest proximities

The recent years have seen extensive work on statistics-based network traffic classification using machine learning (ML) techniques. In the particular scenario of learning from unlabeled traffic data, some classic unsupervised clustering algorithms (e.g. K-Means and EM) have been applied but the reported results are unsatisfactory in terms of low accuracy. This paper presents a novel approach for the task, which performs clustering based on Random Forest (RF) proximities instead of Euclidean distances. The approach consists of two steps. In the first step, we derive a proximity measure for each pair of data points by performing a RF classification on the original data and a set of synthetic data. In the next step, we perform a K-Medoids clustering to partition the data points into K groups based on the proximity matrix. Evaluations have been conducted on real-world Internet traffic traces and the experimental results indicate that the proposed approach is more accurate than the previous methods.

Classifying network traffic in the big data era

With the arrival of Big Data Era, properly utilizing the power of big data is becoming increasingly essential for the strength and competitiveness of businesses and organizations. We are facing grand challenges from big data from different perspectives, such as processing, communication, security, and privacy. In this talk, we discuss the big data challenges in network traffic classification and our solutions to the challenges. The significance of the research lies in the fact that each year the network traffic increase exponentially on the current Internet. Traffic classification has wide applications in network management, from security monitoring to quality of service measurements. Recent research tends to apply machine-learning techniques to flow statistical feature based classification methods. In this talk, we propose a series of novel approaches for traffic classification, which can improve the classification performance effectively by incorporating correlated information into the classification process. We analyze the new classification approaches and their performance benefit from both theoretical and empirical perspectives. A large number of experiments are carried out on two real-world traffic datasets to validate the proposed approach. The results show the traffic classification performance can be improved significantly even under the extreme difficult circumstance of very few training samples. Our work has significant impact on security applications.

Feel bored? Join Verse! Engineering Vehicular Proximity Social Network

Travellers in vehicles often have strong willingness to share their travel experience and exchange information to each other through social networks2, such as Facebook and Twitter. This, however, can be costly due to the limited connections to Internet on the road. In this paper we develop Verse to facilitate the social communications among vehicle travellers on highways. Verse enables passengers on-board vehicles to share the content information, such as travel blogs with pictures, among each other using the impromptu wireless inter-vehicle communications. Unlike traditional online social networks, which are built upon the reliable IP networks, vehicular social networks face fundamental challenges in that: 1) users are anonymous and strangers to each other and hard to identify potential friends of shared interests, and 2) users communicate through intermittent and unreliable inter-vehicle connections. On addressing the two challenges, Verse implements a friend recommendation function, which helps passengers efficiently identify potential social friends with both shared interests and relatively reliable wireless connections. In addition, Verse is equipped with a social-aware rate control scheme towards efficient utilization of network bandwidth. Using extensive simulations, we show that the friend recommendation function of Verse can effectively predict the mobility of vehicles to assist the social communication, and the social-aware rate control scheme quickly and efficiently adapts the vehicle’s transmission rate according to their social impacts.

Network dynamics of mobile social networks

With the rapid development of smartphones and mobile Internet technology, we witness an overwhelming growth of mobile social networks (MSN), which is a type of social network, forming virtual communities among people with similar interests or commonalities. In MSNs, users play a crucial role for their development, deployment and success. Understanding the MSN user behavior therefore attracts interests of different entities - ISPs, service providers, and researchers. However, it is hard to gather a comprehensive real data set, little is known and even less has been published about MSN user activities. In this paper, we focus on analyzing MSN user behavior from the perspective of ISP network, which is seldom reported in literature. Based on the real data set collected from the mobile network gateway of a major mobile carrier who has more than five million subscribers, we present an in-depth user behavior analysis of four popular social networks. We study the MSN user behavior from six aspects: user requests, active online time, sessions, inter-session, the number of requests in a session, and inter-request. We found that power law and lognormal are two popular features of the studied objects, and exposed some interesting findings as well. We hope our work could be helpful for ISPs, MSN content providers, and researchers. © 2014 IEEE.

On addressing the imbalance problem : A correlated KNN approach for network traffic classification

With the arrival of big data era, the Internet traffic is growing exponentially. A wide variety of applications arise on the Internet and traffic classification is introduced to help people manage the massive applications on the Internet for security monitoring and quality of service purposes. A large number of Machine Learning (ML) algorithms are introduced to deal with traffic classification. A significant challenge to the classification performance comes from imbalanced distribution of data in traffic classification system. In this paper, we proposed an Optimised Distance-based Nearest Neighbor (ODNN), which has the capability of improving the classification performance of imbalanced traffic data. We analyzed the proposed ODNN approach and its performance benefit from both theoretical and empirical perspectives. A large number of experiments were implemented on the real-world traffic dataset. The results show that the performance of “small classes” can be improved significantly even only with small number of training data and the performance of “large classes” remains stable.

Uso do network simulator-NS para simulação de sistemas distribuídos em cenários com defeitos

O desenvolvimento de protocolos distribuídos é uma tarefa complexa. Em sistemas tolerantes a falhas, a elaboração de mecanismos para detectar e mascarar defeitos representam grande parte do esforço de desenvolvimento. A técnica de simulação pode auxiliar significativamente nessa tarefa. Entretanto, existe uma carência de ferramentas de simulação para investigação de protocolos distribuídos em cenários com defeitos, particularmente com suporte a experimentos em configurações “típicas” da Internet. O objetivo deste trabalho é investigar o uso do simulador de redes NS (Network Simulator) como ambiente para simulação de sistemas distribuídos, particularmente em cenários sujeitos à ocorrência de defeitos. O NS é um simulador de redes multi-protocolos, que tem código aberto e pode ser estendido. Embora seja uma ferramenta destinada ao estudo de redes de computadores, o ajuste adequado de parâmetros e exploração de características permitiu utilizá-lo para simular defeitos em um sistema distribuído. Para isso, desenvolveu-se dois modelos de sistemas distribuídos que podem ser implementados no NS, dependendo do protocolo de transporte utilizado: um baseado em TCP e o outro baseado em UDP. Também, foram estudadas formas de modelar defeitos através do simulador. Para a simulação de defeito de colapso em um nodo, foi proposta a implementação de um método na classe de cada aplicação na qual se deseja simular defeitos. Para ilustrar como os modelos de sistemas distribuídos e de defeitos propostos podem ser utilizados, foram implementados diversos algoritmos distribuídos em sistemas síncronos e assíncronos. Algoritmos de eleição e o protocolo Primário-Backup são exemplos dessas implementações. A partir desses algoritmos, principalmente do Primário-Backup, no qual a simulação de defeitos foi realizada, foi possível constatar que o NS pode ser uma ferramenta de grande auxílio no desenvolvimento de novas técnicas de Tolerância a Falhas. Portanto, o NS pode ser estendido possibilitando que, com a utilização dos modelos apresentados nesse trabalho, simule-se defeitos em um sistema distribuído.

IP network usage accounting

De modo a manter políticas de utilização aceitável dos seus serviços de Internet, a NOS Madeira tem usado um sistema de fabrico próprio onde os clientes são catalogados de acordo com o tráfego que realizam. Contudo, esse sistema tornou-se demasiado antigo para as necessidades atuais da empresa. Usava tecnologias descontinuadas, não tinha interfaces de integração, faltava modularidade e não tinha a flexibilidade necessária para expandir as regras de negócio. Este projeto centra-se na implementação de um dos três subsistemas que substituem o sistema antigo: o subsistema controlador. O objetivo é modernizar, facilitar a manutenção e garantir maior flexibilidade. Tudo isto com recurso a linguagens de programação atuais como o PHP, ferramentas como a Zend Framework e mantendo em mente as melhores práticas de programação. São apresentados a especificação e modelação do sistema, assim como todos os detalhes da implementação em conjunto com as decisões e problemas encontrados. Os testes e resultados, incluindo a entrada com sucesso em produção do sistema, juntamente com sugestões de melhorias futuras concluem este trabalho.

Statistical model applied to NetFlow for network intrusion detection

The computers and network services became presence guaranteed in several places. These characteristics resulted in the growth of illicit events and therefore the computers and networks security has become an essential point in any computing environment. Many methodologies were created to identify these events; however, with increasing of users and services on the Internet, many difficulties are found in trying to monitor a large network environment. This paper proposes a methodology for events detection in large-scale networks. The proposal approaches the anomaly detection using the NetFlow protocol, statistical methods and monitoring the environment in a best time for the application. © 2010 Springer-Verlag Berlin Heidelberg.

Network node with wireless and wired interfaces: Nios II processor and uClinux to development of a NCAP embedded (IEEE 1451.1) with two interfaces, wireless (IEEE 1451.5) and wired (IEEE p1451.2)

This paper presents a NCAP embedded on DE2 kit with Nios II processor and uClinux to development of a network gateway with two interfaces, wireless (ZigBee) and wired (RS232) based on IEEE 1451. Both the communications, wireless and wired, were developed to be point-to-point and working with the same protocols, based on IEEE 1451.0-2007. The tests were made using a microcomputer, which through of browser was possible access the web page stored in the DE2 kit and send commands of control and monitoring to both TIMs (WTIM and STIM). The system describes a different form of development of the NCAP node to be applied in different environments with wired or wireless in the same node. © 2011 IEEE.

Network for Cooperation in Integrated Water Resource Management for Sustainable Development in Latin America and the Caribbean N° 41

• Editorial remarks.-- Open discussion: Tariffs and subsidies: the current situation and trends in the region ; State-owned utilities and the flight from public law: challenges and trends ; Challenges and opportunities in access to water and sanitation in rural areas.-- Meetings: Proposals based on the Water and Environment Initiative consensuses.-- News of the Network: Peru’s Compensation Mechanisms for Ecosystem Services Act ; Ecuador’s Act on Water Resources and Water Use and Exploitation ; The environmental dynamics of groundwater in Mexico ; The Water Citizenship Programme in the province of Mendoza, Argentina.-- Internet and WWW News

Network for Cooperation in Integrated Water Resource Management for Sustainable Development in Latin America and the Caribbean No. 42

Editorial remarks.-- Open discussion: Conceptual change in regulation in a model of public service provision ; Policies and institutional frameworks for drinking water supply and sanitation ; Strategies for low-carbon development in megacities in Latin America ; Adapting to climate change in water management in the irrigation sector.-- Meetings: Towards a vision on natural resource governance for equality ; Water resources faced with uncertainty and the risk of climate change ; Regulation challenges in the water sector.-- News of the Network: Lessons to be drawn from the interprovincial Colorado River flow distribution agreement ; Rural drinking water programme in Chile ; Ecuador’s Act on Water Resources and Water Use and Exploitation.-- Internet and WWW News

Network for Cooperation in Integrated Water Resource Management for Sustainable Development in Latin America and the Caribbean No. 43

Editorial remarks.-- Open discussion: Energy efficiency and economic regulation in water supply and sewerage services ; Policies and institutional frameworks for drinking water supply and sanitation.-- News of the network: The goal of food self-sufficiency of the countries of the cooperation council for de Arab States of the Gulf ; Narrowing the gaps in drinking water and sanitation: an opportunity for users ; Water Users'Organizations Act of Peru ; Commission for the Integrated management of the Tárcoles River Basin, Costa Rica ; Water and sanitation Management Unit in Panama.-- Courses: Course "Strategic aspects of irrigation".-- Internet and WWW news

Network for Cooperation in Integrated Water Resource Management for Sustainable Development in Latin America and the Caribbean No. 7

Editorial remarks.-- Open discussion: public policies for sustainable development.-- Meetings: Workshop for Managers of River Basin Organizations in Latin America and the Caribbean ; General Assembly of the International Network of Basin Organizations ; Seminar workshop on environmental accounting for water resources in Chile.-- Future activities.-- Courses.-- CELAA, CINARA and IMTA.-- Internet and WWW News

Network for Cooperation in Integrated Water Resource Management for Sustainable Development in Latin America and the Caribbean No. 24

Editorial remarks.-- Open discussion: Integrating economics, legislation and administration in water and water services management ; Water and Free Trade Agreements ; Provision of transport infraestructure.-- News of the network: WALIR ; Prevention and reduction of the Danger Posed by natural disasters ; Ministry of Water in Bolivia ; Water management council of the Paute Basin in Ecuador ; CRSS.-- Internet and WWW news