Multiple time series anomaly detection based on compression and correlation analysis: a medical surveillance case study

In this paper, we present a novel anomaly detection framework for multiple heterogeneous yet correlated time series, such as the medical surveillance series data. In our framework, we propose an anomaly detection algorithm from the viewpoint of trend and correlation analysis. Moreover, to efficiently process huge amount of observed time series, a new clustering-based compression method is proposed. Experimental results indicate that our framework is more effective and efficient than its peers. © 2012 Springer-Verlag Berlin Heidelberg.

Profiling pedestrian distribution and anomaly detection in a dynamic environment

Pedestrians movements have a major impact on the dynamics of cities and provide valuable guidance to city planners. In this paper we model the normal behaviours of pedestrian flows and detect anomalous events from pedestrian counting data of the City of Melbourne. Since the data spans an extended period, and pedestrian activities can change intermittently (e.g., activities in winter vs. summer), we applied an Ensemble Switching Model, which is a dynamic anomaly detection technique that can accommodate systems that switch between different states. The results are compared with those produced by a static clustering model (Hy-CARCE) and also cross-validated with known events. We found that the results from the Ensemble Switching Model are valid and more accurate than HyCARCE.

Experience report: anomaly detection of cloud application operations using log and cloud metric correlation analysis

Failure of application operations is one of the maincauses of system-wide outages in cloud environments. Thisparticularly applies to DevOps operations, such as backup,redeployment, upgrade, customized scaling, and migration that areexposed to frequent interference from other concurrent operations,configuration changes, and resources failure. However, currentpractices fail to provide a reliable assurance of correct execution ofthese kinds of operations. In this paper, we present an approach toaddress this problem that adopts a regression-based analysistechnique to find the correlation between an operation’s activity logsand the operation activity’s effect on cloud resources. Thecorrelation model is then used to derive assertion specifications,which can be used for runtime verification of running operations andtheir impact on resources. We evaluated our proposed approach onAmazon EC2 with 22 rounds of rolling upgrade operations whileother types of operations were running and random faults wereinjected. Our experiment shows that our approach successfullymanaged to raise alarms for 115 random injected faults, with aprecision of 92.3%.

R1SVM: a randomised nonlinear approach to large-scale anomaly detection

The problem of unsupervised anomaly detection arises in a wide variety of practical applications. While one-class support vector machines have demonstrated their effectiveness as an anomaly detection technique, their ability to model large datasets is limited due to their memory and time complexity for training. To address this issue for supervised learning of kernel machines, there has been growing interest in random projection methods as an alternative to the computationally expensive problems of kernel matrix construction and support vector optimisation. In this paper we leverage the theory of nonlinear random projections and propose the Randomised One-class SVM (R1SVM), which is an efficient and scalable anomaly detection technique that can be trained on large-scale datasets. Our empirical analysis on several real-life and synthetic datasets shows that our randomised 1SVM algorithm achieves comparable or better accuracy to deep autoen-coder and traditional kernelised approaches for anomaly detection, while being approximately 100 times faster in training and testing.

Detection and classification of anomaly intrusion using hierarchy clustering and SVM

Anomaly detection as a kind of intrusion detection is good at detecting the unknown attacks or new attacks, and it has attracted much attention during recent years. In this paper, a new hierarchy anomaly intrusion detection model that combines the fuzzy c-means (FCM) based on genetic algorithm and SVM is proposed. During the process of detecting intrusion, the membership function and the fuzzy interval are applied to it, and the process is extended to soft classification from the previous hard classification. Then a fuzzy error correction sub interval is introduced, so when the detection result of a data instance belongs to this range, the data will be re-detected in order to improve the effectiveness of intrusion detection. Experimental results show that the proposed model can effectively detect the vast majority of network attack types, which provides a feasible solution for solving the problems of false alarm rate and detection rate in anomaly intrusion detection model.

A visual-numeric approach to clustering and anomaly detection for trajectory data

This paper proposes a novel application of Visual Assessment of Tendency (VAT)-based hierarchical clustering algorithms (VAT, iVAT, and clusiVAT) for trajectory analysis. We introduce a new clustering based anomaly detection framework named iVAT+ and clusiVAT+ and use it for trajectory anomaly detection. This approach is based on partitioning the VAT-generated Minimum Spanning Tree based on an efficient thresholding scheme. The trajectories are classified as normal or anomalous based on the number of paths in the clusters. On synthetic datasets with fixed and variable numbers of clusters and anomalies, we achieve 98 % classification accuracy. Our two-stage clusiVAT method is applied to 26,039 trajectories of vehicles and pedestrians from a parking lot scene from the real life MIT trajectories dataset. The first stage clusters the trajectories ignoring directionality. The second stage divides the clusters obtained from the first stage by considering trajectory direction. We show that our novel two-stage clusiVAT approach can produce natural and informative trajectory clusters on this real life dataset while finding representative anomalies.

Anomaly detection in wireless sensor networks in a non-stationary environment

Anomaly detection in a WSN is an important aspect of data analysis in order to identify data items that significantly differ from normal data. A characteristic of the data generated by a WSN is that the data distribution may alter over the lifetime of the network due to the changing nature of the phenomenon being observed. Anomaly detection techniques must be able to adapt to a non-stationary data distribution in order to perform optimally. In this survey, we provide a comprehensive overview of approaches to anomaly detection in a WSN and their operation in a non-stationary environment.

Ellipsoidal neighbourhood outlier factor for distributed anomaly detection in resource constrained networks

Anomaly detection in resource constrained wireless networks is an important challenge for tasks such as intrusion detection, quality assurance and event monitoring applications. The challenge is to detect these interesting events or anomalies in a timely manner, while minimising energy consumption in the network. We propose a distributed anomaly detection architecture, which uses multiple hyperellipsoidal clusters to model the data at each sensor node, and identify global and local anomalies in the network. In particular, a novel anomaly scoring method is proposed to provide a score for each hyperellipsoidal model, based on how remote the ellipsoid is relative to their neighbours. We demonstrate using several synthetic and real datasets that our proposed scheme achieves a higher detection performance with a significant reduction in communication overhead in the network compared to centralised and existing schemes. © 2014 Elsevier Ltd.

Hyperspherical cluster based distributed anomaly detection in wireless sensor networks

This article describes a distributed hyperspherical cluster based algorithm for identifying anomalies in measurements from a wireless sensor network, and an implementation on a real wireless sensor network testbed. The communication overhead incurred in the network is minimised by clustering sensor measurements and merging clusters before sending a compact description of the clusters to other nodes. An evaluation on several real and synthetic datasets demonstrates that the distributed hyperspherical cluster-based scheme achieves comparable detection accuracy with a significant reduction in communication overhead compared to a centralised scheme, where all the sensor node measurements are communicated to a central node for processing. .

An adaptive elliptical anomaly detection model for wireless sensor networks

Wireless Sensor Networks (WSNs) provide a low cost option for monitoring different environments such as farms, forests and water and electricity networks. However, the restricted energy resources of the network impede the collection of raw monitoring data from all the nodes to a single location for analysis. This has stimulated research into efficient anomaly detection techniques to extract information about unusual events such as malicious attacks or faulty sensors at each node. Many previous anomaly detection methods have relied on centralized processing of measurement data, which is highly communication intensive. In this paper, we present an efficient algorithm to detect anomalies in a decentralized manner. In particular, we propose a novel adaptive model for anomaly detection, as well as a robust method for modeling normal behavior. Our evaluation results on both real-life and simulated data sets demonstrate the accuracy of our approach compared to existing methods.

Drying front in a sloping aquifier: nonlinear effects

The profiles for the water table height h(x, t) in a shallow sloping aquifer are reexamined with a solution of the nonlinear Boussinesq equation. We demonstrate that the previous anomaly first reported by Brutsaert [1994] that the point at which the water table h first becomes zero at x = L at time t = t c remains fixed at this point for all times t > t c is actually a result of the linearization of the Boussinesq equation and not, as previously suggested [ Brutsaert, 1994 ; Verhoest and Troch, 2000 ], a result of the Dupuit assumption. Rather, by examination of the nonlinear Boussinesq equation the drying front, i.e., the point x f at which h is zero for times t ≥ t c , actually recedes downslope as physically expected. This points out that the linear Boussinesq equation should be used carefully when a zero depth is obtained as the concept of an “average” depth loses meaning at that time.

Discomfort at the coalface : issues for sessional tutors teaching in online enhanced learning environments

Higher education institutions are undergoing a period of rapid change during which time a number of strategic professional development efforts have been made to improve teaching in order to improve students’ learning outcomes. Sessional tutors, who are consistently at the coalface and have close contact with students, have often been excluded from formal opportunities for professional development offered to more permanent staff. This anomaly is now being recognised and more efforts are being made across the sector to ensure that tutors are better equipped to teach in contemporary learning environments. This paper discusses issues of concern to tutors that arose from recent professional development workshops and suggests that some of the major issues currently confronting sessional staff relate to the need to be able to teach effectively in online enhanced learning environments.

Where else have you been? The effects of diaspora consciousness and transcultural mixtures on ethnic identity

In social science research, the demographic categories of ethnicity are linked to what the census bureau considers as a person’s ethnic heritage. However, these categories are based on the societal assumption that members of a given category share the same characteristics and life experiences, even though the heterogeneity between members within a category may be as diverse as between categories. The paper examines the 15 interview subjects of a research study drawn from 10 minority migrant groups, where seven of them indicated significant transcultural experiences before migrating to Australia. It argues that their lived experiences and subjectivity vary from others who migrated directly from their native countries. The formers’ diaspora consciousness and transcultural mixtures may introduce an artifact to a research study’s design, affecting the validity of the data collected. The paper examines other situations where this anomaly can occur and proposes precautions
to minimize its negative effects.

People, places and public health : evaluating the benefits of civic environmentalism in an urban local government context

Recent political, economic and social trends pose threats to the sustainability both of ecosystems and of human health. Australia’s environmental management record is poor, and while by international standards Australians enjoy good health, this is variable (AIHW, 2000). Within developed nations, heart disease, depression, alcohol dependence and stroke are major health issues (Mathers et al. 2002). In Australia, mental disorder is the number one contributor to the disease burden (Vos & Mathers 2000). Recent research has highlighted the role of social capital as a key determinant of health (Kawachi et al., 1997). Despite this, Putnam (1995) observes that social connectedness and civic engagement are in decline. People have less time for leisure and for volunteering, as many juggle paid work and caring for children. Anecdotal evidence suggests that engagement in civic environmentalism has human health benefits, relating to a combination of exposure to natural environments and increased social capital (Maller, Brown, Townsend & St. Leger, 2002). This link is supported by Furnass (1996) who defines well-being as including: satisfactory human relationships, meaningful occupation, opportunities for contact with nature, creative expression, and making a positive contribution to human society. Research conducted by Deakin University confirms the efficacy of linking people and places through civic environmentalism for addressing both ecosystem sustainability and human health and wellbeing. The research has included a pilot study to explore the human health benefits of membership of a local parkland ‘Friends’ group, and a more detailed follow-up study. The aims of the pilot study included:- To identify the range of motivations for joining the Friends group;- To document members’ perceptions of the benefits gained from membership of the group;- To assess the potential for Friends groups to be used as an ‘upstream’ health promotion measure.Face-to-face interviews were conducted with eleven members of a ‘Friends’ group in the eastern suburbs of Melbourne. Data was analysed thematically and key findings included:- Motivations: environmental; social; and pragmatic.- General benefits: community belonging; personal satisfaction; learning opportunities; physical activity; and better environment.- Health benefits: physical health; mental health; and social support. There was unanimous support for the use of ‘Friends’ groups as a tool for health promotion.The follow-up study, in the western suburbs of Melbourne, expanded on the pilot study by measuring the group’s social capital and by collecting self-report data on levels of health service usage. Data was collected through face-to-face interviews and a questionnaire. The findings were similar to the pilot study in relation to the motivations, benefits and the health promotion potential of such groups. However, health service usage data highlighted an apparent anomaly: while respondents perceived significant health benefits, some were nevertheless utilising health services at a relatively high level. This poses some questions requiring further exploration: Is this due to the poorer baseline health of the high health service usage members compared with their fellow members? Does involvement in the group offer health benefits that enable people who would otherwise be too unhealthy to participate in community groups to continue such involvement?If this is the case, then we may do well to look to locally-based mechanisms for promoting ecological sustainability as a tool also for promoting human health. Instead of prescribing a pill, connecting people and places through engagement with a local friends group may address our health problems at the same time as addressing local environmental problems.

On the communal analysis suspicion scoring for identity crime in streaming credit applications

This paper describes a rapid technique: communal analysis suspicion scoring (CASS), for generating numeric suspicion scores on streaming credit applications based on implicit links to each other, over both time and space. CASS includes pair-wise communal scoring of identifier attributes for applications, definition of categories of suspiciousness for application-pairs, the incorporation of temporal and spatial weights, and smoothed k-wise scoring of multiple linked application-pairs. Results on mining several hundred thousand real credit applications demonstrate that CASS reduces false alarm rates while maintaining reasonable hit rates. CASS is scalable for this large data sample, and can rapidly detect early symptoms of identity crime. In addition, new insights have been observed from the relationships between applications.