A big data platform for large scale event processing

To date, big data applications have focused on the store-and-process paradigm. In this paper we describe an initiative to deal with big data applications for continuous streams of events. In many emerging applications, the volume of data being streamed is so large that the traditional ‘store-then-process’ paradigm is either not suitable or too inefficient. Moreover, soft-real time requirements might severely limit the engineering solutions. Many scenarios fit this description. In network security for cloud data centres, for instance, very high volumes of IP packets and events from sensors at firewalls, network switches and routers and servers need to be analyzed and should detect attacks in minimal time, in order to limit the effect of the malicious activity over the IT infrastructure. Similarly, in the fraud department of a credit card company, payment requests should be processed online and need to be processed as quickly as possible in order to provide meaningful results in real-time. An ideal system would detect fraud during the authorization process that lasts hundreds of milliseconds and deny the payment authorization, minimizing the damage to the user and the credit card company.

Event-based time series data preprocessing: application to traffic flow time series

Traffic flow time series data are usually high dimensional and very complex. Also they are sometimes imprecise and distorted due to data collection sensor malfunction. Additionally, events like congestion caused by traffic accidents add more uncertainty to real-time traffic conditions, making traffic flow forecasting a complicated task. This article presents a new data preprocessing method targeting multidimensional time series with a very high number of dimensions and shows its application to real traffic flow time series from the California Department of Transportation (PEMS web site). The proposed method consists of three main steps. First, based on a language for defining events in multidimensional time series, mTESL, we identify a number of types of events in time series that corresponding to either incorrect data or data with interference. Second, each event type is restored utilizing an original method that combines real observations, local forecasted values and historical data. Third, an exponential smoothing procedure is applied globally to eliminate noise interference and other random errors so as to provide good quality source data for future work.

An Investigation of Ozone Related Data and Development of a Model to Evaluate Ozone Episodes for Exceptional Event Criteria

The EPA promulgated the Exceptional Events Rule codifying guidance regarding exclusion of monitoring data from compliance decisions due to uncontrollable natural or exceptional events. This capstone examines documentation systems utilized by agencies requesting data be excluded from compliance decisions due to exceptional events. A screening tool is developed to determine whether an event would meet exceptional event criteria. New data sources are available to enhance analysis but evaluation shows many are unusable in their current form. The EPA and States must collaborate to develop consistent evaluation methodologies documenting exceptional events to improve the efficiency and effectiveness of the new rule. To utilize newer sophisticated data, consistent, user-friendly translation systems must be developed.

Efficient detection of emergency event from moving object data streams

The advance of positioning technology enables us to online collect moving object data streams for many applications. One of the most significant applications is to detect emergency event through observed abnormal behavior of objects for disaster prediction. However, the continuously generated moving object data streams are often accumulated to a massive dataset in a few seconds and thus challenge existing data analysis techniques. In this paper, we model a process of emergency event forming as a process of rolling a snowball, that is, we compare a size-rapidly-changed (e.g., increased or decreased) group of moving objects to a snowball. Thus, the problem of emergency event detection can be resolved by snowball discovery. Then, we provide two algorithms to find snowballs: a clustering-and-scanning algorithm with the time complexity of O(n 2) and an efficient adjacency-list-based algorithm with the time complexity of O(nlogn). The second method adopts adjacency lists to optimize efficiency. Experiments on both real-world dataset and large synthetic datasets demonstrate the effectiveness, precision and efficiency of our algorithms © 2014 Springer International Publishing Switzerland.

An investigation of differences in crash characteristics between males and females involved in a fatigue-related crash or close call event

There is consensus among community and road safety agencies that driver fatigue is a major road safety issue and it is well known that excessive fatigue is linked with an increased risk of a motor vehicle crash. Previous research has implicated a wide variety of factors involved in fatigue-related crashes and the effects of these various factors in regard to crash risk can be interpreted as causal (i.e. alcohol and/or drugs may induce fatigue states) or additive (e.g. where a lack of sleep is combined with alcohol). As such, the purpose of this investigation was to examine self-report data to determine whether there are any differences in the prevalence, crash characteristics, and travel patterns of males and females involved in a fatigue-related crash or close call event. Such research is important to understand how fatigue related incidents occur within the typical driving patterns of men and women and it provides a starting point in order to explore if males and females experience and understand the risk of diving when tired in the same way. A representative sample of (N = 1,600) residents living in the Australian Capital Territory (ACT) and New South Wales (NSW), Australia, were surveyed regarding their experience of fatigue and their involvement in fatigue-related crashes and close call incidents. Results revealed that over 35% of participants reported having had a close call or crash due to driving when tired in the five years prior to the study being conducted. In addition, the results obtained revealed a number of interesting characteristics that provide preliminary evidence that gender differences do exist when examining the prevalence, crash characteristics, and travel patterns of males and females involved in a fatigue-related crash or close call event. It is argued that the results obtained can provide particularly useful information for the refinement and further development of appropriate countermeasures that better target this complex issue.

Building partnerships as a key strategy in developing an event management model for an international dance event : a case study of the 2008 WDA (World Dance Alliance) Global Summit, Brisbane, Australia

With the increasing growth of cultural events both in Australia and internationally, there has also been an increase in event management studies; in theory and in practice. Although a series of related knowledge and skills required specifically by event managers has already been identified by many researchers (Perry et al., 1996; Getz, 2002 & Silvers et al., 2006) and generic event management models proposed, including ‘project management’ strategies in an event context (Getz, 2007), knowledge gaps still exist in relation to identifying specific types of events, especially for not-for-profit arts events. For events of a largely voluntary nature, insufficient resources are recognised as the most challenging; including finance, human resources and infrastructure. Therefore, the concepts and principles which are adopted by large scale commercial events may not be suitable for not-for-profit arts events aiming at providing professional network opportunities for artists. Building partnerships are identified as a key strategy in developing an effective event management model for this type of event. Using the 2008 World Dance Alliance Global Summit (WDAGS) in Brisbane 13-18 July, as a case study, the level, nature and relationship of key partners are investigated. Data is triangulated from interviews with organisers of the 2008 WDAGS, on-line and email surveys of delegates, participant observation and analysis of formal and informal documents, to produce a management model suited to this kind of event.

ERTP : Energy-efficient and reliable transport protocol for data streaming in wireless sensor networks

Emerging data streaming applications in Wireless Sensor Networks require reliable and energy-efficient Transport Protocols. Our recent Wireless Sensor Network deployment in the Burdekin delta, Australia, for water monitoring [T. Le Dinh, W. Hu, P. Sikka, P. Corke, L. Overs, S. Brosnan, Design and deployment of a remote robust sensor network: experiences from an outdoor water quality monitoring network, in: Second IEEE Workshop on Practical Issues in Building Sensor Network Applications (SenseApp 2007), Dublin, Ireland, 2007] is one such example. This application involves streaming sensed data such as pressure, water flow rate, and salinity periodically from many scattered sensors to the sink node which in turn relays them via an IP network to a remote site for archiving, processing, and presentation. While latency is not a primary concern in this class of application (the sampling rate is usually in terms of minutes or hours), energy-efficiency is. Continuous long-term operation and reliable delivery of the sensed data to the sink are also desirable. This paper proposes ERTP, an Energy-efficient and Reliable Transport Protocol for Wireless Sensor Networks. ERTP is designed for data streaming applications, in which sensor readings are transmitted from one or more sensor sources to a base station (or sink). ERTP uses a statistical reliability metric which ensures the number of data packets delivered to the sink exceeds the defined threshold. Our extensive discrete event simulations and experimental evaluations show that ERTP is significantly more energyefficient than current approaches and can reduce energy consumption by more than 45% when compared to current approaches. Consequently, sensor nodes are more energy-efficient and the lifespan of the unattended WSN is increased.

A prototype system for the digital lodgement of spatial data

This dissertation develops the model of a prototype system for the digital lodgement of spatial data sets with statutory bodies responsible for the registration and approval of land related actions under the Torrens Title system. Spatial data pertain to the location of geographical entities together with their spatial dimensions and are classified as point, line, area or surface. This dissertation deals with a sub-set of spatial data, land boundary data that result from the activities performed by surveying and mapping organisations for the development of land parcels. The prototype system has been developed, utilising an event-driven paradigm for the user-interface, to exploit the potential of digital spatial data being generated from the utilisation of electronic techniques. The system provides for the creation of a digital model of the cadastral network and dependent data sets for an area of interest from hard copy records. This initial model is calibrated on registered control and updated by field survey to produce an amended model. The field-calibrated model then is electronically validated to ensure it complies with standards of format and content. The prototype system was designed specifically to create a database of land boundary data for subsequent retrieval by land professionals for surveying, mapping and related activities. Data extracted from this database are utilised for subsequent field survey operations without the need to create an initial digital model of an area of interest. Statistical reporting of differences resulting when subsequent initial and calibrated models are compared, replaces the traditional checking operations of spatial data performed by a land registry office. Digital lodgement of survey data is fundamental to the creation of the database of accurate land boundary data. This creation of the database is fundamental also to the efficient integration of accurate spatial data about land being generated by modem technology such as global positioning systems, and remote sensing and imaging, with land boundary information and other information held in Government databases. The prototype system developed provides for the delivery of accurate, digital land boundary data for the land registration process to ensure the continued maintenance of the integrity of the cadastre. Such data should meet also the more general and encompassing requirements of, and prove to be of tangible, longer term benefit to the developing, electronic land information industry.

Procuring Sydney’s Olympic Stadium : a world class event?

This paper describes and analyses the procurement processes employed in delivering the Sydney Olympic Stadium – arguably the most significant stadia project in the region today. This current high profile project is discussed in terms of a case study into the procurement processes used. Interviews, personal site visits and questionnaires were used to obtain information on the procurement processes used and comments on their application to the project. The alternative procurement process used on this project—Design and Construction within a Build, Own, Operate and Transfer (BOOT) project—is likely to impact on the construction industry as a whole. Already other projects and sectors are following this lead. Based on a series of on-site interviews and questionnaires, a series of benefits and drawbacks to this procurement strategy are provided.The Olympic Stadium project has also been further analysed during construction through a Degree of Interaction framework to determine anticipated project success. This analysis investigates project interaction and user satisfaction to provide a comparable rating. A series of questionnaires were used to collect data to calculate the Degree of Interaction and User Satisfaction ratings.

Dynamic texture reconstruction from sparse codes for unusual event detection in crowded scenes

Unusual event detection in crowded scenes remains challenging because of the diversity of events and noise. In this paper, we present a novel approach for unusual event detection via sparse reconstruction of dynamic textures over an overcomplete basis set, with the dynamic texture described by local binary patterns from three orthogonal planes (LBPTOP). The overcomplete basis set is learnt from the training data where only the normal items observed. In the detection process, given a new observation, we compute the sparse coefficients using the Dantzig Selector algorithm which was proposed in the literature of compressed sensing. Then the reconstruction errors are computed, based on which we detect the abnormal items. Our application can be used to detect both local and global abnormal events. We evaluate our algorithm on UCSD Abnormality Datasets for local anomaly detection, which is shown to outperform current state-of-the-art approaches, and we also get promising results for rapid escape detection using the PETS2009 dataset.

Planning volunteer responses to low-volume mass gatherings: do event characteristics predict patient workload?

INTRODUCTION: Workforce planning for first aid and medical coverage of mass gatherings is hampered by limited research. In particular, the characteristics and likely presentation patterns of low-volume mass gatherings of between several hundred to several thousand people are poorly described in the existing literature. OBJECTIVES: This study was conducted to: 1. Describe key patient and event characteristics of medical presentations at a series of mass gatherings, including events smaller than those previously described in the literature; 2. Determine whether event type and event size affect the mean number of patients presenting for treatment per event, and specifically, whether the 1:2,000 deployment rule used by St John Ambulance Australia is appropriate; and 3. Identify factors that are predictive of injury at mass gatherings. METHODS: A retrospective, observational, case-series design was used to examine all cases treated by two Divisions of St John Ambulance (Queensland) in the greater metropolitan Brisbane region over a three-year period (01 January 2002-31 December 2004). Data were obtained from routinely collected patient treatment forms completed by St John officers at the time of treatment. Event-related data (e.g., weather, event size) were obtained from event forms designed for this study. Outcome measures include: total and average number of patient presentations for each event; event type; and event size category. Descriptive analyses were conducted using chi-square tests, and mean presentations per event and event type were investigated using Kruskal-Wallis tests. Logistic regression analyses were used to identify variables independently associated with injury presentation (compared with non-injury presentations). RESULTS: Over the three-year study period, St John Ambulance officers treated 705 patients over 156 separate events. The mean number of patients who presented with any medical condition at small events (less than or equal to 2,000 attendees) did not differ significantly from that of large (>2,000 attendees) events (4.44 vs. 4.67, F = 0.72, df = 1, 154, p = 0.79). Logistic regression analyses indicated that presentation with an injury compared with non-injury was independently associated with male gender, winter season, and sporting events, even after adjusting for relevant variables. CONCLUSIONS: In this study of low-volume mass gatherings, a similar number of patients sought medical treatment at small (<2,000 patrons) and large (>2,000 patrons) events. This demonstrates that for low-volume mass gatherings, planning based solely on anticipated event size may be flawed, and could lead to inappropriate levels of first-aid coverage. This study also highlights the importance of considering other factors, such as event type and patient characteristics, when determining appropriate first-aid resourcing for low-volume events. Additionally, identification of factors predictive of injury presentations at mass gatherings has the potential to significantly enhance the ability of event coordinators to plan effective prevention strategies and response capability for these events.

Event-related brain potentials dissociate the developmental time-course of automatic numerical magnitude analysis and cognitive control functions during the first three years of primary school

In this study we set out to dissociate the developmental time course of automatic symbolic number processing and cognitive control functions in grade 1-3 British primary school children. Event-related potential (ERP) and behavioral data were collected in a physical size discrimination numerical Stroop task. Task-irrelevant numerical information was processed automatically already in grade 1. Weakening interference and strengthening facilitation indicated the parallel development of general cognitive control and automatic number processing. Relationships among ERP and behavioral effects suggest that control functions play a larger role in younger children and that automaticity of number processing increases from grade 1 to 3.

Sustainable Road Management in Texas : network-level flexible pavement structural condition analysis using data mining techniques

The research team recognized the value of network-level Falling Weight Deflectometer (FWD) testing to evaluate the structural condition trends of flexible pavements. However, practical limitations due to the cost of testing, traffic control and safety concerns and the ability to test a large network may discourage some agencies from conducting the network-level FWD testing. For this reason, the surrogate measure of the Structural Condition Index (SCI) is suggested for use. The main purpose of the research presented in this paper is to investigate data mining strategies and to develop a prediction method of the structural condition trends for network-level applications which does not require FWD testing. The research team first evaluated the existing and historical pavement condition, distress, ride, traffic and other data attributes in the Texas Department of Transportation (TxDOT) Pavement Maintenance Information System (PMIS), applied data mining strategies to the data, discovered useful patterns and knowledge for SCI value prediction, and finally provided a reasonable measure of pavement structural condition which is correlated to the SCI. To evaluate the performance of the developed prediction approach, a case study was conducted using the SCI data calculated from the FWD data collected on flexible pavements over a 5-year period (2005 – 09) from 354 PMIS sections representing 37 pavement sections on the Texas highway system. The preliminary study results showed that the proposed approach can be used as a supportive pavement structural index in the event when FWD deflection data is not available and help pavement managers identify the timing and appropriate treatment level of preventive maintenance activities.

Suspended sediment properties and suspended sediment flux estimates in an inundated urban environment during a major flood event

During a major flood event, the inundation of urban environments leads to some complicated flow motion most often associated with significant sediment fluxes. In the present study, a series of field measurements were conducted in an inundated section of the City of Brisbane (Australia) about the peak of a major flood in January 2011. Some experiments were performed to use ADV backscatter amplitude as a surrogate estimate of the suspended sediment concentration (SSC) during the flood event. The flood water deposit samples were predominantly silty material with a median particle size about 25 μm and they exhibited a non-Newtonian behavior under rheological testing. In the inundated urban environment during the flood, estimates of suspended sediment concentration presented a general trend with increasing SSC for decreasing water depth. The suspended sediment flux data showed some substantial sediment flux amplitudes consistent with the murky appearance of floodwaters. Altogether the results highlighted the large suspended sediment loads and fluctuations in the inundated urban setting associated possibly with a non-Newtonian behavior. During the receding flood, some unusual long-period oscillations were observed (periods about 18 min), although the cause of these oscillations remains unknown. The field deployment was conducted in challenging conditions highlighting a number of practical issues during a natural disaster.