Content-aware compression for big textual data analysis

A substantial amount of information on the Internet is present in the form of text. The value of this semi-structured and unstructured data has been widely acknowledged, with consequent scientific and commercial exploitation. The ever-increasing data production, however, pushes data analytic platforms to their limit. This thesis proposes techniques for more efficient textual big data analysis suitable for the Hadoop analytic platform. This research explores the direct processing of compressed textual data. The focus is on developing novel compression methods with a number of desirable properties to support text-based big data analysis in distributed environments. The novel contributions of this work include the following. Firstly, a Content-aware Partial Compression (CaPC) scheme is developed. CaPC makes a distinction between informational and functional content in which only the informational content is compressed. Thus, the compressed data is made transparent to existing software libraries which often rely on functional content to work. Secondly, a context-free bit-oriented compression scheme (Approximated Huffman Compression) based on the Huffman algorithm is developed. This uses a hybrid data structure that allows pattern searching in compressed data in linear time. Thirdly, several modern compression schemes have been extended so that the compressed data can be safely split with respect to logical data records in distributed file systems. Furthermore, an innovative two layer compression architecture is used, in which each compression layer is appropriate for the corresponding stage of data processing. Peripheral libraries are developed that seamlessly link the proposed compression schemes to existing analytic platforms and computational frameworks, and also make the use of the compressed data transparent to developers. The compression schemes have been evaluated for a number of standard MapReduce analysis tasks using a collection of real-world datasets. In comparison with existing solutions, they have shown substantial improvement in performance and significant reduction in system resource requirements.

Modelling and control of a floating oscillating water column

A novel numerical model of a Bent Backwards Duct Buoy (BBDB) Oscillating Water Column (OWC) Wave Energy Converter was created based on existing isolated numerical models of the different energy conversion systems utilised by an OWC. The novel aspect of this numerical model is that it incorporates the interdependencies of the different power conversion systems rather than modelling each system individually. This was achieved by accounting for the dynamic aerodynamic damping caused by the changing turbine rotational velocity by recalculating the turbine damping for each simulation sample and applying it via a feedback loop. The accuracy of the model was validated using experimental data collected during the Components for Ocean Renewable Energy Systems (CORES) EU FP-7 project that was tested in Galway Bay, Ireland. During the verification process, it was discovered that the model could also be applied as a valuable tool when troubleshooting device performance. A new turbine was developed and added to a full scale model after being investigated using Computational Fluid Dynamics. The energy storage capacity of the impulse turbine was investigated by modelling the turbine with both high and low inertia and applying three turbine control theories to the turbine using the full scale model. A single Maximum Power Point Tracking algorithm was applied to the low-inertia turbine, while both a fixed and dynamic control algorithm was applied to the high-inertia turbine. These results suggest that the highinertia turbine could be used as a flywheel energy storage device that could help minimize output power variation despite the low operating speed of the impulse turbine. This research identified the importance of applying dynamic turbine damping to a BBDB OWC numerical model, revealed additional value of the model as a device troubleshooting tool, and found that an impulse turbine could be applied as an energy storage system.