Content-aware compression for big textual data analysis

A substantial amount of information on the Internet is present in the form of text. The value of this semi-structured and unstructured data has been widely acknowledged, with consequent scientific and commercial exploitation. The ever-increasing data production, however, pushes data analytic platforms to their limit. This thesis proposes techniques for more efficient textual big data analysis suitable for the Hadoop analytic platform. This research explores the direct processing of compressed textual data. The focus is on developing novel compression methods with a number of desirable properties to support text-based big data analysis in distributed environments. The novel contributions of this work include the following. Firstly, a Content-aware Partial Compression (CaPC) scheme is developed. CaPC makes a distinction between informational and functional content in which only the informational content is compressed. Thus, the compressed data is made transparent to existing software libraries which often rely on functional content to work. Secondly, a context-free bit-oriented compression scheme (Approximated Huffman Compression) based on the Huffman algorithm is developed. This uses a hybrid data structure that allows pattern searching in compressed data in linear time. Thirdly, several modern compression schemes have been extended so that the compressed data can be safely split with respect to logical data records in distributed file systems. Furthermore, an innovative two layer compression architecture is used, in which each compression layer is appropriate for the corresponding stage of data processing. Peripheral libraries are developed that seamlessly link the proposed compression schemes to existing analytic platforms and computational frameworks, and also make the use of the compressed data transparent to developers. The compression schemes have been evaluated for a number of standard MapReduce analysis tasks using a collection of real-world datasets. In comparison with existing solutions, they have shown substantial improvement in performance and significant reduction in system resource requirements.

Complexity as key to designing cognitive-friendly environments for older people

The lived environment is the arena where our cognitive skills, preferences, and attitudes come together to determine our ability to interact with the world. The mechanisms through which lived environments can benefit cognitive health in older age are yet to be fully understood. The existing literature suggests that environments which are perceived as stimulating, usable and aesthetically appealing can improve or facilitate cognitive performance both in young and older age. Importantly, optimal stimulation for cognition seems to depend on experiencing sufficiently stimulating environments while not too challenging. Environmental complexity is an important contributor to determining whether an environment provides such an optimal stimulation. The present paper reviews a selection of studies which have explored complexity in relation to perceptual load, environmental preference and perceived usability to propose a framework which explores direct and indirect environmental influences on cognition, and to understand these influences in relation to aging processes. We identify ways to define complexity at different environmental scales, going from micro low-level perceptual features of scenes, to design qualities of proximal environments (e.g., streets, neighborhoods), to broad geographical areas (i.e., natural vs. urban environments). We propose that studying complexity at these different scales will provide new insight into the design of cognitive-friendly environments.