Scaling up data mining techniques to large datasets using parallel and distributed processing

Advances in hardware and software technology enable us to collect, store and distribute large quantities of data on a very large scale. Automatically discovering and extracting hidden knowledge in the form of patterns from these large data volumes is known as data mining. Data mining technology is not only a part of business intelligence, but is also used in many other application areas such as research, marketing and financial analytics. For example medical scientists can use patterns extracted from historic patient data in order to determine if a new patient is likely to respond positively to a particular treatment or not; marketing analysts can use extracted patterns from customer data for future advertisement campaigns; finance experts have an interest in patterns that forecast the development of certain stock market shares for investment recommendations. However, extracting knowledge in the form of patterns from massive data volumes imposes a number of computational challenges in terms of processing time, memory, bandwidth and power consumption. These challenges have led to the development of parallel and distributed data analysis approaches and the utilisation of Grid and Cloud computing. This chapter gives an overview of parallel and distributed computing approaches and how they can be used to scale up data mining to large datasets.

Is Energy Efficiency Priced in the Housing Market? Some Evidence from the United Kingdom.

This paper investigates whether energy performance ratings, as measured by mandatory Energy Performance Certificates (EPCs), are reflected in the sale prices of residential properties. This is the first large-scale empirical study of this topic in the UK involving approximately 400,000 dwellings in the period from 1995 to 2011. Applying hedonic regression and an augmented repeat sales regression, we find a positive relationship between the energy efficiency rating of a dwelling and the transaction price per square metre. The price effects of superior energy performance tend to be higher for terraced dwellings and flats compared to detached and semi-detached dwellings. The evidence is less clear-cut for house price growth rates but remains supportive of an overall positive association. Overall, the results of this study appear to support the hypothesis that energy efficiency levels are reflected in UK house prices, at least in recent years.

Non-uniform data distribution for communication-efficient parallel clustering

Global communicationrequirements andloadimbalanceof someparalleldataminingalgorithms arethe major obstacles to exploitthe computational power of large-scale systems. This work investigates how non-uniform data distributions can be exploited to remove the global communication requirement and to reduce the communication costin parallel data mining algorithms and, in particular, in the k-means algorithm for cluster analysis. In the straightforward parallel formulation of the k-means algorithm, data and computation loads are uniformly distributed over the processing nodes. This approach has excellent load balancing characteristics that may suggest it could scale up to large and extreme-scale parallel computing systems. However, at each iteration step the algorithm requires a global reduction operationwhichhinders thescalabilityoftheapproach.Thisworkstudiesadifferentparallelformulation of the algorithm where the requirement of global communication is removed, while maintaining the same deterministic nature ofthe centralised algorithm. The proposed approach exploits a non-uniform data distribution which can be either found in real-world distributed applications or can be induced by means ofmulti-dimensional binary searchtrees. The approachcanalso be extended to accommodate an approximation error which allows a further reduction ofthe communication costs. The effectiveness of the exact and approximate methods has been tested in a parallel computing system with 64 processors and in simulations with 1024 processing element

Parallel pipelined histogram architecture via c-slow retiming

A parallel pipelined array of cells suitable for realtime computation of histograms is proposed. The cell architecture builds on previous work to now allow operating on a stream of data at 1 pixel per clock cycle. This new cell is more suitable for interfacing to camera sensors or to microprocessors of 8-bit data buses which are common in consumer digital cameras. Arrays using the new proposed cells are obtained via C-slow retiming techniques and can be clocked at a 65% faster frequency than previous arrays. This achieves over 80% of the performance of two-pixel per clock cycle parallel pipelined arrays.

A parallel quantum histogram architecture

A parallel formulation of an algorithm for the histogram computation of n data items using an on-the-fly data decomposition and a novel quantum-like representation (QR) is developed. The QR transformation separates multiple data read operations from multiple bin update operations thereby making it easier to bind data items into their corresponding histogram bins. Under this model the steps required to compute the histogram is n/s + t steps, where s is a speedup factor and t is associated with pipeline latency. Here, we show that an overall speedup factor, s, is available for up to an eightfold acceleration. Our evaluation also shows that each one of these cells requires less area/time complexity compared to similar proposals found in the literature.

High yield production of a soluble bifidobacterial β-galactosidase (BbgIV) inE. coliDH5α with improved catalytic efficiency for the synthesis of prebiotic galactooligosaccharides

The bifidobacterial β-galactosidase (BbgIV) was produced in E. coli DH5α at 37 and 30 °C in a 5 L bioreactor under varied conditions of dissolved oxygen (dO2) and pH. The yield of soluble BbgIV was significantly (P < 0.05) increased once the dO2 dropped to 0–2% and remained at such low values during the exponential phase. Limited dO2 significantly (P < 0.05) increased the plasmid copy number and decreased the cells growth rate. Consequently, the BbgIV yield increased to its maximum (71–75 mg per g dry cell weight), which represented 20–25% of the total soluble proteins in the cells. In addition, the specific activity and catalytic efficiency of BbgIV were significantly (P < 0.05) enhanced under limited dO2 conditions. This was concomitant with a change in the enzyme secondary structure, suggesting a link between the enzyme structure and function. The knowledge generated from this work is very important for producing BbgIV as a biocatalyst for the development of a cost-effective process for the synthesis of prebiotic galactooligosaccharides from lactose.

Is intrinsic energy efficiency reflected in the pricing of office leases?

This paper investigates whether the intrinsic energy efficiency rating of an office building has a significant impact on its rental value. A sample of 817 transactions for offices with Energy Performance Certificates (EPCs) in the UK is used to assess whether a pricing differential can be identified, depending on the energy rating. While previous analyses of this topic have typically relied on appraisal-based and/or asking rent data, the dataset used in this research contains actual contract rents as well as information on lease terms. The results indicate a significant rental premium for energy-efficient buildings. However, it is found that this premium appears to be mainly driven by the youngest cohort of state-of-the-art energy-efficient buildings. The results also show that tenants of more energy-efficient buildings tend to pay a lower service charge, but this link appears to be rather weak and limited to newer buildings. Hence, it is argued that the information contained in the EPC is still not fully taken into account in the UK commercial property market with the possible exception of both the highest and the lowest EPC ratings.

Rigorous bounds on the nonlinear saturation of instabilities to parallel shear flows

A novel method is presented for obtaining rigorous upper bounds on the finite-amplitude growth of instabilities to parallel shear flows on the beta-plane. The method relies on the existence of finite-amplitude Liapunov (normed) stability theorems, due to Arnol'd, which are nonlinear generalizations of the classical stability theorems of Rayleigh and Fjørtoft. Briefly, the idea is to use the finite-amplitude stability theorems to constrain the evolution of unstable flows in terms of their proximity to a stable flow. Two classes of general bounds are derived, and various examples are considered. It is also shown that, for a certain kind of forced-dissipative problem with dissipation proportional to vorticity, the finite-amplitude stability theorems (which were originally derived for inviscid, unforced flow) remain valid (though they are no longer strictly Liapunov); the saturation bounds therefore continue to hold under these conditions.

An exact local conservation theorem for finite-amplitude disturbances to non-parallel shear flows, with remarks on Hamiltonian structure and on Arnol'd's stability theorems

Disturbances of arbitrary amplitude are superposed on a basic flow which is assumed to be steady and either (a) two-dimensional, homogeneous, and incompressible (rotating or non-rotating) or (b) stably stratified and quasi-geostrophic. Flow over shallow topography is allowed in either case. The basic flow, as well as the disturbance, is assumed to be subject neither to external forcing nor to dissipative processes like viscosity. An exact, local ‘wave-activity conservation theorem’ is derived in which the density A and flux F are second-order ‘wave properties’ or ‘disturbance properties’, meaning that they are O(a2) in magnitude as disturbance amplitude a [rightward arrow] 0, and that they are evaluable correct to O(a2) from linear theory, to O(a3) from second-order theory, and so on to higher orders in a. For a disturbance in the form of a single, slowly varying, non-stationary Rossby wavetrain, $\overline{F}/\overline{A}$ reduces approximately to the Rossby-wave group velocity, where (${}^{-}$) is an appropriate averaging operator. F and A have the formal appearance of Eulerian quantities, but generally involve a multivalued function the correct branch of which requires a certain amount of Lagrangian information for its determination. It is shown that, in a certain sense, the construction of conservable, quasi-Eulerian wave properties like A is unique and that the multivaluedness is inescapable in general. The connection with the concepts of pseudoenergy (quasi-energy), pseudomomentum (quasi-momentum), and ‘Eliassen-Palm wave activity’ is noted. The relationship of this and similar conservation theorems to dynamical fundamentals and to Arnol'd's nonlinear stability theorems is discussed in the light of recent advances in Hamiltonian dynamics. These show where such conservation theorems come from and how to construct them in other cases. An elementary proof of the Hamiltonian structure of two-dimensional Eulerian vortex dynamics is put on record, with explicit attention to the boundary conditions. The connection between Arnol'd's second stability theorem and the suppression of shear and self-tuning resonant instabilities by boundary constraints is discussed, and a finite-amplitude counterpart to Rayleigh's inflection-point theorem noted

Influence of earthworm Eisenia fetida on removal efficiency of N and P in vertical flow constructed wetland.

This study investigates biomass, density, photosynthetic activity, and accumulation of nitrogen (N) and phosphorus (P) in three wetland plants (Canna indica, Typha augustifolia, and Phragmites austrail) in response to the introduction of the earthworm Eisenia fetida into a constructed wetland. The removal efficiency of N and P in constructed wetlands were also investigated. Results showed that the photosynthetic rate (P n), transpiration rate (T r), and stomatal conductance (S cond) of C. indica and P. austrail were (p < 0.05) significantly higher when earthworms were present. The addition of E. fetida increased the N uptake value by above-ground of C. indica, T. augustifolia, and P. australis by 185, 216, and 108 %, respectively; and its P uptake value increased by 300, 355, and 211 %, respectively. Earthworms could enhance photosynthetic activity, density, and biomass of wetland plants in constructed wetland, resulting in the higher N and P uptake. The addition of E. fetida into constructed wetland increased the removal efficiency of TN and TP by 10 and 7 %, respectively. The addition of earthworms into vertical flow constructed wetland increased the removal efficiency of TN and TP, which was related to higher photosynthetic activity and N and P uptake. The addition of earthworms into vertical flow constructed wetland and plant harvests could be the significantly sustainable N and P removal strategy

Runge-Kutta IMEX schemes for the Horizontally Explicit/Vertically Implicit (HEVI) solution of wave equations

Many operational weather forecasting centres use semi-implicit time-stepping schemes because of their good efficiency. However, as computers become ever more parallel, horizontally explicit solutions of the equations of atmospheric motion might become an attractive alternative due to the additional inter-processor communication of implicit methods. Implicit and explicit (IMEX) time-stepping schemes have long been combined in models of the atmosphere using semi-implicit, split-explicit or HEVI splitting. However, most studies of the accuracy and stability of IMEX schemes have been limited to the parabolic case of advection–diffusion equations. We demonstrate how a number of Runge–Kutta IMEX schemes can be used to solve hyperbolic wave equations either semi-implicitly or HEVI. A new form of HEVI splitting is proposed, UfPreb, which dramatically improves accuracy and stability of simulations of gravity waves in stratified flow. As a consequence it is found that there are HEVI schemes that do not lose accuracy in comparison to semi-implicit ones. The stability limits of a number of variations of trapezoidal implicit and some Runge–Kutta IMEX schemes are found and the schemes are tested on two vertical slice cases using the compressible Boussinesq equations split into various combinations of implicit and explicit terms. Some of the Runge–Kutta schemes are found to be beneficial over trapezoidal, especially since they damp high frequencies without dropping to first-order accuracy. We test schemes that are not formally accurate for stiff systems but in stiff limits (nearly incompressible) and find that they can perform well. The scheme ARK2(2,3,2) performs the best in the tests.

Improving EU biofuels policy? Greenhouse gas emissions, policy efficiency, and WTO compatibility

Both the EU’s Renewable Energy Directive (RED) and Article 7a of its Fuel Quality Directive (FQD) seek to reduce greenhouse gas (GHG) emissions from transport fuels. The RED mandates a 10% share of renewable energy in transport fuels by 2020, whilst the FQD requires a 6% reduction in GHG emissions (from a 2010 base) by the same date. In practice, it will mainly be biofuels that economic operators will use to meet these requirements, but the different approaches can lead to either the RED, or the FQD, acting as the binding constraint. A common set of environmental sustainability criteria apply to biofuels under both the RED and the FQD. In particular, biofuels have to demonstrate a 35% (later increasing to 50/60%) saving in life-cycle GHG emissions. This could be problematic in the World Trade Organization (WTO), as a non-compliant biofuel with a 34% emissions saving would probably be judged to be ‘like’ a compliant biofuel. A more economically rational way to reduce GHG emissions, and one that might attract greater public support, would be for the RED to reward emission reductions along the lines of the FQD. Moreover, this modification would probably make the provisions more acceptable in the WTO, as there would be a clearer link between policy measures and the objective of reductions in GHG emissions; and the combination of the revised RED and the FQD would lessen the commercial incentive to import biofuels with modest GHG emission savings, and thus reduce the risk of trade tension.

C-slow retimed parallel histogram architectures for consumer imaging devices

A parallel pipelined array of cells suitable for real-time computation of histograms is proposed. The cell architecture builds on previous work obtained via C-slow retiming techniques and can be clocked at 65 percent faster frequency than previous arrays. The new arrays can be exploited for higher throughput particularly when dual data rate sampling techniques are used to operate on single streams of data from image sensors. In this way, the new cell operates on a p-bit data bus which is more convenient for interfacing to camera sensors or to microprocessors in consumer digital cameras.

Precision placement of fertiliser for optimising the early nutrition of vegetable crops: a review of the implications for the yield and quality of crops, and their nutrient use efficiency

The research outlined in this paper highlights the importance of the early nutrition of vegetable crops, and its long-term effects on their subsequent growth and development. Results are also presented to demonstrate how the nutrient supply during the establishment stages of young seedlings and transplants can be enhanced by targeting fertiliser to a zone close to their developing roots. Three different precision fertiliser placement techniques are compared for this purpose: starter, band or side-injected fertiliser. The use of each of these methods consistently produced the same (or greater) yields at lower application rates than those from conventional broadcast applications, increasing the apparent recovery of N, P and K, and the overall efficiency of nutrient use, while reducing the levels of residual nutrients in the soil. Starter fertilisers also advanced the maturity of some crops, and enhanced produce quality by increasing the proportions of the larger and/or more desirable marketable grades. The benefits of the different placement techniques are illustrated with selected examples from research at Warwick HRI using different vegetable crops, including lettuce, onion and carrot.