Learning from large data : bias, variance, sampling, and learning curves

One of the fundamental machine learning tasks is that of predictive classification. Given that organisations collect an ever increasing amount of data, predictive classification methods must be able to effectively and efficiently handle large amounts of data. However, it is understood that present requirements push existing algorithms to, and sometimes beyond, their limits since many classification prediction algorithms were designed when currently common data set sizes were beyond imagination. This has led to a significant amount of research into ways of making classification learning algorithms more effective and efficient. Although substantial progress has been made, a number of key questions have not been answered. This dissertation investigates two of these key questions. The first is whether different types of algorithms to those currently employed are required when using large data sets. This is answered by analysis of the way in which the bias plus variance decomposition of predictive classification error changes as training set size is increased. Experiments find that larger training sets require different types of algorithms to those currently used. Some insight into the characteristics of suitable algorithms is provided, and this may provide some direction for the development of future classification prediction algorithms which are specifically designed for use with large data sets. The second question investigated is that of the role of sampling in machine learning with large data sets. Sampling has long been used as a means of avoiding the need to scale up algorithms to suit the size of the data set by scaling down the size of the data sets to suit the algorithm. However, the costs of performing sampling have not been widely explored. Two popular sampling methods are compared with learning from all available data in terms of predictive accuracy, model complexity, and execution time. The comparison shows that sub-sampling generally products models with accuracy close to, and sometimes greater than, that obtainable from learning with all available data. This result suggests that it may be possible to develop algorithms that take advantage of the sub-sampling methodology to reduce the time required to infer a model while sacrificing little if any accuracy. Methods of improving effective and efficient learning via sampling are also investigated, and now sampling methodologies proposed. These methodologies include using a varying-proportion of instances to determine the next inference step and using a statistical calculation at each inference step to determine sufficient sample size. Experiments show that using a statistical calculation of sample size can not only substantially reduce execution time but can do so with only a small loss, and occasional gain, in accuracy. One of the common uses of sampling is in the construction of learning curves. Learning curves are often used to attempt to determine the optimal training size which will maximally reduce execution time while nut being detrimental to accuracy. An analysis of the performance of methods for detection of convergence of learning curves is performed, with the focus of the analysis on methods that calculate the gradient, of the tangent to the curve. Given that such methods can be susceptible to local accuracy plateaus, an investigation into the frequency of local plateaus is also performed. It is shown that local accuracy plateaus are a common occurrence, and that ensuring a small loss of accuracy often results in greater computational cost than learning from all available data. These results cast doubt over the applicability of gradient of tangent methods for detecting convergence, and of the viability of learning curves for reducing execution time in general.

Reverse ascertainment bias in microsatellite allelic diversity in owls (Aves, Strigiformes)

A rich source of markers may be overlooked by screening for polymorphism in the source species only. We screened 129 microsatellite loci isolated from the powerful owl (Ninox strenua) against two closely related species; Ninox  connivens and Ninox novaeseelandiae. From the screening effort 20 polymorphic markers were isolated, including six loci which were originally discarded as they were monomorphic in the source species. Further cross-species amplification of all 20 loci across species from two families, Strigidae and Tytonidae, revealed unusually high levels of polymorphism within closely related species, and limited success within phylogenetically distant species. Routine screening of multiple  species during the marker development phase can yield a wider range of  polymorphic markers which can subsequently enhance cross-species  amplification attempts.

Breeding ecology and bias in offspring sex ratio in little grassbirds (Megalurus gramineus)

Little grassbirds (Megalurus gramineus) are small, sexually monomorphic passerines that live in reed beds, lignum swamps and salt marshes in southern Australia. The breeding biology and patterns of sex allocation of the little grassbird were investigated over a single breeding season. Our observations of this species in the Edithvale Wetland Reserve revealed a highly male-biased population sex ratio, with some breeding territories containing several additional males. Nevertheless, there was little compelling evidence that little grassbirds breed cooperatively. The growth rates of male and female nestlings were similar and, as predicted by theory, there was no overall primary sex ratio bias. However, the primary sex ratio was female-biased early in the breeding season and became increasingly male-biased later in the breeding season.

Reducing performance bias by intrinsically insensitive learning for unbalanced text mining

This thesis proposes three effective strategies to solve the significant performance-bias problem in imbalance text mining: (1) creation of a novel inexact field learning algorithm to overcome the dual-imbalance problem; (2) introduction of the one-class classification-framework to optimize classifier-parameters, and (3) proposal of a maximal-frequent-item-set discovery approach to achieve higher accuracy and efficiency.

Muddy ovals under threat from climate change

Questions are raised about the wisdom of continuing the policy of unending global economic growth in the face of climate change. Alternate forms of economic organisation need to be devised, without which global warming may even lead to the elimination of cold, muddy football fields.

The effect of potential bias on the formation of ionic liquid generated surface films on Mg alloys

An electrochemical approach to the formation of a protective surface film on Mg alloys immersed in the ionic liquid (IL), trihexyl(tetradecyl)phosphonium–bis 2,4,4-trimethylpentylphosphinate, was investigated in this work. Initially, cyclic voltammetry was used with the Mg alloy being cycled from OCP to more anodic potentials. EIS data indicate that, under these circumstances, an optimum level of protection was achieved at intermediate potentials (e.g., 0 or 0.25 V versus Ag/AgCl). In the second part of this paper, a small constant bias was applied to the Mg alloy immersed in the IL for extended periods using a novel cell design. This electrochemical cell allowed us to monitor in situ surface film formation on the metal surface as well as the subsequent corrosion behaviour of the metal in a corrosive medium. This apparatus was used to investigate the evolution of the surface film on an AZ31 magnesium alloy under a potential bias (between ±100 mV versus open circuit) applied for over 24 h, and the film evolution was monitored using electrochemical impedance spectroscopy (EIS). A film resistance was determined from the EIS data and it was shown that this increased substantially during the first few hours (independent of the bias potential used) with a subsequent decrease upon longer exposure of the surface to the IL. Preliminary characterization of the film formed on the Mg alloy surface using ToF-SIMS indicates that a multilayer surface exists with a phosphorous rich outer layer and a native oxide/hydroxide film underlying this. The corrosion performance of a treated AZ31 specimen when exposed to 0.1 M NaCl aqueous solution showed considerable improvement, consistent with electrochemical data.

Extreme response bias in measuring susceptibility to smoking.

Pierce, Choi, Gilpin, Farkas, and Berry (1998) were the first to claim that they could provide causal evidence that tobacco industry advertising and promotion caused adolescent smoking. This claim continues to significantly influence the theory and conceptualization of how youth react to tobacco marketing. The Pierce et al. (1998) methodology has been used by many researchers to establish the influence of tobacco marketing on adolescent smoking (Goldberg, 2003; NCI, 2006; Sargent, Dalton, & Beach, 2000). Pierce et al. (1998) selected respondents for only the second of their two survey longitudinal study because they chose the extreme-negative response. This choice could be the result of the tendency of some significant number of sample members exhibiting extreme-response bias. The results from an analysis of several questions from the original data used by Pierce et al. (1998) has suggested that there is a significant extreme-response style pattern in the Pierce et al. data. This unaccounted for bias in the responses of their sample was due to the procedure used by Pierce et al. (1998) in the selection of their respondents. The Pierce et al. (1998) sample selection procedure requires more research before the causal link can be claimed.

Ecological correlates of the threat of extinction in neotropical bird species

Predicting the threat of extinction aids efficient distribution of conservation resources. This paper utilises a comparative macroecological approach to investigate the threat of extinction in Neotropical birds. Data on ecological variables for 1708 species are analysed using stepwise regression to produce minimum adequate models, first using raw species values and then using independent contrasts (to control for phylogenetic effects). The models differ, suggesting phylogeny has significant effects. The raw species analysis reveals that number of zoogeographical regions occupied, elevational range and utilisation of specialised microhabitats were negatively associated with threat, while minimum elevation and body mass were positively associated, whereas the independent contrasts analysis only identifies zoogeographical regions as important. Confining the analysis to the 582 species restricted to a single zoogeographical region reveals elevational range and number of habitats occupied to be negatively correlated with threat whether the analysis is based on the raw data or on independent contrasts. Analysis of four contrasting zoogeographical regions highlights regional variation in the models. In two Andean regions the threat of extinction declines as the elevation range across which the species occurs increases. In the presence of substantial human populations on high Andean plateaus, a species with a greater elevational range may be more likely to persist at some (relatively) unsettled altitudes. In Central South America, the strongest predictor of threat is minimum elevation of occurrence: species with a lower minimum are less threatened. The minimum elevation result suggests that lowland species experiencing an ecological limit to their minimum elevation (min. elevation >0 m) may be more at risk than those not experiencing such a limit (min. elevation = 0 m). Finally, in southern Amazonia, where there is little altitudinal variation, the only weak predictors of threat are body size, larger species being more threatened, and number of habitats, species occupying more habitats being less threatened. These contrasting results emphasise the importance of undertaking extinction risk analyses at an appropriate geographical scale. Since the models explained only a low percentage of total variance in the data, the effects of human-mediated habitat disturbance across a wide range of habitats may be important.

Exploring negativity bias in brand beliefs and stated brand switching propensity

Negativity bias has been well studied by psychologists but limited research has been conducted on it in a marketing context. Given previous research, this exploratory study aims to examine whether there are any negativity bias effects in brand beliefs and whether there is any influence on stated brand switching propensity amongst current users of a brand. The results suggest that there is a negativity bias evident in brand image data.

The inclusion of ‘then-test’ questions in post-test questionnaires alters post-test responses: a randomized study of bias in health program evaluation

Objectives Program evaluations are frequently based on ‘then-test’ data, i.e., pre-test collected in retrospect. While the application of the then-test has practical advantages, little is known about the validity of then-test data. Because of the collection of then-test in close proximity to post-test questions, this study was aimed at exploring whether the presence of then-test questions in post-test questionnaires influenced subjects’ responses to post-test.
Patients and methods To test the influence of then-test questions, we designed a randomized three-group study in the context of chronic disease self-management programs. Interventions had comparable goals and philosophies, and all 949 study participants filled out identical Health Education Impact Questionnaires (heiQ) at pre-test. At post-test, participants were then randomized to one of the following three groups: Group A responded to post-test questions only (n = 331); Group B filled out transition questions in addition to post-test (n = 304); and Group C filled out then-test questions in addition to post-test (n = 314).
Results Significant post-test differences were found in six of eight heiQ scales, with respondents who filled out then test questions reporting significantly higher post-test scores than respondents of the other groups.
Conclusions This study provides evidence that the inclusion of then-test questions alters post-test responses,
suggesting that change scores based on then-test data be interpreted with care.

Is Web 2.0 a threat to representative democracy? A deliberation through the Australian carbon tax debate

The influence of social media is intensifying in global societies. As the technologies become cheaper and the acceptance of Web 2.0 becomes widespread, the power of social media on citizens, particularly the integrated influence of Facebook, Twitter, YouTube and blogs cannot be underestimated. In this paper, we attempt a deliberation through the lens of carbon tax debate in Australia where the influence of social media has perhaps begun to portend the role of elected representation in this representative democracy.