Learning from large data : bias, variance, sampling, and learning curves

One of the fundamental machine learning tasks is that of predictive classification. Given that organisations collect an ever increasing amount of data, predictive classification methods must be able to effectively and efficiently handle large amounts of data. However, it is understood that present requirements push existing algorithms to, and sometimes beyond, their limits since many classification prediction algorithms were designed when currently common data set sizes were beyond imagination. This has led to a significant amount of research into ways of making classification learning algorithms more effective and efficient. Although substantial progress has been made, a number of key questions have not been answered. This dissertation investigates two of these key questions. The first is whether different types of algorithms to those currently employed are required when using large data sets. This is answered by analysis of the way in which the bias plus variance decomposition of predictive classification error changes as training set size is increased. Experiments find that larger training sets require different types of algorithms to those currently used. Some insight into the characteristics of suitable algorithms is provided, and this may provide some direction for the development of future classification prediction algorithms which are specifically designed for use with large data sets. The second question investigated is that of the role of sampling in machine learning with large data sets. Sampling has long been used as a means of avoiding the need to scale up algorithms to suit the size of the data set by scaling down the size of the data sets to suit the algorithm. However, the costs of performing sampling have not been widely explored. Two popular sampling methods are compared with learning from all available data in terms of predictive accuracy, model complexity, and execution time. The comparison shows that sub-sampling generally products models with accuracy close to, and sometimes greater than, that obtainable from learning with all available data. This result suggests that it may be possible to develop algorithms that take advantage of the sub-sampling methodology to reduce the time required to infer a model while sacrificing little if any accuracy. Methods of improving effective and efficient learning via sampling are also investigated, and now sampling methodologies proposed. These methodologies include using a varying-proportion of instances to determine the next inference step and using a statistical calculation at each inference step to determine sufficient sample size. Experiments show that using a statistical calculation of sample size can not only substantially reduce execution time but can do so with only a small loss, and occasional gain, in accuracy. One of the common uses of sampling is in the construction of learning curves. Learning curves are often used to attempt to determine the optimal training size which will maximally reduce execution time while nut being detrimental to accuracy. An analysis of the performance of methods for detection of convergence of learning curves is performed, with the focus of the analysis on methods that calculate the gradient, of the tangent to the curve. Given that such methods can be susceptible to local accuracy plateaus, an investigation into the frequency of local plateaus is also performed. It is shown that local accuracy plateaus are a common occurrence, and that ensuring a small loss of accuracy often results in greater computational cost than learning from all available data. These results cast doubt over the applicability of gradient of tangent methods for detecting convergence, and of the viability of learning curves for reducing execution time in general.

Monitoring the corrosion of concrete reinforcement using control curves

Corrosion testing (half-cell and LPR) was carried out on a number reinforced concrete panels which had been taken from the fascia of a twenty five year old high rise building in Melbourne, Australia. Corrosion, predominantly as a result of carbonation of the concrete, was associated with a limited amount of cracking. A monitoring technique was established in which probe electrodes (reference and counter) were retro-fitted into the concrete. The probe electrode setup was identical for all panels tested. It was found that the corrosion behaviour of all panels tested closely fitted a family of results when the corrosion potential is plotted against the polarisation resistance (Rp). This enabled the development of a so-called 'control curve' relating the corrosion potential to the Rp for all of the panels under investigation. This relationship was also confirmed on laboratory samples, indicating that for a fixed geometry and experimental conditions a relationship between the potential and polarisation resistance of steel can be established for the steel-concrete system. Experimental results will be presented which indicate that for a given monitoring cell geometry, it may be possible to propose criteria for the point at which remediation measures should be considered. The establishment of such a control curve has enabled the development of a powerful monitoring tool for the assessment of a number of proposed corrosion remediation techniques. The actual effect of any corrosion remediation technique becomes clearly apparent via the type and magnitude of deviation of post remediation data from the original (preremediation) control curve.

Effects of replication on measured S-N curves of metals

Replication, or repeated tests at the same stress amplitude, is used to provide statistical confidence in life data during the development of S-N curves. This paper discusses the effects of replication on the measurement of S-N curves and presents an alternative to traditional replication methods for the determination of S-N curves, particularly for the development of preliminary S-N curves. Using specimens made out of the extruded bars of a magnesium alloy, it is demonstrated that the S-N curve estimated using the data from non-replication tests is almost same as that from replication tests. The advantage of using non-replication fatigue tests is that it uses fewer specimens, in this instance, only half of that required for 50% replication fatigue test, to achieve the same estimation as that of the replication fatigue tests. Another advantage of using non-replication fatigue tests is that it can detect the non-linearity using limited specimens.

Effects of replication on measured S-N curves of metals

Replication, or repeated tests at the same stress amplitude, is used to provide statistical confidence in life data during the development of S-N curves. This paper discusses the effects of replication on the measurement of S-N curves and presents an alternative to traditional replication methods for the determination of S-N curves, particularly for the development of preliminary S-N curves. Using specimens made out of the extruded bars of a magnesium alloy, it is demonstrated that the S-N curve estimated using the data from non-replication tests is almost same as that from replication tests. The advantage of using non-replication fatigue tests is that it uses fewer specimens, in this instance, only half of that required for 50% replication fatigue test, to achieve the same estimation as that of the replication fatigue tests. Another advantage of using non-replication fatigue tests is that it can detect the non-linearity using limited specimens.

A comparison of two methods in acquiring stimulus-response curves with transcranial magnetic stimulation

Background
The stimulus–response (S–R) curve is a well accepted constituent in transcranial magnetic stimulation (TMS) studies. However, it has been suggested that parameters of the S–R curve differ when stimuli are provided in a “ramped” (measured steps from low to high intensity), or “random” fashion.

Hypothesis
We hypothesized that there would be no difference in the parameters of the S–R curve between either methodologies.

Methods
Using a randomised cross-over design, 10 healthy participants (29.6 ± 6.4 yrs, 3 f) completed “ramped” or “random” curves in biceps brachii (BB) and first dorsal interosseous (FDI) muscles of both limbs. Curves were compared using mixed-factor ANOVA and correlated between limbs and methodologies.

Results
No differences (P > 0.05) and high correlations (range 0.71–0.97; P < 0.001) were observed in BB and FDI data between curves.

Conclusions
This study demonstrated that either methodology provides similar parameters of the S–R curve in healthy participants.

Effects of time since fire on birds : how informative are generalized fire response curves for conservation management?

Fire is both a widespread natural disturbance that affects the distribution of species and a tool that can be used to manage habitats for species. Knowledge of temporal changes in the occurrence of species after fire is essential for conservation management in fire-prone environments. Two key issues are: whether postfire responses of species are idiosyncratic or if multiple species show a limited number of similar responses; and whether such responses to time since fire can predict the occurrence of species across broad spatial scales. We examined the response of bird species to time since fire in semiarid shrubland in southeastern Australia using data from surveys at 499 sites representing a 100-year chronosequence. We used nonlinear regression to model the probability of occurrence of 30 species with time since fire in two vegetation types, and compared species' responses with generalized response shapes from the literature. The occurrence of 16 species was significantly influenced by time since fire: they displayed six main responses consistent with generalized response shapes. Of these 16 species, 15 occurred more frequently in mid- or later-successional vegetation (>20 years since fire), and only one species occurred more often in early succession (<5 years since fire). The models had reasonable predictive ability for eight species, some predictive ability for seven species, and were little better than random for one species. Bird species displayed a limited range of responses to time since fire; thus a small set of fire ages should allow the provision of habitat for most species. Postfire successional changes extend for decades and management of the age class distribution of vegetation will need to reflect this timescale. Response curves revealed important seral stages for species and highlighted the importance of mid- to late-successional vegetation (>20 years). Although time since fire clearly influences the distribution of numerous bird species, predictive models of the spatial distribution of species in fire-prone landscapes need to incorporate other factors in addition to time since fire.

Shifting curves? Trends in thinness and obesity among Australian youth, 1985 to 2010

Objective To describe 25-year trends in the prevalence of ≤Grade 2 thinness and obesity among Australian children by sex, age and socioeconomic (SES) background.

Methods Cross-sectional surveys of New South Wales school-aged children aged 6.0–16.9 years conducted in 1985–1997–2004–2010 (n = 19 434). Height/weight were measured, and thinness and obesity were defined by international standards. SES was derived from children's residential postcode using the Australian Bureau of Statistics' Index of Relative Socioeconomic Disadvantage, most proximal to the survey year.

Results Since 1985, the prevalence of thinness has not varied by survey year. Age was not associated with thinness; however, thinness was lower among middle SES boys, compared with high SES (OR: 0.45, 95%CI: 0.21, 0.97). The prevalence of obesity trebled between 1985 and 1997 (1.7% vs. 5.1% P = 0.000); however, since 1997, obesity prevalence has not significantly changed. Since 1997, obesity was higher among younger compared with older girls (OR: 2.11, 95%CI: 1.48, 3.00) and SES was inversely associated with obesity in boys (OR: 2.05, 95%CI: 1.44, 2.92) and girls (OR: 1.86, 95%CI: 1.27, 2.74).

Conclusions The apparent plateau in child obesity is a welcome finding; however, the SES gradients are of concern. If the obesity stabilization is associated with the impact of multiple lifestyle behavioural interventions, the findings suggest obesity programmes have done ‘no harm’, but potentially the dose/delivery of interventions has not been sufficient or appropriate to reduce child obesity levels.

Individual variation in thermal performance curves: swimming burst speed and jumping endurance in wild-caught tropical clawed frogs

The importance of studying individual variation in locomotor performance has long been recognized as it may determine the ability of an organism to escape from predators, catch prey or disperse. In ectotherms, locomotor performance is highly influenced by ambient temperature (Ta), yet several studies have showed that individual differences are usually retained across a Ta gradient. Less is known, however, about individual differences in thermal sensitivity of performance, despite the fact that it could represent adaptive sources of phenotypic variation and/or additional substrate for selection to act upon. We quantified swimming and jumping performance in 18 wild-caught tropical clawed frogs (Xenopus tropicalis) across a Ta gradient. Maximum swimming velocity and acceleration were not repeatable and individuals did not differ in how their swimming performance varied across Ta. By contrast, time and distance jumped until exhaustion were repeatable across the Ta gradient, indicating that individuals that perform best at a given Ta also perform best at another Ta. Moreover, thermal sensitivity of jumping endurance significantly differed among individuals, with individuals of high performance at low Ta displaying the highest sensitivity to Ta. Individual differences in terrestrial performance increased with decreasing Ta, which is opposite to results obtained in lizards at the inter-specific and among-individual levels. To verify the generality of these patterns, we need more studies on individual variation in thermal reaction norms for locomotor performance in lizards and frogs.

Curriculum as colour and curves: a synthesis of Black theory, design and creativity realised as critical curriculum writing

This article looks to three inspirational Black women, bell hooks, Stacey McBride-Irby and Patricia Williams, in the pursuit of radical curriculum. While today curriculum is critiqued as racialised, gendered, sexualised and classed, the formats of curriculum documents such as text books, units of work and lesson plans have changed little. These documents are often conceived as linear sequences of steps leading to outcomes, and their voices are distanced and ‘neutral’. Drawing on a doctoral study of curriculum design in Australia, this article embraces a different approach by opening up a unit of work on girls’ popular culture to hooks’ invocations to teach to transgress, so that curriculum might be experienced as colour and curves, rather than a monochrome route to a pre-determined end point. Through this, along with hooks, I invite teachers to live pedagogy, rather than to deliver it.

Curriculum as colour and curves: a synthesis of black theory, design and creativity realised as critical curriculum writing

This article looks to three inspirational Black women, bell hooks, Stacey McBride-Irby and Patricia Williams, in the pursuit of radical curriculum. While today curriculum is critiqued as racialized, gendered, sexualised and classed, the formats of curriculum documents such as text books, units of work and lesson plans have changed little. These documents are often conceived as linear sequences of steps leading to outcomes, and their voices are distanced and “neutral”. Drawing on a doctoral study of curriculum design in Australia, this article embraces a different approach by opening up a unit of work on girls’ popular culture to hooks’ invocations to teach to transgress, so that curriculum might be experienced as colour and curves, rather than a monochrome route to a pre-determined end point. Through this, along with hooks, I invite teachers to live pedagogy, rather than to deliver it.