Boosting the margin: A new explanation for the effectiveness of voting methods

One of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated classifier usually does not increase as its size becomes very large, and often is observed to decrease even after the training error reaches zero. In this paper, we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule, where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label. We show that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. We also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples. Finally, we compare our explanation to those based on the bias-variance decomposition.

Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Exponentiated gradient algorithms for large-margin structured classification

We consider the problem of structured classification, where the task is to predict a label y from an input x, and y has meaningful internal structure. Our framework includes supervised training of Markov random fields and weighted context-free grammars as special cases. We describe an algorithm that solves the large-margin optimization problem defined in [12], using an exponential-family (Gibbs distribution) representation of structured objects. The algorithm is efficient—even in cases where the number of labels y is exponential in size—provided that certain expectations under Gibbs distributions can be calculated efficiently. The method for structured labels relies on a more general result, specifically the application of exponentiated gradient updates [7, 8] to quadratic programs.

Margin-adaptive model selection in statistical learning

A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one. Requiring this “strong margin adaptivity” makes the model selection problem more challenging. We first prove, in a general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.

From the margin to the centre : towards an art strategy for the ethnic enclaves in the city

Sunnybank represents a distinctly Australian take on the classic ‘Chinatown’ – or indeed other ethic community enclaves such as ‘little Italy’, ‘little Bombay’, ‘little Athens’ and so on. In the Northern Hemisphere these tended to grow up in the dense working class neighbourhoods of industrial cities, especially in port cities like Liverpool, London, New York and San Francisco. The existing Chinatowns of Sydney and Melbourne, and to some extent Brisbane’s Fortitude Valley, are of this variety. In the late 1970s, with the growth of suburbanisation and the de-industrialisation and consequent dereliction of the ‘inner city’, these ethnic communities were one of the few signs of life in the city. Apart from the daily commute into the CBD, business with the city council or a trip to the big shopping streets these areas were one of the few reasons for visiting city centres stigmatised by urban decay and petty crime.

Performance evaluation of eight contemporary passive solar homes in subtropical Australia

Subtropical south-east Queensland’s expanding population is expected to lead to a demand for an additional 754,000 dwellings by 2031. A legacy of poor housing design, minimal building regulations, an absence of building performance evaluation and various social and market factors has lead to a high and growing penetration of, and reliance on, air conditioners to provide comfort in this relatively benign climate. This reliance impacts on policy goals to adapt to and mitigate against global warming, electricity infrastructure investment and household resilience. Based on the concept of bioclimatic design, this field study scrutinizes eight non-air conditioned homes to develop a deeper understanding of the role of contemporary passive solar architecture in the delivery of thermally comfortable and resilient homes in the subtropics. These homes were found to provide inhabitants with an acceptable level of thermal comfort (18-28oC) for 77 – 97% of the year. Family expectations and experiences of comfort, and the various design strategies utilized were compared against the measured performance outcomes. This comparison revealed issues that limited quantification and implementation of design intent and highlighted factors that constrained system optimisation.

Diffusion determinants for passive building technologies

This summary is based on an international review of leading peer reviewed journals, in both technical and management fields. It draws on highly cited articles published between 2000 and 2009 to investigate the research question, "What are the diffusion determinants for passive building technologies in Australia?". Using a conceptual framework drawn from the innovation systems literature, this paper synthesises and interprets the literature to map the current state of passive building technologies in Australia and to analyse the drivers for, and obstacles to, their optimal diffusion. The paper concludes that the government has a key role to play through its influence over the specification of building codes.

Evaluation of an inactivated Ross River virus vaccine in active and passive mouse immunization models and establishment of a correlate of protection

Ross River Virus has caused reported outbreaks of epidemic polyarthritis, a chronic debilitating disease associated with significant long-term morbidity in Australia and the Pacific region since the 1920s. To address this public health concern, a formalin- and UV-inactivated whole virus vaccine grown in animal protein-free cell culture was developed and tested in preclinical studies to evaluate immunogenicity and efficacy in animal models. After active immunizations, the vaccine dose-dependently induced antibodies and protected adult mice from viremia and interferon α/β receptor knock-out (IFN-α/βR(-/-)) mice from death and disease. In passive transfer studies, administration of human vaccinee sera followed by RRV challenge protected adult mice from viremia and young mice from development of arthritic signs similar to human RRV-induced disease. Based on the good correlation between antibody titers in human sera and protection of animals, a correlate of protection was defined. This is of particular importance for the evaluation of the vaccine because of the comparatively low annual incidence of RRV disease, which renders a classical efficacy trial impractical. Antibody-dependent enhancement of infection, did not occur in mice even at low to undetectable concentrations of vaccine-induced antibodies. Also, RRV vaccine-induced antibodies were partially cross-protective against infection with a related alphavirus, Chikungunya virus, and did not enhance infection. Based on these findings, the inactivated RRV vaccine is expected to be efficacious and protect humans from RRV disease

Active versus passive screen time for young children

In this paper, we report some initial findings from our investigations into the Australian Government’s Longitudinal Study of Australian Children dataset. It is revealed that the majority of Australian children are exceeding the government’s Screen Time recommendations and that most of their screen time is spent as TV viewing, as opposed to video game play or computer use. In light of this finding,we review the body of research surrounding children’s engagement in Screen Time activities and the associated positive and negative effects. Based on existing evidence,we define two categories of Screen Time—Active Screen Time and Passive Screen Time. It is proposed that this distinction provides a more accurate classification of Screen Time and a more informative lens through which to consider the associated benefits and detrimental effects for young children.

V1-inspired features induce a weighted margin in SVMs

Image representations derived from simplified models of the primary visual cortex (V1), such as HOG and SIFT, elicit good performance in a myriad of visual classification tasks including object recognition/detection, pedestrian detection and facial expression classification. A central question in the vision, learning and neuroscience communities regards why these architectures perform so well. In this paper, we offer a unique perspective to this question by subsuming the role of V1-inspired features directly within a linear support vector machine (SVM). We demonstrate that a specific class of such features in conjunction with a linear SVM can be reinterpreted as inducing a weighted margin on the Kronecker basis expansion of an image. This new viewpoint on the role of V1-inspired features allows us to answer fundamental questions on the uniqueness and redundancies of these features, and offer substantial improvements in terms of computational and storage efficiency.

Post-glacial sea-level changes around the Australian margin : a review

It has been known since Rhodes Fairbridge’s first attempt to establish a global pattern of Holocene sea-level change by combining evidence from Western Australia and from sites in the northern hemisphere that the details of sea-level history since the Last Glacial Maximum vary considerably across the globe. The Australian region is relatively stable tectonically and is situated in the ‘far-field’ of former ice sheets. It therefore preserves important records of post-glacial sea levels that are less complicated by neotectonics or glacio-isostatic adjustments. Accordingly, the relative sea-level record of this region is dominantly one of glacio-eustatic (ice equivalent) sea-level changes. The broader Australasian region has provided critical information on the nature of post-glacial sea level, including the termination of the Last Glacial Maximum when sea level was approximately 125 m lower than present around 21,000–19,000 years BP, and insights into meltwater pulse 1A between 14,600 and 14,300 cal. yr BP. Although most parts of the Australian continent reveals a high degree of tectonic stability, research conducted since the 1970s has shown that the timing and elevation of a Holocene highstand varies systematically around its margin. This is attributed primarily to variations in the timing of the response of the ocean basins and shallow continental shelves to the increased ocean volumes following ice-melt, including a process known as ocean siphoning (i.e. glacio-hydro-isostatic adjustment processes). Several seminal studies in the early 1980s produced important data sets from the Australasian region that have provided a solid foundation for more recent palaeo-sea-level research. This review revisits these key studies emphasising their continuing influence on Quaternary research and incorporates relatively recent investigations to interpret the nature of post-glacial sea-level change around Australia. These include a synthesis of research from the Northern Territory, Queensland, New South Wales, South Australia and Western Australia. A focus of these more recent studies has been the re-examination of: (1) the accuracy and reliability of different proxy sea-level indicators; (2) the rate and nature of post-glacial sea-level rise; (3) the evidence for timing, elevation, and duration of mid-Holocene highstands; and, (4) the notion of mid- to late Holocene sea-level oscillations, and their basis. Based on this synthesis of previous research, it is clear that estimates of past sea-surface elevation are a function of eustatic factors as well as morphodynamics of individual sites, the wide variety of proxy sea-level indicators used, their wide geographical range, and their indicative meaning. Some progress has been made in understanding the variability of the accuracy of proxy indicators in relation to their contemporary sea level, the inter-comparison of the variety of dating techniques used and the nuances of calibration of radiocarbon ages to sidereal years. These issues need to be thoroughly understood before proxy sea-level indicators can be incorporated into credible reconstructions of relative sea-level change at individual locations. Many of the issues, which challenged sea-level researchers in the latter part of the twentieth century, remain contentious today. Divergent opinions remain about: (1) exactly when sea level attained present levels following the most recent post-glacial marine transgression (PMT); (2) the elevation that sea-level reached during the Holocene sea-level highstand; (3) whether sea-level fell smoothly from a metre or more above its present level following the PMT; (4) whether sea level remained at these highstand levels for a considerable period before falling to its present position; or (5) whether it underwent a series of moderate oscillations during the Holocene highstand.

Passive control of a biventricular assist device with compliant inflow cannulae

Rotary ventricular assist device (VAD) support of the cardiovascular system is susceptible to suction events due to the limited preload sensitivity of these devices. This may be of particular concern with rotary biventricular support (BiVAD) where the native, flow-balancing Starling response is diminished in both ventricles. The reliability of sensor and sensor-less based control systems which aim to control VAD flow based on preload have limitations and thus an alternative solution is desired. This study introduces a compliant inflow cannula (CIC) which could improve the preload sensitivity of a rotary VAD by passively altering VAD flow depending on preload. To evaluate the design, both the CIC and a standard rigid inflow cannula were inserted into a mock circulation loop to enable biventricular heart failure support using configurations of atrial and ventricular inflow, and arterial outflow cannulation. A range of left (LVAD) and right VAD (RVAD) rotational speeds were tested as well as step changes in systemic/pulmonary vascular resistance to alter relative preloads, with resulting flow rates recorded. Simulated suction events were observed, particularly at higher VAD speeds, during support with the rigid inflow cannula, while the CIC prevented suction events under all circumstances. The compliant section passively restricted its internal diameter as preload was reduced, which increased the VAD circuit resistance and thus reduced VAD flow. Therefore, a compliant inflow cannula could potentially be used as a passive control system to prevent suction events in rotary left, right and biventricular support.