Large margin vector quantization

In this paper we describe the Large Margin Vector Quantization algorithm (LMVQ), which uses gradient ascent to maximise the margin of a radial basis function classifier. We present a derivation of the algorithm, which proceeds from an estimate of the class-conditional probability densities. We show that the key behaviour of Kohonen's well-known LVQ2 and LVQ3 algorithms emerge as natural consequences of our formulation. We compare the performance of LMVQ with that of Kohonen's LVQ algorithms on an artificial classification problem and several well known benchmark classification tasks. We find that the classifiers produced by LMVQ attain a level of accuracy that compares well with those obtained via LVQ1, LVQ2 and LVQ3, with reduced storage complexity. We indicate future directions of enquiry based on the large margin approach to Learning Vector Quantization.

Boosting the margin: A new explanation for the effectiveness of voting methods

One of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated classifier usually does not increase as its size becomes very large, and often is observed to decrease even after the training error reaches zero. In this paper, we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule, where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label. We show that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. We also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples. Finally, we compare our explanation to those based on the bias-variance decomposition.

Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Exponentiated gradient algorithms for large-margin structured classification

We consider the problem of structured classification, where the task is to predict a label y from an input x, and y has meaningful internal structure. Our framework includes supervised training of Markov random fields and weighted context-free grammars as special cases. We describe an algorithm that solves the large-margin optimization problem defined in [12], using an exponential-family (Gibbs distribution) representation of structured objects. The algorithm is efficient—even in cases where the number of labels y is exponential in size—provided that certain expectations under Gibbs distributions can be calculated efficiently. The method for structured labels relies on a more general result, specifically the application of exponentiated gradient updates [7, 8] to quadratic programs.

Margin-adaptive model selection in statistical learning

A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one. Requiring this “strong margin adaptivity” makes the model selection problem more challenging. We first prove, in a general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.

From the margin to the centre : towards an art strategy for the ethnic enclaves in the city

Sunnybank represents a distinctly Australian take on the classic ‘Chinatown’ – or indeed other ethic community enclaves such as ‘little Italy’, ‘little Bombay’, ‘little Athens’ and so on. In the Northern Hemisphere these tended to grow up in the dense working class neighbourhoods of industrial cities, especially in port cities like Liverpool, London, New York and San Francisco. The existing Chinatowns of Sydney and Melbourne, and to some extent Brisbane’s Fortitude Valley, are of this variety. In the late 1970s, with the growth of suburbanisation and the de-industrialisation and consequent dereliction of the ‘inner city’, these ethnic communities were one of the few signs of life in the city. Apart from the daily commute into the CBD, business with the city council or a trip to the big shopping streets these areas were one of the few reasons for visiting city centres stigmatised by urban decay and petty crime.

V1-inspired features induce a weighted margin in SVMs

Image representations derived from simplified models of the primary visual cortex (V1), such as HOG and SIFT, elicit good performance in a myriad of visual classification tasks including object recognition/detection, pedestrian detection and facial expression classification. A central question in the vision, learning and neuroscience communities regards why these architectures perform so well. In this paper, we offer a unique perspective to this question by subsuming the role of V1-inspired features directly within a linear support vector machine (SVM). We demonstrate that a specific class of such features in conjunction with a linear SVM can be reinterpreted as inducing a weighted margin on the Kronecker basis expansion of an image. This new viewpoint on the role of V1-inspired features allows us to answer fundamental questions on the uniqueness and redundancies of these features, and offer substantial improvements in terms of computational and storage efficiency.

Post-glacial sea-level changes around the Australian margin : a review

It has been known since Rhodes Fairbridge’s first attempt to establish a global pattern of Holocene sea-level change by combining evidence from Western Australia and from sites in the northern hemisphere that the details of sea-level history since the Last Glacial Maximum vary considerably across the globe. The Australian region is relatively stable tectonically and is situated in the ‘far-field’ of former ice sheets. It therefore preserves important records of post-glacial sea levels that are less complicated by neotectonics or glacio-isostatic adjustments. Accordingly, the relative sea-level record of this region is dominantly one of glacio-eustatic (ice equivalent) sea-level changes. The broader Australasian region has provided critical information on the nature of post-glacial sea level, including the termination of the Last Glacial Maximum when sea level was approximately 125 m lower than present around 21,000–19,000 years BP, and insights into meltwater pulse 1A between 14,600 and 14,300 cal. yr BP. Although most parts of the Australian continent reveals a high degree of tectonic stability, research conducted since the 1970s has shown that the timing and elevation of a Holocene highstand varies systematically around its margin. This is attributed primarily to variations in the timing of the response of the ocean basins and shallow continental shelves to the increased ocean volumes following ice-melt, including a process known as ocean siphoning (i.e. glacio-hydro-isostatic adjustment processes). Several seminal studies in the early 1980s produced important data sets from the Australasian region that have provided a solid foundation for more recent palaeo-sea-level research. This review revisits these key studies emphasising their continuing influence on Quaternary research and incorporates relatively recent investigations to interpret the nature of post-glacial sea-level change around Australia. These include a synthesis of research from the Northern Territory, Queensland, New South Wales, South Australia and Western Australia. A focus of these more recent studies has been the re-examination of: (1) the accuracy and reliability of different proxy sea-level indicators; (2) the rate and nature of post-glacial sea-level rise; (3) the evidence for timing, elevation, and duration of mid-Holocene highstands; and, (4) the notion of mid- to late Holocene sea-level oscillations, and their basis. Based on this synthesis of previous research, it is clear that estimates of past sea-surface elevation are a function of eustatic factors as well as morphodynamics of individual sites, the wide variety of proxy sea-level indicators used, their wide geographical range, and their indicative meaning. Some progress has been made in understanding the variability of the accuracy of proxy indicators in relation to their contemporary sea level, the inter-comparison of the variety of dating techniques used and the nuances of calibration of radiocarbon ages to sidereal years. These issues need to be thoroughly understood before proxy sea-level indicators can be incorporated into credible reconstructions of relative sea-level change at individual locations. Many of the issues, which challenged sea-level researchers in the latter part of the twentieth century, remain contentious today. Divergent opinions remain about: (1) exactly when sea level attained present levels following the most recent post-glacial marine transgression (PMT); (2) the elevation that sea-level reached during the Holocene sea-level highstand; (3) whether sea-level fell smoothly from a metre or more above its present level following the PMT; (4) whether sea level remained at these highstand levels for a considerable period before falling to its present position; or (5) whether it underwent a series of moderate oscillations during the Holocene highstand.

The "Shackleton Site" (IODP Site U1385) on the Iberian Margin

Nick Shackleton’s research on piston cores from the Iberian margin highlighted the importance of this region for providing high-fidelity records of millennial-scale climate variability, and for correlating climate events from the marine environment to polar ice cores and European terrestrial sequences. During the Integrated Ocean Drilling Program (IODP) Expedition 339, we sought to extend the Iberian margin sediment record by drilling with the D/V JOIDES Resolution. Five holes were cored at Site U1385 using the advanced piston corer (APC) system to a maximum depth of ∼ 155.9 m below sea floor (m b.s.f.). Immediately after the expedition, cores from all holes were analyzed by core scanning X-ray fluorescence (XRF) at 1 cm spatial resolution. Ca/Ti data were used to accurately correlate from hole-to-hole and construct a composite spliced section, containing no gaps or disturbed intervals to 166.5 m composite depth (mcd). A low-resolution (20 cm sample spacing) oxygen isotope record confirms that Site U1385 contains a continuous record of hemipelagic sedimentation from the Holocene to 1.43 Ma (Marine Isotope Stage 46). The sediment profile at Site U1385 extends across the middle Pleistocene transition (MPT) with sedimentation rates averaging ∼ 10 cm kyr−1. Strongprecession cycles in colour and elemental XRF signals provide a powerful tool for developing an orbitally tuned reference timescale. Site U1385 is likely to become an important type section for marine–ice–terrestrial core correlations and the study of orbital- and millennial-scale climate variability.

Fast unscented transformation-based transient stability margin estimation incorporating uncertainty of wind generation

Intermittent generation from wind farms leads to fluctuating power system operating conditions pushing the stability margin to its limits. The traditional way of determining the worst case generation dispatch for a system with several semi-scheduled wind generators yields a conservative solution. This paper proposes a fast estimation of the transient stability margin (TSM) incorporating the uncertainty of wind generation. First, the Kalman filter (KF) is used to provide linear estimation of system angle and then unscented transformation (UT) is used to estimate the distribution of the TSM. The proposed method is compared with the traditional Monte Carlo (MC) method and the effectiveness of the proposed approach is verified using Single Machine Infinite Bus (SMIB) and IEEE 14 generator Australian dynamic system. This method will aid grid operators to perform fast online calculations to estimate TSM distribution of a power system with high levels of intermittent wind generation.

Analysis Of The Energy Quantization Effects On Single Electron Inverter Performance Through Noise Margin Modeling

Possible integration of Single Electron Transistor (SET) with CMOS technology is making the study of semiconductor SET more important than the metallic SET and consequently, the study of energy quantization effects on semiconductor SET devices and circuits is gaining significance. In this paper, for the first time, the effects of energy quantization on SET inverter performance are examined through analytical modeling and Monte Carlo simulations. It is observed that the primary effect of energy quantization is to change the Coulomb Blockade region and drain current of SET devices and as a result affects the noise margin, power dissipation, and the propagation delay of SET inverter. A new model for the noise margin of SET inverter is proposed which includes the energy quantization effects. Using the noise margin as a metric, the robustness of SET inverter is studied against the effects of energy quantization. It is shown that SET inverter designed with CT : CG = 1/3 (where CT and CG are tunnel junction and gate capacitances respectively) offers maximum robustness against energy quantization.

Impact of subsoil constraints on wheat yield and gross margin on fine-textured soils of the southern Victorian Mallee

The APSIM-Wheat module was used to investigate our present capacity to simulate wheat yields in a semi-arid region of eastern Australia (the Victorian Mallee), where hostile subsoils associated with salinity, sodicity, and boron toxicity are known to limit grain yield. In this study we tested whether the effects of subsoil constraints on wheat growth and production could be modelled with APSIM-Wheat by assuming that either: (a) root exploration within a particular soil layer was reduced by the presence of toxic concentrations of salts, or (b) soil water uptake from a particular soil layer was reduced by high concentration of salts through osmotic effects. After evaluating the improved predictive capacity of the model we applied it to study the interactions between subsoil constraints and seasonal conditions, and to estimate the economic effect that subsoil constraints have on wheat farming in the Victorian Mallee under different climatic scenarios. Although the soils had high levels of salinity, sodicity, and boron, the observed variability in root abundance at different soil layers was mainly related to soil salinity. We concluded that: (i) whether the effect of subsoil limitations on growth and yield of wheat in the Victorian Mallee is driven by toxic, osmotic, or both effects acting simultaneously still requires further research, (ii) at present, the performance of APSIM-Wheat in the region can be improved either by assuming increased values of lower limit for soil water extraction, or by modifying the pattern of root exploration in the soil pro. le, both as a function of soil salinity. The effect of subsoil constraints on wheat yield and gross margin can be expected to be higher during drier than wetter seasons. In this region the interaction between climate and soil properties makes rainfall information alone, of little use for risk management and farm planning when not integrated with cropping systems models.

Real-time wide-area loading margin sensitivity (WALMS) in power systems

Loading margin sensitivity (LMS) has been widely used in applications in the realm of voltage stability assessment and control. Typically, LMS is derived based on system equilibrium equations near bifurcation and therefore requires full detailed system model and significant computation effort. Availability of phasor measurement units (PMUs) due to the recent development of wide-area monitoring system (WAMS) provides an alternative computation-friendly approach for calculating LMS. With such motivation, this work proposes measurement-based wide-area loading margin sensitivity (WALMS) in bulk power systems. The proposed sensitivity, with its simplicity, has great potential to be embedded in real-time applications. Moreover, the calculation of the WALMS is not limited to low voltage near bifurcation point. A case study on IEEE 39-bus system verifies the proposed sensitivity. Finally, a voltage control scenario demonstrates the potential application of the WALMS.