V1-inspired features induce a weighted margin in SVMs

Image representations derived from simplified models of the primary visual cortex (V1), such as HOG and SIFT, elicit good performance in a myriad of visual classification tasks including object recognition/detection, pedestrian detection and facial expression classification. A central question in the vision, learning and neuroscience communities regards why these architectures perform so well. In this paper, we offer a unique perspective to this question by subsuming the role of V1-inspired features directly within a linear support vector machine (SVM). We demonstrate that a specific class of such features in conjunction with a linear SVM can be reinterpreted as inducing a weighted margin on the Kronecker basis expansion of an image. This new viewpoint on the role of V1-inspired features allows us to answer fundamental questions on the uniqueness and redundancies of these features, and offer substantial improvements in terms of computational and storage efficiency.

‘Small Pricks’ at Lunchtime : Some notes on Botox

Cosmetic enhancement technologies have been subject to extended discussion in sociological literature. Botox, however, seems to have been mostly sidelined in this discussion in favour of more ‘extreme’ forms of cosmetic enhancement, such as those performed under general anaesthetic. In this paper, we suggest the need to further examine Botox as a sociological issue. We do this by highlighting some of the disparities and parallels that Botox shares with the existing literature on cosmetic enhancement technologies.

Large margin vector quantization

In this paper we describe the Large Margin Vector Quantization algorithm (LMVQ), which uses gradient ascent to maximise the margin of a radial basis function classifier. We present a derivation of the algorithm, which proceeds from an estimate of the class-conditional probability densities. We show that the key behaviour of Kohonen's well-known LVQ2 and LVQ3 algorithms emerge as natural consequences of our formulation. We compare the performance of LMVQ with that of Kohonen's LVQ algorithms on an artificial classification problem and several well known benchmark classification tasks. We find that the classifiers produced by LMVQ attain a level of accuracy that compares well with those obtained via LVQ1, LVQ2 and LVQ3, with reduced storage complexity. We indicate future directions of enquiry based on the large margin approach to Learning Vector Quantization.

Further notes on the application of zero-inflated models in highway safety

The intent of this note is to succinctly articulate additional points that were not provided in the original paper (Lord et al., 2005) and to help clarify a collective reluctance to adopt zero-inflated (ZI) models for modeling highway safety data. A dialogue on this important issue, just one of many important safety modeling issues, is healthy discourse on the path towards improved safety modeling. This note first provides a summary of prior findings and conclusions of the original paper. It then presents two critical and relevant issues: the maximizing statistical fit fallacy and logic problems with the ZI model in highway safety modeling. Finally, we provide brief conclusions.

The blogification of Australian journalism? Notes from the election

My overall contention is that from Mark Latham to Grogsgate, from Tony’s speedos to Julia’s treasonous lack of handbags, Australian political journalism hasn’t exactly wowed us with the quality of its coverage these past months – with ample help, it should be noted, from the two sides of politics and the respective small target strategies themselves. Tim Dunlop has gone as far as to suggest that during the election we’ve seen politics and the media locked in a death spiral (http://www.abc.net.au/unleashed/35594.html) – an observation we might want to take up in the panel discussion – but even without the dramatic language the overall tendency has been that of a race to the bottom in the quality of political discourse in this country, with very few exceptions. And as a result, trust in journalism – the professional esteem in which journalists are held by their audiences – has been steadily declining for some time. Australian journalists are hardly alone in this, of course: this decline is a dynamic which has been observed in many other nations, too.

Critical literacy : foundational notes

This article traces the lineage of critical literacy from Freire through critical pedagogies and discourse analysis. It discusses the need for a contingent definition of critical literacy, given the increasingly sophisticated nature of texts and discourses.

Boosting the margin: A new explanation for the effectiveness of voting methods

One of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated classifier usually does not increase as its size becomes very large, and often is observed to decrease even after the training error reaches zero. In this paper, we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule, where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label. We show that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. We also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples. Finally, we compare our explanation to those based on the bias-variance decomposition.

Multiview point cloud kernels for semisupervised learning [Lecture Notes]

In semisupervised learning (SSL), a predictive model is learn from a collection of labeled data and a typically much larger collection of unlabeled data. These paper presented a framework called multi-view point cloud regularization (MVPCR), which unifies and generalizes several semisupervised kernel methods that are based on data-dependent regularization in reproducing kernel Hilbert spaces (RKHSs). Special cases of MVPCR include coregularized least squares (CoRLS), manifold regularization (MR), and graph-based SSL. An accompanying theorem shows how to reduce any MVPCR problem to standard supervised learning with a new multi-view kernel.

Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Exponentiated gradient algorithms for large-margin structured classification

We consider the problem of structured classification, where the task is to predict a label y from an input x, and y has meaningful internal structure. Our framework includes supervised training of Markov random fields and weighted context-free grammars as special cases. We describe an algorithm that solves the large-margin optimization problem defined in [12], using an exponential-family (Gibbs distribution) representation of structured objects. The algorithm is efficient—even in cases where the number of labels y is exponential in size—provided that certain expectations under Gibbs distributions can be calculated efficiently. The method for structured labels relies on a more general result, specifically the application of exponentiated gradient updates [7, 8] to quadratic programs.

Margin-adaptive model selection in statistical learning

A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one. Requiring this “strong margin adaptivity” makes the model selection problem more challenging. We first prove, in a general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.

From the margin to the centre : towards an art strategy for the ethnic enclaves in the city

Sunnybank represents a distinctly Australian take on the classic ‘Chinatown’ – or indeed other ethic community enclaves such as ‘little Italy’, ‘little Bombay’, ‘little Athens’ and so on. In the Northern Hemisphere these tended to grow up in the dense working class neighbourhoods of industrial cities, especially in port cities like Liverpool, London, New York and San Francisco. The existing Chinatowns of Sydney and Melbourne, and to some extent Brisbane’s Fortitude Valley, are of this variety. In the late 1970s, with the growth of suburbanisation and the de-industrialisation and consequent dereliction of the ‘inner city’, these ethnic communities were one of the few signs of life in the city. Apart from the daily commute into the CBD, business with the city council or a trip to the big shopping streets these areas were one of the few reasons for visiting city centres stigmatised by urban decay and petty crime.