On the consistency of multiclass classification methods

Binary classification methods can be generalized in many ways to handle multiple classes. It turns out that not all generalizations preserve the nice property of Bayes consistency. We provide a necessary and sufficient condition for consistency which applies to a large class of multiclass classification methods. The approach is illustrated by applying it to some multiclass methods proposed in the literature.

Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

On the consistency of multiclass classification methods

Binary classification is a well studied special case of the classification problem. Statistical properties of binary classifiers, such as consistency, have been investigated in a variety of settings. Binary classification methods can be generalized in many ways to handle multiple classes. It turns out that one can lose consistency in generalizing a binary classification method to deal with multiple classes. We study a rich family of multiclass methods and provide a necessary and sufficient condition for their consistency. We illustrate our approach by applying it to some multiclass methods proposed in the literature.

Numerical methods for second-order stochastic differential equations

We seek numerical methods for second‐order stochastic differential equations that reproduce the stationary density accurately for all values of damping. A complete analysis is possible for scalar linear second‐order equations (damped harmonic oscillators with additive noise), where the statistics are Gaussian and can be calculated exactly in the continuous‐time and discrete‐time cases. A matrix equation is given for the stationary variances and correlation for methods using one Gaussian random variable per timestep. The only Runge–Kutta method with a nonsingular tableau matrix that gives the exact steady state density for all values of damping is the implicit midpoint rule. Numerical experiments, comparing the implicit midpoint rule with Heun and leapfrog methods on nonlinear equations with additive or multiplicative noise, produce behavior similar to the linear case.

Lessons learned from implementation of a demonstration program to reduce the burden of anemia and hookworm in women in Yen Bai Province, Viet Nam

Background Iron deficiency, anemia and hookworm disease are important public health problems for women of reproductive age living in developing countries and affect the health of newborns and infants. Iron supplementation and deworming treatment are effective in addressing these problems in both pregnant and non-pregnant women. Daily iron supplementation and deworming after the first trimester is recommended for pregnant women although these programs usually do not operate efficiently or effectively. Weekly iron-folic acid supplementation and regular deworming for non-pregnant women may be a viable approach for improving iron status and preventing anemia during the reproductive years. Addressing these diseases at a population level before women become pregnant could significantly improve women's health before and during pregnancy, as well as their infants' growth and development. Methods and Results This paper describes the major processes undertaken in a demonstration intervention of preventive weekly iron-folic acid supplementation with regular deworming for all 52,000 women aged 15–45 years in two districts of Yen Bai province, in northern Viet Nam. The intervention strategy included extensive consultation with community leaders and village, commune, district and provincial health staff, and training for village health workers. Distribution of the drugs was integrated with the existing health service infrastructure and the village health workers were the direct point of contact with women. Iron-folic acid tablets and deworming treatment were provided free of charge from May 2006. An independent Vietnamese NGO was commissioned to evaluate compliance and identify potential problems. The program resulted in effective distribution of iron-folic acid tablets and deworming treatment to all villages in the target districts, with full or partial compliance of 85%. Conclusion Training for health staff, the strong commitment of all partners and the use of appropriate educational materials led to broad support for weekly iron-folic acid supplementation and high participation in the regular deworming days. In March 2008 the program was expanded to all districts in the province, a target population of approximately 250,000 WRA, and management was handed over to provincial authorities.