Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Exponentiated gradient algorithms for large-margin structured classification

We consider the problem of structured classification, where the task is to predict a label y from an input x, and y has meaningful internal structure. Our framework includes supervised training of Markov random fields and weighted context-free grammars as special cases. We describe an algorithm that solves the large-margin optimization problem defined in [12], using an exponential-family (Gibbs distribution) representation of structured objects. The algorithm is efficient—even in cases where the number of labels y is exponential in size—provided that certain expectations under Gibbs distributions can be calculated efficiently. The method for structured labels relies on a more general result, specifically the application of exponentiated gradient updates [7, 8] to quadratic programs.

Health professionals' recognition of co-occurring alcohol and depressive disorders in youth: a survey of Australian general practitioners, psychiatrists, psychologists and mental health nurses using case vignettes

Objective: To examine whether health professionals who commonly deal with mental disorder are able to identify co occurring alcohol misuse in young people presenting with depression. Method: Between September 2006 and January 2007, a survey examining beliefs regarding appropriate interventions for mental disorder in youth was sent to 1710 psychiatrists, 2000 general practitioners (GPs), 1628 mental health nurses, and 2000 psychologists in Australia. Participants within each professional group were randomly given one of four vignettes describing a young person with a DSM-IV mental disorder. Herein is reported data from the depression and depression with alcohol misuse vignettes. Results: A total of 305 psychiatrists, 258 GPs, 292 mental health nurses and 375 psychologists completed one of the depression vignettes. A diagnosis of mood disorder was identified by at least 83.8% of professionals, with no significant differences noted between professional groups. Rates of reported co-occurring substance use disorders were substantially lower, particularly among older professionals and psychologists. Conclusions: GPs, psychologists and mental health professionals do not readily identify co-occurring alcohol misuse in young people with depression. Given the substantially negative impact of co-occurring disorders, it is imperative that health-care professionals are appropriately trained to detect such disorders promptly, to ensure young people have access to effective, early intervention.

Influence of preexercise muscle glycogen content on transcriptional activity of metabolic and myogenic genes in well-trained humans

To determine whether pre-exercise muscle glycogen content influences the transcription of several early-response genes involved in the regulation of muscle growth, seven male strength-trained subjects performed one-legged cycling exercise to exhaustion to lower muscle glycogen levels (Low) in one leg compared with the leg with normal muscle glycogen (Norm) and then the following day completed a unilateral bout of resistance training (RT). Muscle biopsies from both legs were taken at rest, immediately after RT, and after 3 h of recovery. Resting glycogen content was higher in the control leg (Norm leg) than in the Low leg (435 ± 87 vs. 193 ± 29 mmol/kg dry wt; P < 0.01). RT decreased glycogen content in both legs (P < 0.05), but postexercise values remained significantly higher in the Norm than the Low leg (312 ± 129 vs. 102 ± 34 mmol/kg dry wt; P < 0.01). GLUT4 (3-fold; P < 0.01) and glycogenin mRNA abundance (2.5-fold; not significant) were elevated at rest in the Norm leg, but such differences were abolished after exercise. Preexercise mRNA abundance of atrogenes was also higher in the Norm compared with the Low leg [atrogin: ?14-fold, P < 0.01; RING (really interesting novel gene) finger: ?3-fold, P < 0.05] but decreased for atrogin in Norm following RT (P < 0.05). There were no differences in the mRNA abundance of myogenic regulatory factors and IGF-I in the Norm compared with the Low leg. Our results demonstrate that 1) low muscle glycogen content has variable effects on the basal transcription of select metabolic and myogenic genes at rest, and 2) any differences in basal transcription are completely abolished after a single bout of heavy resistance training. We conclude that commencing resistance exercise with low muscle glycogen does not enhance the activity of genes implicated in promoting hypertrophy.