Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Relative entropy rate based design for linear hybrid system models

Hybrid system representations have been applied to many challenging modeling situations. In these hybrid system representations, a mixture of continuous and discrete states is used to capture the dominating behavioural features of a nonlinear, possible uncertain, model under approximation. Unfortunately, the problem of how to best design a suitable hybrid system model has not yet been fully addressed. This paper proposes a new joint state measurement relative entropy rate based approach for this design purpose. Design examples and simulation studies are presented which highlight the benefits of our proposed design approaches.

Co-constructing self-efficacy narratives : a study of four mature age university students

This thesis reports on a study in which research participants, four mature aged females starting an undergraduate degree at a regional Australian university, collaborated with the researcher in co-constructing a self-efficacy narrative. For the purpose of the study, self-efficacy was conceptualized as a means by which an individual initiates action to engage in a task or set of tasks, applies effort to perform the task or set of tasks, and persists in the face of obstacles encountered in order to achieve successful completion of the task or set of tasks. Qualitative interviews were conducted with the participants, initially investigating their respective life histories for an understanding of how they made the decision to embark on their respective academic program. Additional data were generated from a written exercise, prompting participants to furnish specific examples of self-efficacy. These data were incorporated into the individual's self-efficacy narrative, produced as the outcome of the "narrative analysis". Another aspect of the study entailed "analysis of narrative" in which analytic procedures were used to identify themes common to the self-efficacy narratives. Five main themes were identified: (a) participants' experience of schooling . for several participants their formative experience of school was not always positive, and yet their narratives demonstrated their agency in persevering and taking on university-level studies as mature aged persons; (b) recognition of family as an early influence . these influences were described as being both positive, in the sense of being supportive and encouraging, as well as posing obstacles that participants had to overcome in order to pursue their goals; (c) availability of supportive persons – the support of particular persons was acknowledged as a factor that enabled participants to persist in their respective endeavours; (d) luck or chance factors were recognised as placing participants at the right place at the right time, from which circumstances they applied considerable effort in order to convert the opportunity into a successful outcome; and (e) self-efficacy was identified as a major theme found in the narratives. The study included an evaluation of the research process by participants. A number of themes were identified in respect of the manner in which the research process was experienced as a helpful process. Participants commented that: (a) the research process was helpful in clarifying their respective career goals; (b) they appreciated opportunities provided by the research process to view their life from a different perspective and to better understand what motivated them, and what their preferred learning styles were; (c) their past successes in a range of different spheres were made more evident to them as they were guided in self-reflection, and their self-efficacious behaviour was affirmed; and (d) the opportunities provided by their participation in the research process to identify strengths of which they had not been consciously aware, to find confirmation of strengths they knew they possessed, and in some instances to rectify misconceptions they had held about aspects of their personality. The study made three important contributions to knowledge. Firstly, it provided a detailed explication of a qualitative narrative method in exploring self-efficacy, with the potential for application to other issues in educational, counselling and psychotherapy research. Secondly, it consolidated and illustrated social cognitive theory by proposing a dynamic model of self-efficacy, drawing on constructivist and interpretivist paradigms and extending extant theory and models. Finally, the study made a contribution to the debate concerning the nexus of qualitative research and counselling by providing guidelines for ethical practice in both endeavours for the practitioner-researcher.

Robust designs for Poisson regression models

We consider the problem of how to construct robust designs for Poisson regression models. An analytical expression is derived for robust designs for first-order Poisson regression models where uncertainty exists in the prior parameter estimates. Given certain constraints in the methodology, it may be necessary to extend the robust designs for implementation in practical experiments. With these extensions, our methodology constructs designs which perform similarly, in terms of estimation, to current techniques, and offers the solution in a more timely manner. We further apply this analytic result to cases where uncertainty exists in the linear predictor. The application of this methodology to practical design problems such as screening experiments is explored. Given the minimal prior knowledge that is usually available when conducting such experiments, it is recommended to derive designs robust across a variety of systems. However, incorporating such uncertainty into the design process can be a computationally intense exercise. Hence, our analytic approach is explored as an alternative.

Stochastic models for regulatory networks of the genetic toggle switch

Bistability arises within a wide range of biological systems from the λ phage switch in bacteria to cellular signal transduction pathways in mammalian cells. Changes in regulatory mechanisms may result in genetic switching in a bistable system. Recently, more and more experimental evidence in the form of bimodal population distributions indicates that noise plays a very important role in the switching of bistable systems. Although deterministic models have been used for studying the existence of bistability properties under various system conditions, these models cannot realize cell-to-cell fluctuations in genetic switching. However, there is a lag in the development of stochastic models for studying the impact of noise in bistable systems because of the lack of detailed knowledge of biochemical reactions, kinetic rates, and molecular numbers. In this work, we develop a previously undescribed general technique for developing quantitative stochastic models for large-scale genetic regulatory networks by introducing Poisson random variables into deterministic models described by ordinary differential equations. Two stochastic models have been proposed for the genetic toggle switch interfaced with either the SOS signaling pathway or a quorum-sensing signaling pathway, and we have successfully realized experimental results showing bimodal population distributions. Because the introduced stochastic models are based on widely used ordinary differential equation models, the success of this work suggests that this approach is a very promising one for studying noise in large-scale genetic regulatory networks.

Brivanib alaninate, a dual inhibitor of vascular endothelial growth factor receptor and fibroblast growth factor receptor tyrosine kinases, induces growth inhibition in mouse models of human hepatocellular carcinoma

PURPOSE: Hreceptor (VEGFR) and FGF receptor (FGFR) signaling pathways. EXPERIMENTAL DESIGN: Six different s.c. patient-derived HCC xenografts were implanted into mice. Tumor growth was evaluated in mice treated with brivanib compared with control. The effects of brivanib on apoptosis and cell proliferation were evaluated by immunohistochemistry. The SK-HEP1 and HepG2 cells were used to investigate the effects of brivanib on the VEGFR-2 and FGFR-1 signaling pathways in vitro. Western blotting was used to determine changes in proteins in these xenografts and cell lines. RESULTS: Brivanib significantly suppressed tumor growth in five of six xenograft lines. Furthermore, brivanib-induced growth inhibition was associated with a decrease in phosphorylated VEGFR-2 at Tyr(1054/1059), increased apoptosis, reduced microvessel density, inhibition of cell proliferation, and down-regulation of cell cycle regulators. The levels of FGFR-1 and FGFR-2 expression in these xenograft lines were positively correlated with its sensitivity to brivanib-induced growth inhibition. In VEGF-stimulated and basic FGF stimulated SK-HEP1 cells, brivanib significantly inhibited VEGFR-2, FGFR-1, extracellular signal-regulated kinase 1/2, and Akt phosphorylation. CONCLUSION: This study provides a strong rationale for clinical investigation of brivanib in patients with HCC.