Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Toward a fuzzy domain ontology extraction method for adaptive e-learning

With the widespread applications of electronic learning (e-Learning) technologies to education at all levels, increasing number of online educational resources and messages are generated from the corresponding e-Learning environments. Nevertheless, it is quite difficult, if not totally impossible, for instructors to read through and analyze the online messages to predict the progress of their students on the fly. The main contribution of this paper is the illustration of a novel concept map generation mechanism which is underpinned by a fuzzy domain ontology extraction algorithm. The proposed mechanism can automatically construct concept maps based on the messages posted to online discussion forums. By browsing the concept maps, instructors can quickly identify the progress of their students and adjust the pedagogical sequence on the fly. Our initial experimental results reveal that the accuracy and the quality of the automatically generated concept maps are promising. Our research work opens the door to the development and application of intelligent software tools to enhance e-Learning.

Using concept mapping to scaffold learning for students who experience learning difficulties in science classes

In order to develop scientific literacy students need the cognitive tools that enable them to read and evaluate science texts. One cognitive tool that has been widely used in science education to aid the development of conceptual understanding is concept mapping. However, it has been found some students experience difficulty with concept map construction. This study reports on the development and evaluation of an instructional sequence that was used to scaffold the concept-mapping process when middle school students who were experiencing difficulty with science learning used concept mapping to summarise a chapter of a science text. In this study individual differences in working memory functioning are suggested as one reason that students experience difficulty with concept map construction. The study was conducted using a design-based research methodology in the school’s learning support centre. The analysis of student work samples collected during the two-year study identified some of the difficulties and benefits associated with the use of scaffolded concept mapping with these students. The observations made during this study highlight the difficulty that some students experience with the use of concept mapping as a means of developing an understanding of science concepts and the amount of instructional support that is required for such understanding to develop. Specifically, the findings of the study support the use of multi-component, multi-modal instructional techniques to facilitate the development of conceptual understanding with students who experience difficulty with science learning. In addition, the important roles of interactive dialogue and metacognition in the development of conceptual understanding are identified.

Matrix regularization techniques for online multitask learning

In this paper we examine the problem of prediction with expert advice in a setup where the learner is presented with a sequence of examples coming from different tasks. In order for the learner to be able to benefit from performing multiple tasks simultaneously, we make assumptions of task relatedness by constraining the comparator to use a lesser number of best experts than the number of tasks. We show how this corresponds naturally to learning under spectral or structural matrix constraints, and propose regularization techniques to enforce the constraints. The regularization techniques proposed here are interesting in their own right and multitask learning is just one application for the ideas. A theoretical analysis of one such regularizer is performed, and a regret bound that shows benefits of this setup is reported.

L1 Learning Strategy Instruction does make a difference in EFL listening: An empirical study.

In language learning, listening is the basic skill which learners should begin to develop other skills, namely speaking, reading and writing. This sequence of language learning in most English as Foreign Language (EFL) settings goes against the stream, learning first reading and writing and later listening and speaking. This study investigates the effects of cognitive, process-based approach to instructing EFL listening strategies over 11 weeks during a semester in Persian (L1). Lower intermediate female participants (N = 50) came from a couple of EFL classrooms in an English Language Institute in Iran. The experimental group (n = 25) listened to their classroom activities using a methodology that led learners through four cognitive processes (guessing, making inference, identifying topics and repetition) in Persian was basically successful in EFL listening. The same teacher taught the control group (n = 25), which listened to the same classroom listening activities without any guided attention to the learning strategy process in Persian. A pre and post listening test made by a group of experts in the language institute tracked any development in light of cognitive learning strategy instruction in EFL listening through L1. The hypothesis was that the experimental group received the guided attention in L1 during the classroom listening activities made greater gains and was verified despite the partial improvement of the control group.

The use of virtual prototyping to rehearse the sequence of construction work involving mobile cranes

Purpose – Rehearsing practical site operations is without doubt one of the most effective methods for minimising planning mistakes, because of the learning that takes place during the rehearsal activity. However, real rehearsal is not a practical solution for on-site construction activities, as it not only involves a considerable amount of cost but can also have adverse environmental implications. One approach to overcoming this is by the use of virtual rehearsals. The purpose of this paper is to investigate an approach to simulation of the motion of cranes in order to test the feasibility of associated construction sequencing and generate construction schedules for review and visualisation. Design/methodology/approach – The paper describes a system involving two technologies, virtual prototyping (VP) and four-dimensional (4D) simulation, to assist construction planners in testing the sequence of construction activities when mobile cranes are involved. The system consists of five modules, comprising input, database, equipment, process and output, and is capable of detecting potential collisions. A real-world trial is described in which the system was tested and validated. Findings – Feedback from the planners involved in the trial indicated that they found the system to be useful in its present form and that they would welcome its further development into a fully automated platform for validating construction sequencing decisions. Research limitations/implications – The tool has the potential to provide a cost-effective means of improving construction planning. However, it is limited at present to the specific case of crane movement under special consideration. Originality/value – This paper presents a large-scale, real life case of applying VP technology in planning construction processes and activities.

Constructing Technology Projects with Young Children : Four Detailed Case Studies of Mounting Constructivist Learning Projects Using Dimple Technologies for Young Children

The work by graduate teachers in this volume represent intentional design of learning experiences using technology for Early Childhood settings. They were given a two-part design task: a sequence of lessons organised around a themed project; and the collection of resources to support such activities. The project had to be constructive in nature where the children built objects and representations that were meaningful to them. The excellent works presented here offer a range of approaches that would be suitable in a variety of contexts. Because they are reasoned, these projects offer flexibility in implementation along with confidence that they would be effective.

Does teaching sequence matter when teaching high school chemistry with scientific visualisations?

Five Canadian high school Chemistry classes in one school, taught by three different teachers, studied the concepts of dynamic chemical equilibria and Le Chatelier’s Principle. Some students received traditional teacher-led explanations of the concept first and used an interactive scientific visualisation second, while others worked with the visualisation first and received the teacher-led explanation second. Students completed a test of their conceptual understanding of the relevant concepts prior to instruction, after the first instructional session and at the end of instruction. Data on students’ academic achievement (highest, middle or lowest third of the class on the mid-term exam) and gender were also collected to explore the relationship between these factors, conceptual development and instructional sequencing. Results show, within this context at least, that teaching sequence is not important in terms of students’ conceptual learning gains.

Playing with Grammar in the Early Years : Learning about Language in the Australian Curriculum : English

As teacher/researchers interested in the pursuit of socially-just outcomes in early childhood education, the form and function of language occupies a special position in our work. We believe that mastering a range of literacy competences includes not only the technical skills for learning, but also the resources for viewing and constructing the world (Freire and Macdeo, 1987). Rather than seeing knowledge about language as the accumulation of technical skills alone, the viewpoint to which we subscribe treats knowledge about language as a dialectic that evolves from, is situated in, and contributes to a social arena (Halliday, 1978). We do not shy away from this position just because children are in the early years of schooling. In ‘Playing with Grammar’, we focus on the Foundation to Year 2 grouping, in line with the Australian Curriculum, Assessment and Reporting Authority’s (hereafter ACARA) advice on the ‘nature of learners’ (ACARA, 2013). With our focus on the early years of schooling comes our acknowledgement of the importance and complexity of play. At a time where accountability in education has moved many teachers to a sense of urgency to prove language and literacy achievement (Genishi and Dyson, 2009), we encourage space to revisit what we know about literature choices and learning experiences and bring these together to facilitate language learning. We can neither ignore, nor overemphasise, the importance of play for the development of language through: the opportunities presented for creative use and practice; social interactions for real purposes; and, identifying and solving problems in the lives of young children (Marsh and Hallet, 2008). We argue that by engaging young children in opportunities to play with language we are ultimately empowering them to be active in their language learning and in the process fostering a love of language and the intricacies it holds. Our goal in this publication is to provide a range of highly practical strategies for scaffolding young children through some of the Content Descriptions from the Australian Curriculum English Version 5.0, hereafter AC:E V5.0 (ACARA, 2013). This recently released curriculum offers a new theoretical approach to building children’s knowledge about language. The AC:E V5.0 uses selected traditional terms through an approach developed in systemic functional linguistics (see Halliday and Matthiessen, 2004) to highlight the dynamic forms and functions of multimodal language in texts. For example, the following statement, taken from the ‘Language: Knowing about the English language’ strand states: English uses standard grammatical terminology within a contextual framework, in which language choices are seen to vary according to the topics at hand, the nature and proximity of the relationships between the language users, and the modalities or channels of communication available (ACARA, 2013). Put simply, traditional grammar terms are used within a functional framework made up of field, tenor, and mode. An understanding of genre is noted with the reference to a ‘contextual framework’. The ‘topics at hand’ concern the field or subject matter of the text. The ‘relationships between the language users’ is a description of tenor. There is reference to ‘modalities’, such as spoken, written or visual text. We posit that this innovative approach is necessary for working with contemporary multimodal and cross-cultural texts (see Exley and Mills, 2012). We believe there is enormous power in using literature to expose children to the richness of language and in turn develop language and literacy skills. Taking time to look at language patterns within actual literature is a pathway to ‘…capture interest, stir the imagination and absorb the [child]’ into the world of language and literacy (Saxby, 1993, p. 55). In the following three sections, we have tried to remain faithful to our interpretation of the AC:E V5.0 Content Descriptions without giving an exhaustive explanation of the grammatical terms. Other excellent tomes, such as Derewianka (2011), Humphrey, Droga and Feez (2012), and Rossbridge and Rushton (2011) provide these more comprehensive explanations as does the AC:E V5.0 Glossary. We’ve reproduced some of the AC:E V5.0 glossary at the end of this publication. Our focus is on the structure and unfolding of the learning experiences. We outline strategies for working with children in Foundation, Year 1 and Year 2 by providing some demonstration learning experiences based on texts we’ve selected, but maintain that the affordances of these strategies will only be realised when teaching and learning is purposively tied to authentic projects in local contexts. We strongly encourage you not to use only the resource texts we’ve selected, but to capitalise upon your skill for identifying the language features in the texts you and the children are studying and adapt some of the strategies we have outlined. Each learning experience is connected to one of the Content Descriptions from the AC:E V5.0 and contains an experience specific purpose, a suggested resource text and a sequence for the experience that always commences with an orientation to text followed by an examination of a particular grammatical resource. We expect that each of these learning experiences will take a couple if not a few teaching episodes to work through, especially if children are meeting a concept for the first time. We hope you use as much, or as little, of each experience as is needed. Our plans allow for focused discussion, shared exploration and opportunities to revisit the same text for the purpose of enhancing meaning making. We do not want the teaching of grammar to slip into a crisis of irrelevance or to be seen as a series of worksheet drills with finite answers. Strategies for effective practice, however, have much portability. We are both very keen to hear from teachers who are adopting and adapting these learning experiences in their classrooms. Please email us on b.exley@qut.edu.au or lkervin@uow.edu.au. We’d love to continue the conversation with you over time.

Supplementary information : weighted tree kernels for sequence analysis

Genomic sequences are fundamentally text documents, admitting various representations according to need and tokenization. Gene expression depends crucially on binding of enzymes to the DNA sequence at small, poorly conserved binding sites, limiting the utility of standard pattern search. However, one may exploit the regular syntactic structure of the enzyme's component proteins and the corresponding binding sites, framing the problem as one of detecting grammatically correct genomic phrases. In this paper we propose new kernels based on weighted tree structures, traversing the paths within them to capture the features which underpin the task. Experimentally, we and that these kernels provide performance comparable with state of the art approaches for this problem, while offering significant computational advantages over earlier methods. The methods proposed may be applied to a broad range of sequence or tree-structured data in molecular biology and other domains.

Pasture Degradation and Recovery in Australia's Rangelands : Learning from History

"The extended drought periods in each degradation episode have provided a test of the capacity of grazing systems (i.e. land, plants, animals, humans and social structure) to handle stress. Evidence that degradation was already occurring was identified prior to the extended drought sequences. The sequence of dry years, ranging from two to eight years, exposed and/or amplified the degradation processes. The unequivocal evidence was provided by: (a) the physical 'horror' of bare landscapes, erosion scalds and gullies and dust storms; (b) the biological devastation of woody weeds and animal suffering/deaths or forced sales, and; (c) the financial and emotional plight of graziers and their families due to reduced production in some cases leading to abandonment of properties or, sadly, deaths (e.g. McDonald 1991, Ker Conway 1989)."--Publisher website

STAR : Predicting recombination sites from amino acid sequence

Background Designing novel proteins with site-directed recombination has enormous prospects. By locating effective recombination sites for swapping sequence parts, the probability that hybrid sequences have the desired properties is increased dramatically. The prohibitive requirements for applying current tools led us to investigate machine learning to assist in finding useful recombination sites from amino acid sequence alone. Results We present STAR, Site Targeted Amino acid Recombination predictor, which produces a score indicating the structural disruption caused by recombination, for each position in an amino acid sequence. Example predictions contrasted with those of alternative tools, illustrate STAR'S utility to assist in determining useful recombination sites. Overall, the correlation coefficient between the output of the experimentally validated protein design algorithm SCHEMA and the prediction of STAR is very high (0.89). Conclusion STAR allows the user to explore useful recombination sites in amino acid sequences with unknown structure and unknown evolutionary origin. The predictor service is available from http://pprowler.itee.uq.edu.au/star.

Learning the learning rate for prediction with expert advice

Most standard algorithms for prediction with expert advice depend on a parameter called the learning rate. This learning rate needs to be large enough to fit the data well, but small enough to prevent overfitting. For the exponential weights algorithm, a sequence of prior work has established theoretical guarantees for higher and higher data-dependent tunings of the learning rate, which allow for increasingly aggressive learning. But in practice such theoretical tunings often still perform worse (as measured by their regret) than ad hoc tuning with an even higher learning rate. To close the gap between theory and practice we introduce an approach to learn the learning rate. Up to a factor that is at most (poly)logarithmic in the number of experts and the inverse of the learning rate, our method performs as well as if we would know the empirically best learning rate from a large range that includes both conservative small values and values that are much higher than those for which formal guarantees were previously available. Our method employs a grid of learning rates, yet runs in linear time regardless of the size of the grid.

Exploring with Grammar in the Primary Years: Learning about Language in the Australian Curriculum: English

This new volume, Exploring with Grammar in the Primary Years (Exley, Kevin & Mantei, 2014), follows on from Playing with Grammar in the Early Years (Exley & Kervin, 2013). We extend our thanks to the ALEA membership for their take up of the first volume and the vibrant conversations around our first attempt at developing a pedagogy for the teaching of grammar in the early years. Your engagement at locally held ALEA events has motivated us to complete this second volume and reassert our interest in the pursuit of socially-just outcomes in the primary years. As noted in Exley and Kervin (2013), we believe that mastering a range of literacy competences includes not only the technical skills for learning, but also the resources for viewing and constructing the world (Freire and Macdeo, 1987). Rather than seeing knowledge about language as the accumulation of technical skills alone, the viewpoint to which we subscribe treats knowledge about language as a dialectic that evolves from, is situated in, and contributes to active participation within a social arena (Halliday, 1978). We acknowledge that to explore is to engage in processes of discovery as we look closely and examine the opportunities before us. As such, we draw on Janks’ (2000; 2014) critical literacy theory to underpin many of the learning experiences in this text. Janks (2000) argues that effective participation in society requires knowledge about how the power of language promotes views, beliefs and values of certain groups to the exclusion of others. Powerful language users can identify not only how readers are positioned by these views, but also the ways these views are conveyed through the design of the text, that is, the combination of vocabulary, syntax, image, movement and sound. Similarly, powerful designers of texts can make careful modal choices in written and visual design to promote certain perspectives that position readers and viewers in new ways to consider more diverse points of view. As the title of our text suggests, our activities are designed to support learners in exploring the design of texts to achieve certain purposes and to consider the potential for the sharing of their own views through text production. In Exploring with Grammar in the Primary Years, we focus on the Year 3 to Year 6 grouping in line with the Australian Curriculum, Assessment and Reporting Authority’s (hereafter ACARA) advice on the ‘nature of learners’ (ACARA, 2014). Our goal in this publication is to provide a range of highly practical strategies for scaffolding students’ learning through some of the Content Descriptions from the Australian Curriculum: English Version 7.2, hereafter AC:E (ACARA, 2014). We continue to express our belief in the power of using whole texts from a range of authentic sources including high quality children’s literature, the internet, and examples of community-based texts to expose students to the richness of language. Taking time to look at language patterns within actual texts is a pathway to ‘…capture interest, stir the imagination and absorb the [child]’ into the world of language and literacy (Saxby, 1993, p. 55). It is our intention to be more overt this time and send a stronger message that our learning experiences are simply ‘sample’ activities rather than a teachers’ workbook or a program of study to be followed. We’re hoping that teachers and students will continue to explore their bookshelves, the internet and their community for texts that provide powerful opportunities to engage with language-based learning experiences. In the following three sections, we have tried to remain faithful to our interpretation of the AC:E Content Descriptions without giving an exhaustive explanation of the grammatical terms. This recently released curriculum offers a new theoretical approach to building students’ knowledge about language. The AC:E uses selected traditional terms through an approach developed in systemic functional linguistics (see Halliday and Matthiessen, 2004) to highlight the dynamic forms and functions of multimodal language in texts. For example, the following statement, taken from the ‘Language: Knowing about the English language’ strand states: English uses standard grammatical terminology within a contextual framework, in which language choices are seen to vary according to the topics at hand, the nature and proximity of the relationships between the language users, and the modalities or channels of communication available (ACARA, 2014). Put simply, traditional grammar terms are used within a functional framework made up of field, tenor, and mode. An understanding of genre is noted with the reference to a ‘contextual framework’. The ‘topics at hand’ concern the field or subject matter of the text. The ‘relationships between the language users’ is a description of tenor. There is reference to ‘modalities’, such as spoken, written or visual text. We posit that this innovative approach is necessary for working with contemporary multimodal and cross-cultural texts (see Exley & Mills, 2012). Other excellent tomes, such as Derewianka (2011), Humphrey, Droga and Feez (2012), and Rossbridge and Rushton (2011) provide more comprehensive explanations of this unique metalanguage, as does the AC:E Glossary. We’ve reproduced some of the AC:E Glossary at the end of this publication. We’ve also kept the same layout for our learning experiences, ensuring that our teacher notes are not only succinct but also prudent in their placement. Each learning experience is connected to a Content Description from the AC:E and contains an experience with an identified purpose, suggested resource text and a possible sequence for the experience that always commences with an orientation to text followed by an examination of a particular grammatical resource. Our plans allow for focused discussion, shared exploration and opportunities to revisit the same text for the purpose of enhancing meaning making. Some learning experiences finish with deconstruction of a stimulus text while others invite students to engage in the design of new texts. We encourage you to look for opportunities in your own classrooms to move from text deconstruction to text design. In this way, students can express not only their emerging grammatical understandings, but also the ways they might position readers or viewers through the creation of their own texts. We expect that each of these learning experiences will vary in the time taken. Some may indeed take a couple if not a few teaching episodes to work through, especially if students are meeting a concept or a pedagogical strategy for the first time. We hope you use as much, or as little, of each experience as is needed for your students. We do not want the teaching of grammar to slip into a crisis of irrelevance or to be seen as a series of worksheet drills with finite answers. We firmly believe that strategies for effective deconstruction and design practice, however, have much portability. We three are very keen to hear from teachers who are adopting and adapting these learning experiences in their classrooms. Please email us on b.exley@qut.edu.au, lkervin@uow.edu.au or jessicam@ouw.edu.au. We’d love to continue the conversation with you over time. Beryl Exley, Lisa Kervin & Jessica Mantei