Model selection and error estimation

We study model selection strategies based on penalized empirical loss minimization. We point out a tight relationship between error estimation and data-based complexity penalization: any good error estimate may be converted into a data-based penalty function and the performance of the estimate is governed by the quality of the error estimate. We consider several penalty functions, involving error estimates on independent test data, empirical VC dimension, empirical VC entropy, and margin-based quantities. We also consider the maximal difference between the error on the first half of the training data and the second half, and the expected maximal discrepancy, a closely related capacity estimate that can be calculated by Monte Carlo integration. Maximal discrepancy penalty functions are appealing for pattern classification problems, since their computation is equivalent to empirical risk minimization over the training data with some labels flipped.

Rademacher and Gaussian complexities: Risk bounds and structural results

We investigate the use of certain data-dependent estimates of the complexity of a function class, called Rademacher and Gaussian complexities. In a decision theoretic setting, we prove general risk bounds in terms of these complexities. We consider function classes that can be expressed as combinations of functions from basis classes and show how the Rademacher and Gaussian complexities of such a function class can be bounded in terms of the complexity of the basis classes. We give examples of the application of these techniques in finding data-dependent risk bounds for decision trees, neural networks and support vector machines.

Online discovery of similarity mappings

We consider the problem of choosing, sequentially, a map which assigns elements of a set A to a few elements of a set B. On each round, the algorithm suffers some cost associated with the chosen assignment, and the goal is to minimize the cumulative loss of these choices relative to the best map on the entire sequence. Even though the offline problem of finding the best map is provably hard, we show that there is an equivalent online approximation algorithm, Randomized Map Prediction (RMP), that is efficient and performs nearly as well. While drawing upon results from the "Online Prediction with Expert Advice" setting, we show how RMP can be utilized as an online approach to several standard batch problems. We apply RMP to online clustering as well as online feature selection and, surprisingly, RMP often outperforms the standard batch algorithms on these problems.

Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Sparseness vs estimating conditional probabilities : some asymptotic results

One of the nice properties of kernel classifiers such as SVMs is that they often produce sparse solutions. However, the decision functions of these classifiers cannot always be used to estimate the conditional probability of the class label. We investigate the relationship between these two properties and show that these are intimately related: sparseness does not occur when the conditional probabilities can be unambiguously estimated. We consider a family of convex loss functions and derive sharp asymptotic results for the fraction of data that becomes support vectors. This enables us to characterize the exact trade-off between sparseness and the ability to estimate conditional probabilities for these loss functions.

On the consistency of multiclass classification methods

Binary classification is a well studied special case of the classification problem. Statistical properties of binary classifiers, such as consistency, have been investigated in a variety of settings. Binary classification methods can be generalized in many ways to handle multiple classes. It turns out that one can lose consistency in generalizing a binary classification method to deal with multiple classes. We study a rich family of multiclass methods and provide a necessary and sufficient condition for their consistency. We illustrate our approach by applying it to some multiclass methods proposed in the literature.

Fault diagnostics and supervised testing : How fault diagnostic tools can be proactive?

The topic of fault detection and diagnostics (FDD) is studied from the perspective of proactive testing. Unlike most research focus in the diagnosis area in which system outputs are analyzed for diagnosis purposes, in this paper the focus is on the other side of the problem: manipulating system inputs for better diagnosis reasoning. In other words, the question of how diagnostic mechanisms can direct system inputs for better diagnosis analysis is addressed here. It is shown how the problem can be formulated as decision making problem coupled with a Bayesian Network based diagnostic mechanism. The developed mechanism is applied to the problem of supervised testing in HVAC systems.

Overcoming the complexity of diagnostic problems due to sensor network architecture

In fault detection and diagnostics, limitations coming from the sensor network architecture are one of the main challenges in evaluating a system’s health status. Usually the design of the sensor network architecture is not solely based on diagnostic purposes, other factors like controls, financial constraints, and practical limitations are also involved. As a result, it quite common to have one sensor (or one set of sensors) monitoring the behaviour of two or more components. This can significantly extend the complexity of diagnostic problems. In this paper a systematic approach is presented to deal with such complexities. It is shown how the problem can be formulated as a Bayesian network based diagnostic mechanism with latent variables. The developed approach is also applied to the problem of fault diagnosis in HVAC systems, an application area with considerable modeling and measurement constraints.

Predicting wave glider speed from environmental measurements

In the ocean science community, researchers have begun employing novel sensor platforms as integral pieces in oceanographic data collection, which have significantly advanced the study and prediction of complex and dynamic ocean phenomena. These innovative tools are able to provide scientists with data at unprecedented spatiotemporal resolutions. This paper focuses on the newly developed Wave Glider platform from Liquid Robotics. This vehicle produces forward motion by harvesting abundant natural energy from ocean waves, and provides a persistent ocean presence for detailed ocean observation. This study is targeted at determining a kinematic model for offline planning that provides an accurate estimation of the vehicle speed for a desired heading and set of environmental parameters. Given the significant wave height, ocean surface and subsurface currents, wind speed and direction, we present the formulation of a system identification to provide the vehicle’s speed over a range of possible directions.

Implementing dance in a Queensland primary school : a descriptive case study

This research documents and analyses the modes of implementation of the Dance component of The Arts Essential Learnings in a Queensland school. The research identifies what makes good practice in dance education and the multiple modes of implementation that support this practice. The primary purpose of this research is to describe the factors that influence dance education, as it is delivered, in a Queensland primary school to inform the development of further support for primary teachers and to improve the quality of dance education in Prep -7 schooling. The literature review investigates dance education both in Queensland, Australia and internationally, identifying current issues related to the delivery of dance in a primary school environment including barriers to implementation of dance, authentic learning and integrated approaches to learning. Based on Engestrom.s reformulation of Vygotsky.s theory of socially mediated learning, the implementation of dance education curriculum was explored through descriptive case study method. The case study was conducted in a regional Queensland school identified as delivering the dance curriculum in a variety of ways. The research project provided opportunities to observe, document and analyse how teachers deal with pedagogical dilemmas and solve logistical problems associated with teaching the dance component of the Arts curriculum in this school. Teachers. practices were contextualised through investigation of the whole school context of dance curriculum development. The findings revealed a range of teaching approaches that influenced teachers. interpretation and children.s experience of the dance curriculum. The features of a supportive whole school and cultural environment for dance were identified. These have been captured in a reworked version of Engestrom.s Second Generation Activity Theory that can be applied to the implementation of dance education in primary schools.

A transition program in first year law : adopting the First Year Curriculum Principles

The First Year Curriculum Principles espouse a student-focused consistent and explicit curriculum, acknowledging diversity and the need to scaffold skills and learning. Commencing law students are no different to other first year students in that they must deal with changes in teaching and learning approaches and expectations. As well as the generic issues of transition, law students must grapple with learning the skills which are necessary for the study of law from the very start of their degree. A transition program at the commencement of a law degree as part of a planned first year curriculum provides an opportunity to introduce students to the study of law, the requisite skills as well as assist with transition to tertiary education.

Course cohesion : an elusive goal for tertiary education

A program’s development and implementation in a higher education institution is usually launched with great fanfare, goodwill and a huge effort on the part of the whole development team to ensure a worthwhile cohesive set of learning experiences aligned to the desired course learning outcomes. It is often not long before the glue starts to come unstuck arising from staffing changes, subtle migration of course resources, opportunistic inclusions of “off the shelf” or unit based innovative teaching and learning approaches, and perhaps general poor attention to detail with regard to the impact of new introductions and electives. This paper presents an initial investigation into the elusive goal of achieving course cohesion. The authors consider building cohesion into a course as it is being designed through identified cohesion factors and in sustaining course cohesion through active leadership.

Microarray expression profiling in melanoma reveals a BRAF mutation signature

We have used microarray gene expression profiling and machine learning to predict the presence of BRAF mutations in a panel of 61 melanoma cell lines. The BRAF gene was found to be mutated in 42 samples (69%) and intragenic mutations of the NRAS gene were detected in seven samples (11%). No cell line carried mutations of both genes. Using support vector machines, we have built a classifier that differentiates between melanoma cell lines based on BRAF mutation status. As few as 83 genes are able to discriminate between BRAF mutant and BRAF wild-type samples with clear separation observed using hierarchical clustering. Multidimensional scaling was used to visualize the relationship between a BRAF mutation signature and that of a generalized mitogen-activated protein kinase (MAPK) activation (either BRAF or NRAS mutation) in the context of the discriminating gene list. We observed that samples carrying NRAS mutations lie somewhere between those with or without BRAF mutations. These observations suggest that there are gene-specific mutation signals in addition to a common MAPK activation that result from the pleiotropic effects of either BRAF or NRAS on other signaling pathways, leading to measurably different transcriptional changes.

Library and information science education 2.0 : guiding principles and models of best practice ALTC Fellowship Report

This ALTC Teaching Fellowship aimed to establish Guiding Principles for Library and Information Science Education 2.0. The aim was achieved by (i) identifying the current and anticipated skills and knowledge required by successful library and information science (LIS) professionals in the age of web 2.0 (and beyond), (ii) establishing the current state of LIS education in Australia in supporting the development of librarian 2.0, and in doing so, identify models of best practice. The fellowship has contributed to curriculum renewal in the LIS profession. It has helped to ensure that LIS education in Australia continues to meet the changing skills and knowledge requirements of the profession it supports. It has also provided a vehicle through which LIS professionals and LIS educators may find opportunities for greater collaboration and more open communication. This will help bridge the gap between LIS theory and practice and will foster more authentic engagement between LIS education and other parts of the LIS industry in the education of the next generation of professionals. Through this fellowship the LIS discipline has become a role model for other disciplines who will be facing similar issues in the coming years. Eighty-one members of the Australian LIS profession participated in a series of focus groups exploring the current and anticipated skills and knowledge needed by the LIS professional in the web 2.0 world and beyond. Whilst each focus group tended to draw on specific themes of interest to that particular group of people, there was a great deal of common ground. Eight key themes emerged: technology, learning and education, research or evidence-based practice, communication, collaboration and team work, user focus, business savvy and personal traits. It was acknowledged that the need for successful LIS professionals to possess transferable skills and interpersonal attributes was not new. It was noted however that the speed with which things are changing in the web 2.0 world was having a significant impact and that this faster pace is placing a new and unexpected emphasis on the transferable skills and knowledge. It was also acknowledged that all librarians need to possess these skills, knowledge and attributes and not just the one or two role models who lead the way. The most interesting finding however was that web 2.0, library 2.0 and librarian 2.0 represented a ‘watershed’ for the LIS profession. Almost all the focus groups spoke about how they are seeing and experiencing a culture change in the profession. Librarian 2.0 requires a ‘different mindset or attitude’. The Levels of Perspective model by Daniel Kim provides one lens by which to view this finding. The focus group findings suggest that we are witnessing a re-awaking of the Australian LIS profession as it begins to move towards the higher levels of Kim’s model (ie mental models, vision). Thirty-six LIS educators participated in telephone interviews aimed at exploring the current state of LIS education in supporting the development of librarian 2.0. Skills and knowledge of LIS professionals in a web 2.0 world that were identified and discussed by the LIS educators mirrored those highlighted in the focus group discussions with LIS professionals. Similarly it was noted that librarian 2.0 needed a focus less on skills and knowledge and more on attitude. However, whilst LIS professionals felt that there was a paradigm shift within the profession. LIS educators did not speak with one voice on this matter with quite a number of the educators suggesting that this might be ‘overstating it a bit’. This study provides evidence for “disparate viewpoints” (Hallam, 2007) between LIS educators and LIS professionals that can have a significant implications for the future of not just LIS professional education specifically but for the profession generally. Library and information science education 2.0: guiding principles and models of best practice 1 Inviting the LIS academics to discuss how their teaching and learning activities support the development of librarian 2.0 was a core part of the interviews conducted. The strategies used and the challenges faced by LIS educators in developing their teaching and learning approaches to support the formation of librarian 2.0 are identified and discussed. A core part of the fellowship was the identification of best practice examples on how LIS educators were developing librarian 2.0. Twelve best practice examples were identified. Each educator was recorded discussing his or her approach to teaching and learning. Videos of these interviews are available via the Fellowship blog at .The LIS educators involved in making the videos felt uncomfortable with the term ‘best practice’. Many acknowledged that there simply seeking to do the best by their students and that there was always room for improvement. For this reason these videos are offered as examples of “great practice”. The videos are a tool for other educators to use, regardless of discipline, in developing their teaching and learning approaches to supporting web 2.0 professionals. It has been argued that the main purpose of professional education is transformation (Dall’ Alba, 2009; Dall’Alba & Barnacle, 2007). As such professional education should focus not just on skills and knowledge acquisition but also on helping students to develop ways of being the professionals in question (ie LIS professionals, teachers, lawyers, engineers).The aim of this fellowship was to establish Guidelines for Library and Information Science Education 2.0 it has however become apparent that at this point in time it is not yet possible to fulfil this aim. The fellowship has clearly identified skills and knowledge needed by the LIS professional in web 2.0 world (and beyond). It has also identified examples of ‘great practice’ by LIS educators as they endeavour to develop LIS professionals who will be successful in a web 20 world. The fellowship however has also shown that the LIS profession is currently undergoing significant attitudinal and conceptual change. Consequently, before a philosophy of LIS education 2.0 can be expressed, the Australian LIS profession must first explore and articulate what it means to be an LIS professional in the 21st century (ie a world of web 2.0 and beyond). In short, the LIS profession in Australia must take stock not of “what we know and can do” but on “who we are becoming” (Dall’Alba, 2009, p 34).

Nurses driving change to improve access to education and quality in lcinical teaching : A partnership between Australia and Viet Nam

In Viet Nam, standards of nursing care fail to meet international competency standards. This increases risks to patient safety (eg. hospital acquired infection), consequently the Ministry of Health identified the need to strengthen nurse education in Viet Nam. This paper presents experiences of a piloted clinical teaching model developed in Ha Noi, to strengthen nurse led institutional capacity for in-service education and clinical teaching. Historically 90% of nursing education was conducted by physicians and professional development in hospitals for nurses was limited. There was minimal communication between hospitals and nursing schools about expectations of students and assessment and quality of the learning experience. As a result when students came to the clinical sites, no-one understood how to plan their learning objectives and utilise teaching and learning approaches appropriate to their level. Therefore student learning outcomes were variable. They focussed on procedures and techniques and “learning how to do” rather than learning how to plan, implement and evaluate patient care. This project is part of a multi-component capacity building program designed to improve nurse education in Viet Nam. The project was funded jointly by Queensland University of Technology (QUT) and the Australian Agency for International Development. Its aim was to develop a collaborative clinically-based model of teaching to create an environment that encourages evidence-based, student-centred clinical learning. Accordingly, strategies introduced promoted clinical teaching of competency based nursing practice utilising the regionally endorsed nurse core competency standards. Thirty nurse teachers from Viet Duc University Hospital and Hanoi Medical College participated in the program. These nurses and nurse teachers undertook face to face education in three workshops, and completed three assessment items. Assessment was applied, where participants integrated the concepts learned in each workshop and completed assessment tasks related to planning, implementing and evaluating teaching in the clinical area. Twenty of these participants were then selected to undertake a two week study tour in Brisbane, Australia where the clinical teaching model was refined and an action plan developed to integrate into both organisations with possible implementation across Viet Nam. Participants on this study tour also experienced clinical teaching and learning at QUT by attending classes held at the university, and were able to visit selected hospitals to experience clinical teaching in these settings as well. Effectiveness of the project was measured throughout the implementation phase and in follow up visits to the clinical site. To date changes have been noted on an individual and organisational level. There is also significant planning underway to incorporate the clinical teaching model developed across the organisation and how this may be implemented in other regions. Two participants have also been involved in disseminating aspects of this approach to clinical teaching in Ho Chi Minh, with further plans for more in-depth dissemination to occur throughout the country.