Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

University students’ regulation of learning and text processing – examples from medical and teacher education

The general aim of the thesis was to study university students’ learning from the perspective of regulation of learning and text processing. The data were collected from the two academic disciplines of medical and teacher education, which share the features of highly scheduled study, a multidisciplinary character, a complex relationship between theory and practice and a professional nature. Contemporary information society poses new challenges for learning, as it is not possible to learn all the information needed in a profession during a study programme. Therefore, it is increasingly important to learn how to think and learn independently, how to recognise gaps in and update one’s knowledge and how to deal with the huge amount of constantly changing information. In other words, it is critical to regulate one’s learning and to process text effectively. The thesis comprises five sub-studies that employed cross-sectional, longitudinal and experimental designs and multiple methods, from surveys to eye tracking. Study I examined the connections between students’ study orientations and the ways they regulate their learning. In total, 410 second-, fourth- and sixth-year medical students from two Finnish medical schools participated in the study by completing a questionnaire measuring both general study orientations and regulation strategies. The students were generally deeply oriented towards their studies. However, they regulated their studying externally. Several interesting and theoretically reasonable connections between the variables were found. For instance, self-regulation was positively correlated with deep orientation and achievement orientation and was negatively correlated with non-commitment. However, external regulation was likewise positively correlated with deep orientation and achievement orientation but also with surface orientation and systematic orientation. It is argued that external regulation might function as an effective coping strategy in the cognitively loaded medical curriculum. Study II focused on medical students’ regulation of learning and their conceptions of the learning environment in an innovative medical course where traditional lectures were combined wth problem-based learning (PBL) group work. First-year medical and dental students (N = 153) completed a questionnaire assessing their regulation strategies of learning and views about the PBL group work. The results indicated that external regulation and self-regulation of the learning content were the most typical regulation strategies among the participants. In line with previous studies, self-regulation wasconnected with study success. Strictly organised PBL sessions were not considered as useful as lectures, although the students’ views of the teacher/tutor and the group were mainly positive. Therefore, developers of teaching methods are challenged to think of new solutions that facilitate reflection of one’s learning and that improve the development of self-regulation. In Study III, a person-centred approach to studying regulation strategies was employed, in contrast to the traditional variable-centred approach used in Study I and Study II. The aim of Study III was to identify different regulation strategy profiles among medical students (N = 162) across time and to examine to what extent these profiles predict study success in preclinical studies. Four regulation strategy profiles were identified, and connections with study success were found. Students with the lowest self-regulation and with an increasing lack of regulation performed worse than the other groups. As the person-centred approach enables us to individualise students with diverse regulation patterns, it could be used in supporting student learning and in facilitating the early diagnosis of learning difficulties. In Study IV, 91 student teachers participated in a pre-test/post-test design where they answered open-ended questions about a complex science concept both before and after reading either a traditional, expository science text or a refutational text that prompted the reader to change his/her beliefs according to scientific beliefs about the phenomenon. The student teachers completed a questionnaire concerning their regulation and processing strategies. The results showed that the students’ understanding improved after text reading intervention and that refutational text promoted understanding better than the traditional text. Additionally, regulation and processing strategies were found to be connected with understanding the science phenomenon. A weak trend showed that weaker learners would benefit more from the refutational text. It seems that learners with effective learning strategies are able to pick out the relevant content regardless of the text type, whereas weaker learners might benefit from refutational parts that contrast the most typical misconceptions with scientific views. The purpose of Study V was to use eye tracking to determine how third-year medical studets (n = 39) and internal medicine residents (n = 13) read and solve patient case texts. The results revealed differences between medical students and residents in processing patient case texts; compared to the students, the residents were more accurate in their diagnoses and processed the texts significantly faster and with a lower number of fixations. Different reading patterns were also found. The observed differences between medical students and residents in processing patient case texts could be used in medical education to model expert reasoning and to teach how a good medical text should be constructed. The main findings of the thesis indicate that even among very selected student populations, such as high-achieving medical students or student teachers, there seems to be a lot of variation in regulation strategies of learning and text processing. As these learning strategies are related to successful studying, students enter educational programmes with rather different chances of managing and achieving success. Further, the ways of engaging in learning seldom centre on a single strategy or approach; rather, students seem to combine several strategies to a certain degree. Sometimes, it can be a matter of perspective of which way of learning can be considered best; therefore, the reality of studying in higher education is often more complicated than the simplistic view of self-regulation as a good quality and external regulation as a harmful quality. The beginning of university studies may be stressful for many, as the gap between high school and university studies is huge and those strategies that were adequate during high school might not work as well in higher education. Therefore, it is important to map students’ learning strategies and to encourage them to engage in using high-quality learning strategies from the beginning. Instead of separate courses on learning skills, the integration of these skills into course contents should be considered. Furthermore, learning complex scientific phenomena could be facilitated by paying attention to high-quality learning materials and texts and other support from the learning environment also in the university. Eye tracking seems to have great potential in evaluating performance and growing diagnostic expertise in text processing, although more research using texts as stimulus is needed. Both medical and teacher education programmes and the professions themselves are challenging in terms of their multidisciplinary nature and increasing amounts of information and therefore require good lifelong learning skills during the study period and later in work life.

A study of the role learning and socialization during the transition from teacher to elementary school administrator

This thesis examined the role transition from an elementary teacher to an elementary principal. In particular, the training and socialization process of becoming an elementary principal was explored through the study of the hierarchical and political structure of a southern Ontario school board, and how this influenced the learning experiences of new elementary principals. A qualitative methodology, with a grounded theory design, was employed to investigate this process through interviews with 10 participants to examine their experiences and role learning occurs during their development. Specifically, participants perspective shifts, developmental experiences, understanding of group culture, and expansion of a board profile were highlighted in the data. One of the compelling results of the study was the degree to which principals of aspiring administrators influence the socialization of their subordinates. The beliefs and practices of the school principal determine the socialization orientation that teachers and vice-principals will experience during role learning. The results of this study also imply that role orientation needs to be understood as a continuum between custodial and innovative role assumption. Varying degrees of custodianship or innovation depended on the context of the administrative placement and the personal attributes of administrative candidates. Principals who are willing to share responsibilities, who are good communicators, and who wish to develop a collaborative relationship with their viceprincipals are the individuals the participants in this study described as making the best mentors.

Linking classroom learning and research to advance ideas about social-ecological resilience

There is an increasing demand in higher education institutions for training in complex environmental problems. Such training requires a careful mix of conventional methods and innovative solutions, a task not always easy to accomplish. In this paper we review literature on this theme, highlight relevant advances in the pedagogical literature, and report on some examples resulting from our recent efforts to teach complex environmental issues. The examples range from full credit courses in sustainable development and research methods to project-based and in-class activity units. A consensus from the literature is that lectures are not sufficient to fully engage students in these issues. A conclusion from the review of examples is that problem-based and project-based, e.g., through case studies, experiential learning opportunities, or real-world applications, learning offers much promise. This could greatly be facilitated by online hubs through which teachers, students, and other members of the practitioner and academic community share experiences in teaching and research, the way that we have done here.

The "reflection protocol" - a way to develop learning and independent thinking?

It is now-a-days more and more common in the academic world to use new forms of “learning-tools”. One of those is the “reflection protocol”, which usually consist of a few pages of freely written text, related to something the students have read. There seems to be a lot of different opinions about the value to use this method. Some teachers and students are enthusiastic and others are rather critical. To write a “reflection protocol” is not in the first place to do a summery, a review, not even to analyze a text. Instead it is about to write down thoughts and questions that comes up as a result of the reading. It is also about doing associations, reflections and to interpret a text and relate this to a theme of some kind. The purpose to use “reflection protocols” is, as we see it, mainly for the student to practice independent thinking from a scientific point of view, but it also gives a possibility to a better understanding of another person’s thinking. This seems to open up for a fruitful dialogue and a way to learn. We will in this paper discuss if that could be the case.

An easy-learning and easy-teaching tool for indoor thermal analysis - ArcTech

ArcTech is a software being developed, applied and improved with the aim of becoming an efficient sensitization tool to support the teaching-learning process of Architecture courses. The application deals initially with the thermal comfort of buildings. The output generated by the software shows if a student is able to produce a pleasant environment, in terms of thermal sensation along a 24-hours period. Although one can find the very same characteristics in fully-developed commercial software, the reason to create ArcTech is related to the flexibility of the system to be adapted by the instructor and also to the need of simple tools for the evaluation of specific topics along the courses. The first part of ArcTech is dedicated to data management and that was developed using the visual programming language Delphi 7 and Firebird as the database management system. The second part contains the parameters that can be changed by the system administrator and those related to project visualization. The interface of the system, in which the student will learn how to implement and to evaluate the project alternatives, was built using Macromedia Flash. The software was applied to undergraduate students revealing its easy-learning and easy-teaching interface.

Training manual: innovative fiscal and regulatory incentives for energy efficiency and renewable energy initiatives

This course will be designed for the officers within government departments who have responsibilty for guiding the country’s energy policy and energy management framework. Other stakeholders also will include private sector representatives who have interest in providing energy efficiency equipment and renewable energy solutions to the market towards advancing improvements in both energy efficiency and meeting renewable energy targets. The course will provide insight into all aspects of energy management with specific emphasis on energy efficiency as well as renewable energy. Emphasis will be placed on highlighting issues and challenges that countries face in pursuing energy efficiency and renewable energy strategies. International and regional best practices will be highlighted as a means of showcasing how various countries have overcome the barriers to advancing renewable energy targets and increasing energy efficiencies towards meeting national energy goals. The curriculum is divided into five modules and is designed to be covered over a 3-day period. The course will be designed to ensure practical application of the learning. The course also is designed to enable the Caribbean to demonstrate leadership in energy efficiency practices and the adoption of renewable energy strategies, serving as a model for other small island developing states.

Methods and Tools for Objective Assessment of Psychomotor Skills in Laparoscopic Surgery

Training and assessment paradigms for laparoscopic surgical skills are evolving from traditional mentor–trainee tutorship towards structured, more objective and safer programs. Accreditation of surgeons requires reaching a consensus on metrics and tasks used to assess surgeons’ psychomotor skills. Ongoing development of tracking systems and software solutions has allowed for the expansion of novel training and assessment means in laparoscopy. The current challenge is to adapt and include these systems within training programs, and to exploit their possibilities for evaluation purposes. This paper describes the state of the art in research on measuring and assessing psychomotor laparoscopic skills. It gives an overview on tracking systems as well as on metrics and advanced statistical and machine learning techniques employed for evaluation purposes. The later ones have a potential to be used as an aid in deciding on the surgical competence level, which is an important aspect when accreditation of the surgeons in particular, and patient safety in general, are considered. The prospective of these methods and tools make them complementary means for surgical assessment of motor skills, especially in the early stages of training. Successful examples such as the Fundamentals of Laparoscopic Surgery should help drive a paradigm change to structured curricula based on objective parameters. These may improve the accreditation of new surgeons, as well as optimize their already overloaded training schedules.

Enhanced recommendations for e-learning authoring tools based on a proactive context-aware recommender

Authoring tools are powerful systems in the area of e-Learning that make easier for teachers to create new learning objects by reusing or editing existing educational resources coming from learning repositories or content providers. However, due to the overwhelming number of resources these tools can access, sometimes it is difficult for teachers to find the most suitable resources taking into account their needs in terms of content (e.g. topic) or pedagogical aspects (e.g. target level associated to their students). Recommender systems can take an important role trying to mitigate this problem. In this paper we propose a new model to generate proactive context-aware recommendations on resources during the creation process of a new learning object that a teacher carries out by using an authoring tool. The common use cases covered by the model for having recommendations in online authoring tools and details about the recommender model itself are presented.

An Architecture by means of Anthropology. Beyond learning the tools of social science

Los arquitectos y urbanistas tienen una larga tradición en el aprendizaje de las herramientas de las ciencias sociales, especialmente las que les permiten analizar y describir mejor los entornos y las personas para las que trabajan. Esto ha llevado a los arquitectos a desarrollar mejores herramientas de observación y descripción del ámbito social y no sólo el material. Sin embargo, la mayoría de las veces este acercamiento interdisciplinar ha identificado las ciencias sociales, especialmente la antropología, con la etnografía. Este artículo parte de la crítica a esta identificación hecha por el antropólogo Tim Ingold y se centra en lo que él propone como el método central de la antropología, la observación participante. Para después revisar varias propuestas actuales de científicos sociales que tratan de desarrollar una disciplina no representacional y orientada al futuro, un objetivo más cercano al de la arquitectura. El artículo intenta imaginar cómo esta práctica transdisciplinar podría desarrollarse.

Teaching College Writing in a High School Setting: The Impact of Concurrent Enrollment on Teacher Learning and Practice

Thesis (Ph.D.)--University of Washington, 2016-06

Integrating science and management through collaborative learning and better information management

Achieving more sustainable land and water use depends on high-quality information and its improved use. In other words, better linkages are needed between science and management. Since many stakeholders with different relationships to the natural resources are inevitably involved, we suggest that collaborative learning environments and improved information management are prerequisites for integrating science and management. Case studies that deal with resource management issues are presented that illustrate the creation of collaborative learning environments through systems analyses with communities, and an integration of scientific and experiential knowledge of components of the system. This new knowledge needs to be captured and made accessible through innovative information management systems designed collaboratively with users, in forms which fit the users' 'mental models' of how their systems work. A model for linking science and resource management more effectively is suggested. This model entails systems thinking in a collaborative learning environment, and processes to help convergence of views and value systems, and make scientists and different kinds of managers aware of their interdependence. Adaptive management provides a mechanism for applying and refining scientists' and managers' knowledge. Copyright (C) 2003 John Wiley Sons, Ltd.

Good practice guide in learning and teaching

Once again this publication is produced to celebrate and promote good teaching and learning support and to offer encouragement to those imaginative and innovative staff who continue to wish to challenge students to learn to maximum effect. It is hoped that others will pick up some good ideas from the articles contained in this volume. We have again changed our approach for this 2006/07 edition (our fourth) of the Aston Business School Good Practice Guide. As before, some contributions were selected from those identifying interesting best practice on their Annual Module reflection forms in 2005/2006. Other contributors received HELM (Research Centre in Higher Education Learning and Management) small research grants in 2005/2006. Part of the conditions were for them to write an article for this publication. We have also been less tight on the length of the articles this year. Some contributions are, therefore, on the way to being journal articles. HELM will be working with these authors to help develop these for publication. The themes covered in this year?s articles are all central to the issues faced by those providing HE teaching and learning opportunities in the 21st Century. Specifically this is providing support and feedback to students in large classes, embracing new uses of technology to encourage active learning and addressing cultural issues in a diverse student population. Michael Grojean and Yves Guillaume used Blackboard™ to give a more interactive learning experience and improve feedback to students. It would be easy for other staff to adopt this approach. Patrick Tissington and Qin Zhou (HELM small research grant holders) were keen to improve the efficiency of student support, as does Roger McDermott. Celine Chew shares her action learning project, completed as part of the Aston University PG Certificate in Teaching and Learning. Her use of Blackboard™ puts emphasis on the learner having to do something to help them meet the learning outcomes. This is what learning should be like, but many of our students seem used to a more passive learning experience, so much needs to be done on changing expectations and cultures about learning. Regina Herzfeldt also looks at cultures. She was awarded a HELM small research grant and carried out some significant new research on cultural diversity in ABS and what it means for developing teaching methods. Her results fit in with what many of us are experiencing in practice. Gina leaves us with some challenges for the future. Her paper certainly needs to be published. This volume finishes with Stuart Cooper and Matt Davies reflecting on how to keep students busy in lectures and Pavel Albores working with students on podcasting. Pavel?s work, which was the result of another HELM small research grant, will also be prepared for publication as a journal article. The students learnt more from this work that any formal lecture and Pavel will be using the approach again this year. Some staff have been awarded HELM small research grants in 2006/07 and these will be published in the next Good Practice Guide. In the second volume we mentioned the launch of the School?s Research Centre in Higher Education Learning and Management (HELM). Since then HELM has stimulated a lot of activity across the School (and University) particularly linking research and teaching. A list of the HELM seminars for 2006/2007 is listed as Appendix 1 of this publication. Further details can be obtained from Catherine Foster (c.s.foster@aston.ac.uk), who coordinates the HELM seminars. For 2006 and 2005 HELM listed, 20 refereed journal articles, 7 book chapters, 1 published conference papers, 20 conference presentations, two official reports, nine working papers and £71,535 of grant money produced in this research area across the School. I hope that this shows that reflection on learning is alive and well in ABS. We have also been working on a list of target journals to guide ABS staff who wish to publish in this area. These are included as Appendix 2 of this publication. May I thank the contributors for taking time out of their busy schedules to write the articles and to Julie Green, the Quality Manager, for putting the varying diverse approaches into a coherent and publishable form and for agreeing to fund the printing of this volume.

Good practice guide in learning and teaching

Once again this publication is produced to celebrate and promote good teaching and learning support and to offer encouragement to those imaginative and innovative staff who continue to wish to challenge students to learn to maximum effect. It is hoped that others will pick up some good ideas from the articles contained in this volume. We had changed our editorial approach in drawing together the articles for this 2005/6 edition (our third) of the ABS Good Practice Guide. Firstly we have expanded our contributors beyond ABS academics. This year?s articles have also been written by staff from other areas of the University, a PhD student, a post-doctoral researcher and staff working in learning support. We see this as an acknowledgement that the learning environment involves a range of people in the process of student support. We have also expanded the maximum length of the articles from two to five pages, in order to allow greater reflection on the issues. The themes of the papers cluster around issues relating to diversity (widening participation and internationalisation of the student body), imaginative use of new technology (electronic reading on BlackboardTM ) and reflective practitioners, (reflection on rigour and relevance; on how best to train students in research ethics, relevance in the curriculum and the creativity of the teaching process) Discussion of efforts to train the HE teachers of the future looks forward to the next academic year when the Higher Education Academy?s professional standards will be introduced across the sector. In the last volume we mentioned the launch of the School?s Research Centre in Higher Education Learning and Management (HELM). Since then HELM has stimulated a lot of activity across the School (and University) particularly linking research and teaching. A list of the HELM seminars is listed as an appendix to this publication. Further details can be obtained from Catherine Foster (c.s.foster@aston.ac.uk) who coordinates the HELM seminars. HELM has also won its first independent grant from the EU Leonardo programme to look at the effect of business education on employment. In its annual report to the ABS Research Committee HELM listed for 2004 and 2005, 11 refereed journal articles, 4 book chapters, 3 published conference papers, 18 conference papers, one official reports and £72,500 of grant money produced in this research area across the School. I hope that this shows that reflection on learning is live and well in ABS. May I thank the contributors for taking time out of their busy schedules to write the articles and to Julie Green, the Quality Manager, for putting our diverse approaches into a coherent and publishable form.

Good practice guide in learning and teaching

This is the second edition of our Aston Business School (ABS) Good Practice Guide and the enthusiasm of the contributors appears undiminished. I am again reminded that I work with a group of very committed, dedicated and professional colleagues. Once again this publication is produced to celebrate and promote good teaching across the School and to offer encouragement to those imaginative and innovative staff who continue to wish to challenge students to learn to maximum effect. It is hoped that others will pick up some good ideas from the articles contained in this volume. Contributors to this Guide were not chosen because they are the best teachers in the School, although they are undoubtedly all amongst my colleagues who are exponents of enthusiastic and inspiring approaches to learning. The Quality Unit approached these individuals because they declared on their Annual Module Reflection Forms that they were doing something interesting and worthwhile which they thought others might find useful. Amongst those reading the Guide I am sure that there are many other individuals who are trying to operate similar examples of good practice in their teaching, learning and assessment methods. I hope that this publication will provoke these people into providing comments and articles of their own and that these will form the basis of next year’s Guide. It may also provoke some people to try these methods in their own teaching. The themes of the articles this year can be divided into two groups. The first theme is the quest to help students to help themselves to learn via student-run tutorials, surprise tests and mock examinations linked with individual tutorials. The second theme is making learning come to life in exciting practical ways by, for example, hands-on workshops and simulations, story telling, rhetorical questioning and discussion groups. A common theme is one of enthusiasm, reflection and commitment on behalf of the lecturers concerned. None of the approaches discussed in this publication are low effort activities on the part of the facilitator, but this effort is regarded as worthwhile as a means of creating greater student engagement. As Biggs (2003)[1] says, in his similarly inspiring way, students learn more the less passive they are in their learning. (Ref). The articles in this publication bear witness of this and much more. Since last year Aston Business School has launched its Research Centre in Higher Education Learning and Management (HELM) which is another initiative to promote excellent learning and teaching. Even before this institution has become fully operational, at least one of the articles in this publication has seen the light of day in the research arena and at least two others are ripe for dissemination to a wider audience via journal publication. More news of our successes in this activity will appear in next year’s edition. May I thank the contributors for taking time out of their busy schedules to write the articles this summer, and to Julie Green who runs the ABS Quality Unit, for putting our diverse approaches into a coherent and publishable form and for chasing us when we have needed it! I would also like to thank Ann Morton and her colleagues in the Centre for Staff Development who have supported this publication. During the last year the Centre has further stimulated the learning and teaching life of the School (and the wider University) via their Learning and Teaching Week and sponsorship of Teaching Quality Enhancement Fund (TQEF) projects. Pedagogic excellence is in better health at Aston than ever before – long may this be because this is what life in HE should be about.