Hunting and gathering : new imperatives in mapping and collecting student learning data to assure quality outcomes

Assurance of learning (AOL) is a quality enhancement and quality assurance process used in higher education. It involves a process of determining programme learning outcomes and standards, and systematically gathering evidence to measure students' performance on these. The systematic assessment of whole-of-programme outcomes provides a basis for curriculum development and management, continuous improvement, and accreditation. To better understand how AOL processes operate, a national study of university practices across one discipline area, business and management, was undertaken. To solicit data on AOL practice, interviews were undertaken with a sample of business school representatives (n = 25). Two key processes emerged: (1) mapping of graduate attributes and (2) collection of assurance data. External drivers such as professional accreditation and government legislation were the primary reasons for undertaking AOL outcomes but intrinsic motivators in relation to continuous improvement were also evident. The facilitation of academic commitment was achieved through an embedded approach to AOL by the majority of universities in the study. A sustainable and inclusive process of AOL was seen to support wider stakeholder engagement in the development of higher education learning outcomes.

Hunters & gatherers : strategies for curriculum mapping and data collection for assurance of learning

Assurance of learning is a predominant feature in both quality enhancement and assurance in higher education. Assurance of learning is a process that articulates explicit program outcomes and standards, and systematically gathers evidence to determine the extent to which performance matches expectations. Benefits accrue to the institution through the systematic assessment of whole of program goals. Data may be used for continuous improvement, program development, and to inform external accreditation and evaluation bodies. Recent developments, including the introduction of the Tertiary Education and Quality Standards Agency (TEQSA) will require universities to review the methods they use to assure learning outcomes. This project investigates two critical elements of assurance of learning: 1. the mapping of graduate attributes throughout a program; and 2. the collection of assurance of learning data. An audit was conducted with 25 of the 39 Business Schools in Australian universities to identify current methods of mapping graduate attributes and for collecting assurance of learning data across degree programs, as well as a review of the key challenges faced in these areas. Our findings indicate that external drivers like professional body accreditation (for example: Association to Advance Collegiate Schools of Business (AACSB)) and TEQSA are important motivators for assuring learning, and those who were undertaking AACSB accreditation had more robust assurance of learning systems in place. It was reassuring to see that the majority of institutions (96%) had adopted an embedding approach to assuring learning rather than opting for independent standardised testing. The main challenges that were evident were the development of sustainable processes that were not considered a burden to academic staff, and obtainment of academic buy in to the benefits of assuring learning per se rather than assurance of learning being seen as a tick box exercise. This cultural change is the real challenge in assurance of learning practice.

Do perceptions mirror reality? Student perceptions of learning versus grade outcomes

This research addresses whether educators should consider measuring if students have learned what was intended, as recommended by education researchers. Students in an Introductory Marketing subject were asked to complete a voluntary survey rating their own progress on the intended learning outcomes for the course. One hundred and one surveys were completed by students in the second-last teaching week of the semester. Student identification numbers were used to link student perceptions with their grade outcomes. Regression analysis was used to ascertain whether student perceptions of their progress on the intended learning outcomes for the course could be used to predict their grades. While the results were significant, student perceptions of their progress on learning outcomes were a poor predictor of grade outcomes. The results of this study suggest that student perceptions may not mirror the reality. These results are somewhat surprising and future research examining the degree of change in the learning outcomes perceived by students is warranted. This will further contribute to decisions surrounding whether educators should measure if students have learned what was intended.

‘Show Me Your Wiki and I’ll Show you Mine’: Using Online Interactive Media to Improve Academic Writing and Research in a Public Health Under-Graduate Cohort

The number of Internet users in Australia has been steadily increasing, with over 10.9 million people currently subscribed to an internet provider (ABS, 2011). Over the past year, the most avid users of the Internet were 15 – 24 year olds, with approximately 95% accessing the internet on a regular basis (ABS, Social Trends, 2011). While the internet has been described as fundamental to higher education students, social and leisure internet tools are also increasingly being used by these students to generate and maintain their social and professional networks and interactions (Duffy & Bruns 2006). Rapid technological advancements have enabled greater and faster access to information for learning and education (Hemmi et al, 2009; Glassman and Kang, 2011). As such, we sought to integrate interactive, online social media into the assessment profile of a Public Health undergraduate cohort at the Queensland University of Technology (QUT). The aim of this exercise was to engage students to both develop and showcase their research on a range of complex, contemporary health issues within the online forum of Wikispaces (http://www.wikispaces.com/) for review and critique by their peers. We applied Bandura’s Social Learning Theory (SLT) to analyse the interactive processes from which students developed deeper and more sustained learning, and via which their overall academic writing standards were raised. This paper outlines the assessment task, and the students’ feedback on their learning outcomes in relation to the Attentional, Retentional, Motor Reproduction, and Motivational Processes outlined by Bandura in SLT. We conceptualise the findings in a theoretical model, and discuss the implications for this approach within the broader tertiary environment.

Australian Learning and Teaching Council, Learning and Teaching Academic Standards Statement for Architecture

The development of the Learning and Teaching Academic Standards Statement for Architecture (the Statement) centred on requirements for the Master of Architecture and proceeded alongside similar developments in the building and construction discipline under the guidance and support of the Australian Deans of Built Environment and Design (ADBED). Through their representation of Australian architecture programs, ADBED have provided high-level leadership for the Learning and Teaching Academic Standards Project in Architecture (LTAS Architecture). The threshold learning outcomes (TLOs), the description of the nature and extent of the discipline, and accompanying notes were developed through wide consultation with the discipline and profession nationally. They have been considered and debated by ADBED on a number of occasions and have, in their fi nal form, been strongly endorsed by the Deans. ADBED formed the core of the Architecture Reference Group (chaired by an ADBED member) that drew together representatives of every peak organisation for the profession and discipline in Australia. The views of the architectural education community and profession have been provided both through individual submissions and the voices of a number of peak bodies. Over two hundred individuals from the practising profession, the academic workforce and the student cohort have worked together to build consensus about the capabilities expected of a graduate of an Australian Master of Architecture degree. It was critical from the outset that the Statement should embrace the wisdom of the greater ‘tribe’, should ensure that graduates of the Australian Master of Architecture were eligible for professional registration and, at the same time, should allow for scope and diversity in the shape of Australian architectural education. A consultation strategy adopted by the Discipline Scholar involved meetings and workshops in Perth, Melbourne, Sydney, Canberra and Brisbane. Stakeholders from all jurisdictions and most universities participated in the early phases of consultation through a series of workshops that concluded late in October 2010. The Draft Architecture Standards Statement was formed from these early meetings and consultation in respect of that document continued through early 2011. This publication represents the outcomes of work to establish an agreed standards statement for the Master of Architecture. Significant further work remains to ensure the alignment of professional accreditation and recognition procedures with emerging regulatory frameworks cascading from the establishment of the Tertiary Education Quality and Standards Agency (TEQSA). The Australian architecture community hopes that mechanisms can be found to integrate TEQSA’s quality assurance purpose with well-established and understood systems of professional accreditation to ensure the good standing of Australian architectural education into the future. The work to build renewed and integrated quality assurance processes and to foster the interests of this project will continue, for at least the next eighteen months, under the auspices of Australian Learning and Teaching Council (ALTC)-funded Architecture Discipline Network (ADN), led by ADBED and Queensland University of Technology. The Discipline Scholar gratefully acknowledges the generous contributions given by those in stakeholder communities to the formulation of the Statement. Professional and academic colleagues have travelled and gathered to shape the Standards Statement. Debate has been vigorous and spirited and the Statement is rich with the purpose, critical thinking and good judgement of the Australian architectural education community. The commitments made to the processes that have produced this Statement reflect a deep and abiding interest by the constituency in architectural education. This commitment bodes well for the vibrancy and productivity of the emergent Architecture Discipline Network (ADN). Endorsement, in writing, was received from the Australian Institute of Architects National Education Committee (AIA NEC): The National Education Committee (NEC) of the Australian Institute of Architects thank you for your work thus far in developing the Learning and Teaching Academic Standards for Architecture In particular, we acknowledge your close consultation with the NEC on the project along with a comprehensive cross-section of the professional and academic communities in architecture. The TLOs with the nuanced levels of capacities – to identify, develop, explain, demonstrate etc – are described at an appropriate level to be understood as minimum expectations for a Master of Architecture graduate. The Architects Accreditation Council of Australia (AACA) has noted: There is a clear correlation between the current processes for accreditation and what may be the procedures in the future following the current review. The requirement of the outcomes as outlined in the draft paper to demonstrate capability is an appropriate way of expressing the measure of whether the learning outcomes have been achieved. The measure of capability as described in the outcome statements is enhanced with explanatory descriptions in the accompanying notes.

Building sustainable and effective assurance of learning process in a changing higher education environment

In a study of assuring learning in Australian Business Schools, 25 Teaching and Learning Associate Deans were interviewed to identify current issues in developing and measuring the quality of teaching and learning outcomes. Results indicate that for most institutions developing a perspective on graduate attributes and mapping assessments to measure outcomes across an entire program required knowledge creation and the building of new inclusive processes. Common elements of effective practice, namely those which offered consistently superior outcomes, included: inclusive processes; embedded graduate attributes throughout a program; alongside consistent and appropriate assessment. Results indicate that assurance of learning processes are proliferating nationally while quality of teaching and learning outcomes and in the processes for assuring it is increasing as a result.

Bringing the real world into university learning

Introduction: As part of ongoing quality assurance, all university programs must be regularly reviewed to ensure curriculum is current, meets university and national standards, and for medical science, criteria for AIMS Accreditation. With recent developments at the national and international level also signaling change, a course design team (CDT) was assembled and tasked with developing and implementing a new four year Bachelor of Medical Laboratory Science (BMLS) course at QUT. Method: A whole-of-course approach was adopted, incorporating inverted curriculum and Capstone experience. First, course vision and desired graduate profile are defined as course learning outcomes (CLO), i.e. skills, knowledge, behaviours and attributes graduates must demonstrate. CLO are then back-mapped into introductory, developmental and expected phases from fourth to first year on a course plan and assessment map. Unit learning outcomes (ULO) are then defined, and finally, each unit (subject) designed, directly aligned with assessment. Results: The resulting BMLS course represents a deliberate program of study across four years, which from day one, focuses on the professional aspects of MLS, clinical pathology disciplines, and incrementally developing and assessing the skills, knowledge, behaviours and attributes required to undertake the Work Integrated Learning Internship (WILI) and Capstone experience in final year, and subsequently, graduate from the program. Conclusions: At the start of the year, the BMLS commenced with higher than anticipated enrolments. To date, survey data and feedback is positive, with particular emphasis on the directed nature of the course. The method of course design also ensures university/national standards, and criteria for AIMS Accreditation have been met.

Systematic evaluation of GoSoapBox in tertiary education : a student response system for improving learning experiences and outcomes

There is currently a wide range of research into the recent introduction of student response systems in higher education and tertiary settings (Banks 2006; Kay and Le Sange, 2009; Beatty and Gerace 2009; Lantz 2010; Sprague and Dahl 2009). However, most of this pedagogical literature has generated ‘how to’ approaches regarding the use of ‘clickers’, keypads, and similar response technologies. There are currently no systematic reviews on the effectiveness of ‘GoSoapBox’ – a more recent, and increasingly popular student response system – for its capacity to enhance critical thinking, and achieve sustained learning outcomes. With rapid developments in teaching and learning technologies across all undergraduate disciplines, there is a need to obtain comprehensive, evidence-based advice on these types of technologies, their uses, and overall efficacy. This paper addresses this current gap in knowledge. Our teaching team, in an undergraduate Sociology and Public Health unit at the Queensland University of Technology (QUT), introduced GoSoapBox as a mechanism for discussing controversial topics, such as sexuality, gender, economics, religion, and politics during lectures, and to take opinion polls on social and cultural issues affecting human health. We also used this new teaching technology to allow students to interact with each other during class – both on both social and academic topics – and to generate discussions and debates during lectures. The paper reports on a data-driven study into how this interactive online tool worked to improve engagement and the quality of academic work produced by students. This paper will firstly, cover the recent literature reviewing student response systems in tertiary settings. Secondly, it will outline the theoretical framework used to generate this pedagogical research. In keeping with the social and collaborative features of Web 2.0 technologies, Bandura’s Social Learning Theory (SLT) will be applied here to investigate the effectiveness of GoSoapBox as an online tool for improving learning experiences and the quality of academic output by students. Bandura has emphasised the Internet as a tool for ‘self-controlled learning’ (Bandura 2001), as it provides the education sector with an opportunity to reconceptualise the relationship between learning and thinking (Glassman & Kang 2011). Thirdly, we describe the methods used to implement the use of GoSoapBox in our lectures and tutorials, and which aspects of the technology we drew on for learning purposes, as well as the methods for obtaining feedback from the students about the effectiveness or otherwise of this tool. Fourthly, we report cover findings from an examination of all student/staff activity on GoSoapBox as well as reports from students about the benefits and limitations of it as a learning aid. We then display a theoretical model that is produced via an iterative analytical process between SLT and our data analysis for use by academics and teachers across the undergraduate curriculum. The model has implications for all teachers considering the use of student response systems to improve the learning experiences of their students. Finally, we consider some of the negative aspects of GoSoapBox as a learning aid.

A little learning is a dangerous thing? The ethics of teaching accidental tourists in higher education

This paper raises questions about the ethical issues that arise for academics and universities when under-graduate students enrol in classes outside of their discipline - classes that are not designed to be multi-disciplinary or introductory. We term these students ‘accidental tourists'. Differences between disciplines in terms of pedagogy, norms, language and understanding may pose challenges for accidental tourists in achieving desired learning outcomes. This paper begins a discussion about whether lecturers and universities have any ethical obligations towards supporting the learning of these students. Recognising that engaging with only one ethical theory leads to a fragmented moral vision, this paper employs a variety of ethical theories to examine any possible moral obligations that may fall upon lecturers and/or universities. In regards to lecturers, the paper critically engages with the ethical theories of utilitarianism, Kantianism and virtue ethics (Aristotle) to determine the extent of any academic duty to accidental tourists. In relation to universities, this paper employs the emerging ethical theory of organisational ethics as a lens through which to critically examine any possible obligations. Organisational ethics stems from the recognition that moral demands also exist for organisations so organisations must be reconceptualised as ethical actors and their policies and practices subject to ethical scrutiny. The analysis in this paper illustrates the challenges faced by lecturers some of whom, we theorise, may experience a form of moral distress facing a conflict between personal beliefs and organisational requirements. It also critically examines the role and responsibilities of universities towards students and towards their staff and the inherent moral tensions between a market model and demands for ‘good' learning experiences. This paper highlights the tensions for academics, between academics and universities and within university policy and indicates the need for greater reflection about this issue, especially given the many constraints facing lecturers and universities.

Comparing different approach and avoidance models of learning and personality in the prediction of work, university, and leadership outcomes

Jackson (2005) developed a hybrid model of personality and learning, known as the learning styles profiler (LSP) which was designed to span biological, socio-cognitive, and experiential research foci of personality and learning research. The hybrid model argues that functional and dysfunctional learning outcomes can be best understood in terms of how cognitions and experiences control, discipline, and re-express the biologically based scale of sensation-seeking. In two studies with part-time workers undertaking tertiary education (N equals 137 and 58), established models of approach and avoidance from each of the three different research foci were compared with Jackson's hybrid model in their predictiveness of leadership, work, and university outcomes using self-report and supervisor ratings. Results showed that the hybrid model was generally optimal and, as hypothesized, that goal orientation was a mediator of sensation-seeking on outcomes (work performance, university performance, leader behaviours, and counterproductive work behaviour). Our studies suggest that the hybrid model has considerable promise as a predictor of work and educational outcomes as well as dysfunctional outcomes.

The peer leader capacity building model: A student learning journey

The benefits of peer leader experiences in building graduate skills and capabilities, is well documented and recognised in the higher education sector (Ender & Kay, 2001; Lindsey, Weiler, Zarich, Haddock, Krafchick, & Zimmerman, 2014; Shook & Keup, J., 2012). While benefits are acknowledged, responsibility for identifying, structuring and recording the learning experiences and learning outcomes is charged to the student. This poster describes a framework ‘The Peer Leader Capacity Building Model’ that purposefully structures the peer-leader’s learning journey providing: timely training, moments of critical reflection and goal setting. The model articulates the fundamental interplay of learning and peer leader service which forms the peer ‘learnership’. The journey begins with the ‘aspiration’ phase where students come to understand their leadership opportunities, progressing through ‘enabling’ and ‘mastering’ phases where students shape their learner-leader experience, and finally, to the ‘contributing graduate’ phase where students emerge as competent graduates able to confidently participate in their communities and workplaces. In shifting from a program centric approach that priorities the needs of the mentees, the Peer Leader Capacity Building Model focuses on the individual as a peer leader encouraging the student to shape their individual ‘learnscape’ through consciously navigating both their curricula and co-curricular learning experiences.

Informed learning in the undergraduate classroom: The role of information experiences in shaping outcomes

This research aimed to inform the design of effective information literacy lessons in higher education. Phenomenography, a research approach designed to study human experience, was used to explore the experiences of a teacher and undergraduate students using information to learn about language and gender issues. The findings show that the way learners use information influences content-focused learning outcomes, and reveal an instructional pattern for enabling students to use information while becoming aware of the topic they are investigating. Based on the findings, a design model is offered in which learning outcomes are realized through targeted information literacy activities.

The relationship between class participation and law students' learning, engagement and stress: Do demographics matter?

Student participation in the classroom has long been regarded as an important means of increasing student engagement and enhancing learning outcomes by promoting active learning. However, the approach to class participation common in U.S. law schools, commonly referred to as the Socratic method, has been criticised for its negative impacts on student wellbeing. A multiplicity of American studies have identified that participating in law class discussions can be alienating, intimidating and stressful for some law students, and may be especially so for women, and students from minority backgrounds. Using data from the Law School Student Assessment Survey (LSSAS), conducted at UNSW Law School in 2012, this Chapter provides preliminary insights into whether assessable class participation (ACP) at an Australian law school is similarly alienating and stressful for students, including the groups identified in the American literature. In addition, we compare the responses of undergraduate Bachelor of Laws (LLB) and graduate Juris Doctor (JD) students. The LSSAS findings indicate that most respondents recognise the potential learning and social benefits associated with class participation in legal education, but remain divided over their willingness to participate. Further, in alignment with general trends identified in American studies, LLB students, women, international students, and non-native English speakers perceive they contribute less frequently to class discussions than JD students, males, domestic students, and native English speakers, respectively. Importantly, the LSSAS indicates students are more likely to be anxious about contributing to class discussions if they are LLB students (compared to their JD counterparts), and if English is not their first language (compared to native English speakers). There were no significant differences in students’ self-reported anxiety levels based on gender, which diverges from the findings of American research.

LSP communication in a digital age: the outcomes of teaching and learning in tertiary education

This paper discusses the changes brought by the communication revolution in teaching and learning in the scope of LSP. Its aim is to provide an insight on how teaching which was bi-dimensional, turned into a multidimensional system, gathering other complementary resources that have transformed, in a incredibly short time, the ways we receive share and store information, for instance as professionals, and keep in touch with our peers. The increasing rise of electronic publications, the incredible boom of social and professional networks, search engines, blogs, list servs, forums, e-mail blasts, Facebook pages, YouTube contents, Tweets and Apps, have twisted the way information is conveyed. Classes ceased to be predictable and have been empowered by digital platforms, innumerous and different data repositories (TILDE, IATE, LINGUEE, and so many other terminological data banks) that have definitely transformed the academic world in general and tertiary education in particular. There is a bulk of information to be digested by students, who are no longer passive but instead responsible and active for their academic outcomes. The question is whether they possess the tools to select only what is accurate and important for a certain subject or assignment, due to that overflow? Due to the reduction of the number of course years in most degrees, after the implementation of Bologna and the shrinking of the curricula contents, have students the possibility of developing critical thinking? Both teaching and learning rely on digital resources to improve the speed of the spreading of knowledge. But have those changes been effective to promote really communication? Furthermore, with the increasing Apps that have already been developed and will continue to appear for learning foreign languages, for translation among others, will the students feel the need of learning them once they have those Apps. These are some the questions we would like to discuss in our paper.

Learning and Experiential Outcomes of Face-to-Face Versus Online Communications Courses

Higher education is rapidly trending toward the implementation of online (OL) courses and a blended facilitation style that incorporates both OL and face-to-face (FTF) classes. Though previous studies have explored the benefits and pitfalls of OL and blended learning formats from institutional, teacher, and student perspectives, scant research has examined learning outcomes for OL and FTF courses sharing identical content. This study used an explanatory mixed methods design—including pre- and post-test assessments, a questionnaire, and interviews—to explore similarities and differences in participant and teacher perceptions and outcomes (gain scores and final grades) of OL versus traditional FTF Communications courses, and to examine effects of students’ age and gender on learning preference and performance. Data collection occurred over a 4-month period and involved 183 student and 2 professor participants. The study used an SPSS program for data analysis and created a Microsoft Excel document to record themes derived from the questionnaire and interviews. Quantitative findings suggest there are no significant differences in gain scores, final grades, or other learning outcomes when comparing OL and FTF versions of identical Communications courses; however, qualitative findings indicate differences between facilitation styles based on student and professor perception. The study sheds light on student and faculty perceptions of facilitation styles and suggests areas for potential improvements in FTF- and OL-facilitated courses. The study ultimately recommends that students and faculty should have options when it comes to preferred delivery of course material.