Short answer versus multiple choice examination questions for first year chemistry

Multiple choice (MC) examinations are frequently used for the summative assessment of large classes because of their ease of marking and their perceived objectivity. However, traditional MC formats usually lead to a surface approach to learning, and do not allow students to demonstrate the depth of their knowledge or understanding. For these reasons, we have trialled the incorporation of short answer (SA) questions into the final examination of two first year chemistry units, alongside MC questions. Students’ overall marks were expected to improve, because they were able to obtain partial marks for the SA questions. Although large differences in some individual students’ performance in the two sections of their examinations were observed, most students received a similar percentage mark for their MC as for their SA sections and the overall mean scores were unchanged. In-depth analysis of all responses to a specific question, which was used previously as a MC question and in a subsequent semester in SA format, indicates that the SA format can have weaknesses due to marking inconsistencies that are absent for MC questions. However, inclusion of SA questions improved student scores on the MC section in one examination, indicating that their inclusion may lead to different study habits and deeper learning. We conclude that questions asked in SA format must be carefully chosen in order to optimise the use of marking resources, both financial and human, and questions asked in MC format should be very carefully checked by people trained in writing MC questions. These results, in conjunction with an analysis of the different examination formats used in first year chemistry units, have shaped a recommendation on how to reliably and cost-effectively assess first year chemistry, while encouraging higher order learning outcomes.

She who can, teaches

The following are real statements from real women taken from two surveys (2001, 2008). While they were all pre-service teachers training to teach specialist secondary IT or computer science subjects, some were straight from school while others were mature-age or career-change students. Some were questioning whether they should follow careers in the IT industry or should they continue into IT teaching. This is a poetic transcription of their voices.

Promoting student well-being and resilience at law school

University can be a psychologically distressing place for students, particularly those studying law. Legal academics have been concerned about this for some time. In the United States, in particular, it has been found that symptoms of psychological distress rise signifi cantly for students in their fi rst year of law (compared to levels in the general population at that time), and persist throughout the degree to post-graduation. Recognised symptoms include depression, obsessive compulsive behaviour, feelings of inadequacy and inferiority, anxiety, hostility, paranoia, and social alienation. Many students experience law school as an isolating, adversarial and competitive environment, which impacts negatively on their values and motivation...

Future directions in course quality assurance

The Course Quality Assurance System at Queensland University of Technology (QUT) has as its centrepiece an exemplar of data visualisation known as the Individual Course Report (ICR). This report provides every course coordinator with an annual snapshot of their performance data evaluated against QUT and national benchmarks. In this article, the impact of the ICR is explored through the case study of one undergraduate course identified as underperforming. The case study features an innovative, ethnographic approach to working with course teams and highlights the importance of context, collaboration and appropriate support in creating evidence-based action plans for course improvement.

Employment opportunities in Australia for educational researchers: A review of recent advertisements

The Australian Association for Research in Education (AARE) by promoting the conference theme has identified a need to be more proactive to ensure growth in the number of educational researchers. Within the Higher Education sector there are a number of methods used to encourage interest in a particular area, and these include policy, funding, sponsorship, employment and scholarships. There are three types of employment for academics: Research, Lecturing and Teaching and Learning and two types of scholarships: either students self-identify the topic or topics are targeted with associated funding. The aim of this study is to review the academic positions and targeted scholarships of Australian Universities and research organisations gathered from advertisements in a national newspaper. This will establish a baseline of recent practice from July to December, 2006 and identify opportunities for researchers in all Disciplines and specifically in education. Results reveal the two main groups for academics are Research and Lecturing, with a small number in Teaching and Learning. Although the Education Discipline is well represented overall (3rd in 12 Disciplines after Health and Science) in terms of research opportunities education then moves to 10th position. A further significant finding is the highly contractual nature of research versus the more stable, tenured environment for lecturing. There are a number of implications arising from this short study. Firstly, the Discipline of Education as a targeted area for research alone is significantly under-represented in the advertised positions but is well represented in lecturing where the role always requires teaching and research. Thus it seems the amount of time devoted to research by academics in the education Discipline is significantly lower than for health or science. Secondly, there are few industry/Government targeted scholarships in the education Discipline therefore any growth in numbers of educational researchers through postgraduate study is not expanded by funding to meet identified needs. In conclusion AARE, an association interested in promoting the growth of educational research, has an obvious need to encourage and review the outcomes of this study and perhaps adopt some of the successful strategies employed by other Disciplines to improve the opportunities for educational researchers in the future.

Finding your research voice : the application of bibliometrics as an analytic literature tool on the pedagogic field of higher education

This paper documents the use of bibliometrics as a methodology to bring forth a structured, systematic and rigorous way to analyse and evaluate a range of literature. When starting out and reading broadly for my doctoral studies, one article by Trigwell and Prosser (1996b) led me to reflect about my level of comprehension as the content, concepts and methodology did not resonate with my epistemology. A disconnection between our paradigms emerged. Further reading unveiled the work by Doyle (1987) who categorised research in teaching and teacher education by three main areas: teacher characteristics, methods research and teacher behaviour. My growing concerns that there were gaps in the knowledge also exposed the difficulties in documenting said gaps. As an early researcher who required support to locate myself in the field and to find my research voice, I identified bibliometrics (Budd, 1988; Yeoh & Kaur, 2007) as an appropriate methodology to add value and rigour in three ways. Firstly, the application of bibliometrics to analyse articles is systematic, builds a picture from the characteristics of the literature, and offers a way to elicit themes within the categories. Secondly, by systematic analysis there is occasion to identify gaps within the body of work, limitations in methodology or areas in need of further research. Finally, extension and adaptation of the bibliometrics methodology, beyond citation or content analysis, to investigate the merit of methodology, participants and instruments as a determinant for research worth allowed the researcher to build confidence and contribute new knowledge to the field. Therefore, this paper frames research in the pedagogic field of Higher Education through teacher characteristics, methods research and teacher behaviour, visually represents the literature analysis and locates my research self within methods research. Through my research voice I will present the bibliometrics methodology, the outcomes and document the landscape of pedagogy in the field of Higher Education.

Evaluation of the quality of diverse teaching practice in higher education research methods

The purpose of this paper is to report on a methods research project investigating the evaluation of diverse teaching practice in Higher Education. The research method is a single site case study of an Australian university with data collected through published documents, surveys, interviews and focus groups. This project provides evidence of the wide variety of evaluation practice and diverse teaching practice across the university. This breadth identifies the need for greater flexibility of evaluation processes, tools and support to assist teaching staff to evaluate their diverse teaching practice. The employment opportunities for academics benchmark the university nationally and position the case study in the field. Finally this reaffirms the institutional responsibility for services to support teaching staff in an ongoing manner.

Reframing evaluation of learning and teaching: An approach to change

Evaluation practices in the Higher Education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology (QUT) began a project which aims to reframe the organisation’s approach to the evaluation of learning and teaching. This paper describes the existing evaluation system; the emergence and early development of the project; and formulation of a conceptual framework identifying key dimensions of evaluation. It then compares the draft framework with other conceptualisations and models of evaluation identified in the literature, to determine its validity and suitability for supporting QUT’s plans for the future. Overall, the paper represents a structured evaluation of the REFRAME project at a particular point in its lifecycle. Given that the project follows an evidence based, practice-led process and applies an ongoing action research cycle, the findings are presented in the belief that QUT’s experience is broadly applicable to other institutions which may be contemplating change in relation to evaluation of learning and teaching.

Reflection in Integrated Problem Based Learning : 20 years of continuing application to the teaching of architecture

In 1984 the School of Architecture and Built Environment within the University of Newcastle, Australia introduced an integrated program based on real design projects and using Integrated Problem Based Learning (IPBL) as the teaching method. Since 1984 there have been multiple changes arising from the expectations of the architectural fraternity, enrolling students, lecturers, available facilities, accreditation authorities and many others. These challenges have been successfully accommodated whilst maintaining the original purposes and principles of IPBL. The Architecture program has a combined two-degree structure consisting of a first degree, Bachelor of Science (Architecture), followed by a second degree, Bachelor of Architecture. The program is designed to simulate the problem-solving situations that face a working architect in every day practice. This paper will present the degree structure where each student is enrolled in a single course per semester incorporating design integration and study areas in design studies, professional studies, historical studies, technical studies, environmental studies and communication skills. Each year the design problems increase in complexity and duration set around an annual theme. With 20 years of successful delivery of any program there are highlights and challenges along the way and this paper will discuss some of the successes and barriers experienced within the School of Architecture and Built Environment in delivering IPBL. In addition, the reflective process investigates the currency of IPBL as an appropriate vehicle for delivering the curriculum in 2004 and any additional administrative or staff considerations required to enhance the continuing application of IPBL.

Articulation in property programs : an international multi-campus tertiary model

This paper will present program developers and institutional administrators with a program delivery model suitable for cross cultural international delivery developing students from industry through to master’s level tertiary qualifications. The model was designed to meet the needs of property professionals from an industry where technical qualifications are the norm and tertiary qualifications are emerging. A further need was to develop and deliver a program that enhanced the University’s current program profile in both the domestic and international arenas. Early identification of international educational partners, industry need and the ability to service the program were vital to the successful development of Master of Property program. The educational foundations of the program rest in educational partners, local tutorial support, international course management, cultural awareness of and in content, online communication fora, with a delivery focus on problem-based learning, self-directed study, teamwork and the development of a global understanding and awareness of the international property markets. In enrolling students from a diverse cultural background with technical qualifications and/or extensive work experience there are a number of educational barriers to be overcome for all students to successfully progress and complete the program. These barriers disappear when the following mechanisms are employed: individual student pathways, tutorial support by qualified peers, enculturation into tertiary practice, assessment tasks that recognise cultural norms and values, and finally that value is placed on the experiential knowledge, cultural practices and belief systems of the students.

2005 pedagogical orientation program in the School of Architecture and Built Environment : a case study

The University of Newcastle (UoN) offers various access and support programs for a range of students through the English Language and Foundation Studies Centre and a University orientation for students. At UoN, students are required to engage in a learning experience, meet program outcomes and demonstrate the core attributes of the University at each graduation point. For a University with a strong focus on access is there a missing facet to the access programs where students are required to study within a teaching delivery style which may be vastly different to their previous educational experience? This paper will describe a pedagogical orientation program currently delivered at UoN School of Architecture and Built Environment in 2005 to assist in the transition of students from different cultural and pedagogical backgrounds into “Problem Based Learning” as delivered by this School. Furthermore the paper will analyse how this program has enabled students from diverse backgrounds to understand and successfully embrace the new learning opportunities.

Collaboration and competition on a Wiki : the praxis of online social learning to improve academic writing and research in under-graduate students

The number of internet users in Australia has been steadily increasing, with over 10.9 million people currently subscribed to an internet provider (ABS, 2011). Over the past year, the most avid users of the Internet were 15 – 24 year olds, with approximately 95% accessing the internet on a regular basis (ABS, Social Trends, 2011). While the internet, in particularly Web 2.0, has been described as fundamental to higher education students, social and leisure internet tools are also increasingly being used by these students to generate and maintain their social and professional networks and interactions (Duffy & Bruns, 2006). Rapid technological advancements have enabled greater and faster access to information for learning and education (Hemmi et al, 2009; Glassman & Kang, 2011). As such, we sought to integrate interactive, online social media into the assessment profile of a Public Health undergraduate cohort at the Queensland University of Technology (QUT). The aim of this exercise was to engage undergraduate students to both develop and showcase their research on a range of complex, contemporary health issues within the online forum of Wikispaces for review and critique by their peers. We applied Bandura’s Social Learning Theory (SLT) to analyse the interactive processes from which students developed deeper and more sustained learning, and via which their overall academic writing standards were enriched. This paper outlines the assessment task, and the students’ feedback on their learning outcomes in relation to the Attentional, Retentional, Motor Reproduction, and Motivational Processes outlined by Bandura in SLT. We conceptualise the findings in a theoretical model, and discuss the implications for this approach within the broader tertiary environment.

The pedagogical balancing act : teaching reflection in higher education

Despite the common use of the term reflection in higher education assessment tasks, learners are not often taught how to communicate their disciplinary knowledge through reflection. This paper argues that students can and should be taught how to reflect in deep and transformative ways. It highlights the reflexive pedagogical balancing act of attending to different levels of reflection as a way to stimulate focused, thoughtful and reasoned reflections that show evidence of new ways of thinking and doing. The paper uses data from a current project to illustrate the effects of focusing on particular levels of reflection in the pedagogical strategies used, and argues that while the goal of academic or professional reflection is generally to move students to the highest level of reflection to transform their learning/practice, unless higher education teachers attend to every level of reflection, there are specific, observable gaps in the reflections that students produce.

Implementing academagogy : the first case study

As proposed in McAuliffe, Winter, Chadwick and Hargreaves (2008), academagogy could be used as an “umbrella” term allowing the teacher to select from a range of teaching approaches – pedagogy (teacher as source of all information to student without power or knowledge), andragogy (teacher as source of information for adult student), or heutagogy (student with knowledge seeking information from teacher to fill the gaps in their own knowledge). During Semester 1, 2009, one of the authors of this paper decided to treat his third-year students in a more heutagogical manner by allowing them to experience ownership of their own learning. This article is a case study of that experience which reveals that, although initially more time-consuming, academagogy can result in better student outcomes.