Developement and evidencing achievement of graduate learning outcomes in Deakin University's enhanced mooc

There is now a plethora of Massive Open On-line courses (MOOCs) offered worldwide. Whilst many MOOCs focus on discipline-specific content, little attention has been paid to how MOOCs can explicitly help participants develop generic employability skills such as communication, digital literacy, global citizenship and the like. Similarly little attention been paid to explicitly assuring the quality of MOOCs with respect to alignment with regulatory body standards. Deakin University's first MOOC, DeakinPrimer, is an introduction to humanitarian responses to 21st century disasters. It has been designed to assist participants to explicitly evidence generic or employability skills, some of Deakin's eight Graduate Learning Outcomes (GLOs) including communication, digital literacy, critical thinking and global citizenship. Other key features of DeakinPrimer include opportunities for networking with fellow participants and experts within the humanitarian field, and the opportunity to apply for credit towards the Graduate Certificate in International Community Development (level 8 in the Australian Qualifications Framework [AQF]) and for those with a prior Bachelor degree, the Masters in Humanitarian Assistance or the Masters of International Community Development (level 9 in the AQF). DeakinPrimer is designed as a test bed for a learning innovation, particularly micro-credentialing GLOs using digital badges to enable self and peer endorsement of evidence of learning. Badging is integrated in two ways. Firstly, DeakinPrimer participants build portfolios of learning artefacts associated with learning activities, then assess their work against a set of holistic, generic learning outcomes standards rubrics. If they judge their evidence as meeting the required standard, they can claim a badge (self endorsement) associated with particular GLOs. Secondly, participants can request and provide peer feedback and endorsement (using peer badges). The integration of self and peer review in the assessment tasks helps participants develop important employability skills, the ability to critically self-reflect on their own work and critically analyse the work of others and provide evidence-based feedback. DeakinPrimer is scheduled to commence in July 2013. This paper explains the way in which the course curricula has been designed to use technologies to enable participants to curate evidence of learning, and self and peer endorse such learning against defined standards.

Towards a framework for aligning learning outcomes, academic literacies and assessment criteria

Although Australian students spend three or more years studying they can seem quite unaware of any of the expected learning outcomes of their course. They are often single unit focused, paying most attention to individual assessment items thus not developing a holistic view of their course. This paper presents a theoretical framework to support staff and students to recognise, scaffold and achieve learning outcomes and academic skills at unit level and to recognise how these contribute to course and graduate learning outcomes, within the boundaries of Australian university and professional accreditation requirements. A case study is described that demonstrates the manual implementation of the framework. The complex nature of the implementation suggests that a software solution is required to ease the process and ensure the resulting mapping will have some longevity by being maintainable.

Investigating and Understanding Student Learning Outcomes in an Online and Face-to-Face Graduate Level Legal Administration Course: An Embedded Mixed Methods Design

Online education is a new teaching and learning medium with few current guidelines for faculty, administrators or students. Its rapid growth over the last decade has challenged academic institutions to keep up with the demand, while also providing a quality education. Our understanding of the factors that determine quality and effective online learning experiences that lead to student learning outcomes is still evolving. There is a lack of consensus on the effectiveness of online versus face-to-face education in the current research. The U.S. Department of Education conducted a meta-analysis in 2009 and concluded that student-learning outcomes in online courses were equal to and, often times, better than face-to-face traditional courses. Subsequent research has found contradictory findings, and further inquiry is necessary. The purpose of this embedded mixed methods design research study is to further our understanding of the factors that create quality and successful educational outcomes in an online course. To achieve this, the first phase of this study measured and compared learning outcomes in an online and in class graduate-level legal administration course. The second phase of the study entailed interviews with those students in both the online and face-to-face sections to understand their perspectives on the factors contributing to learning outcomes. Six themes emerged from the qualitative findings: convenience, higher order thinking, discussions, professor engagement, professor and student interaction, and face-to-face interaction. Findings from this study indicate the factors students perceive as contributing to learning outcomes in an online course are consistent among all students and are supported in the existing literature. Higher order thinking, however, emerged as a stronger theme than indicated in the current research, and the face-to-face nature of the traditional classroom may be more an issue of familiarity than a factor contributing to learning outcomes. As education continues to reach new heights and developments in technology advance, the factors found to contribute to student learning outcomes will be refined and enhanced. These developments will continue to transform the ways in which we deliver and receive knowledge in both traditional and online classrooms. While there is a growing body of research on online education, the field’s evolution has unsettled earlier findings and posed new areas to investigate.

The impact of learning outcomes in business education: assessing value, relevance and graduate ability in a multi-country study of employers and business graduates

This paper discusses critical findings from a two-year EU-funded research project involving four European countries: Austria, England, Slovenia and Romania. The project had two primary aims. The first of these was to develop a systematic procedure for assessing the balance between learning outcomes acquired in education and the specific needs of the labour market. The second aim was to develop and test a set of meta-level quality indicators aimed at evaluating the linkages between education and employment. The project was distinctive in that it combined different partners from Higher Education, Vocational Training, Industry and Quality Assurance. One of the key emergent themes identified in exploratory interviews was that employers and recent business graduates in all four countries want a well-rounded education which delivers a broad foundation of key business knowledge across the various disciplines. Both groups also identified the need for personal development in critical skills and competencies. Following the exploratory study, a questionnaire was designed to address five functional business areas, as well as a cluster of 8 business competencies. Within the survey, questions relating to the meta-level quality indicators assessed the impact of these learning outcomes on the workplace, in terms of the following: 1) value, 2) relevance and 3) graduate ability. This paper provides an overview of the study findings from a sample of 900 business graduates and employers. Two theoretical models are proposed as tools for predicting satisfaction with work performance and satisfaction with business education. The implications of the study findings for education, employment and European public policy are discussed.

Standards that we can 'own' : Threshold Learning Outcomes for Journalism, Public Relations, and Media & Communication Studies

This paper outlines the progress by the JoMeC (Journalism, Media & Communication) Network in developing TLO (Threshold Learning Outcome) statements for Bachelor-level university programs in the disciplines of Journalism, Public Relations and Media & Communications Studies. The paper presents the finalised TLO statement for Journalism, and outlines moves to engage discipline-based groups to further develop preliminary TLOs for Public Relations and Media & Communication Studies. The JoMeC Network was formed in 2011, in response to requirements that from 2014 all degrees and qualifications at Australian universities would be able to demonstrate that they comply with the threshold learning standards set by the Australian Qualifications Framework (AQF). The AQF’s threshold standards define the minimum types and levels of knowledge, skills and capabilities that a student must demonstrate in order to graduate. The Tertiary Education Quality and Standards Agency (TEQSA) will use the AQF’s threshold standards as a key tool in recording and assessing the performance of higher educational institutions, and determining whether they should be registered as Australian Higher Education Providers under the Higher Education Standards Framework. The Office of Learning & Teaching (OLT) places the onus on discipline communities to collaborate in order to develop and ‘own’ the threshold learning standards that can be considered the minimum learning outcomes of university-level programs in that field. With the support of an OLT Grant, the JoMeC Network’s prime goal has been to develop three sets of discipline-specific TLOs – one each for the Journalism, Public Relations, and Media & Communications Studies disciplines. This paper describes the processes of research, consultation, drafting and ongoing revision of the TLO for Journalism. It outlines the processes that the JoMeC Network has taken in developing a preliminary TLO draft to initiate discussion of Public Relations and Media & Communication Studies. The JoMeC Network plans to hand management of further development of these TLOs to scholars within the discipline who will engage with academics and other stakeholders to develop statements that the respective disciplines can embrace and ‘own’.

Desired learning outcomes for the curricula in financial services industry as perceived by professionals in the financial services industry located in the Greater Toronto Area /

This study sought to explore the changing nature of the financial services industry in Toronto, Canada and the impact that these changes will have on the vocational educational outcomes required by Ontario Colleges of Applied Arts and Technology (CAAT) graduates who wish to enter the financial services industry. The study was descriptive and exploratory, based on both quantitative and qualitative data. Triangulation of 3 data sources (a collection of newspaper articles from the Toronto Star between July 1999 and June 2000, the calendars of the 25 CAATs, and a survey questionnaire prepared by me and distributed to subject matter experts who are key practitioners in the financial services industry) was used. The study contains a discussion of how the financial services industry is changing. The first question to be answered was: What do current practitioners in financial services perceive to be the knowledge, skills, and attitudes that will be required of future graduates for employment within the financial services industry? The study found that Ontario CAAT's graduates entering the financial services field need both business and financial services vocational learning outcomes. Colleges should have 2 programs 1 in accounting and 1 in financial services. The report addresses which specific topics should be included in the financial services program. The second question to be answered was: How does this anticipated profile of knowledge, skills, and attitudes change depending on the degree of implementation of the new technologies by the survey respondent? The study found no pattern. The third question to be answered was: In what way do existing programs need to change in the area of accreditation as perceived by the respondents? The study found that for accreditation, 3 credentials should be addressed within the financial services program. These are the Canadian Securities, the Life Underwriters, and the Certified Financial Planner designations. The last question to be answered was: What new knowledge, skills, and attitudes need to be incorporated into college curricula to address changing needs in the employment sector? For each Ontario CAAT which has a financial services program (excluding accounting), their program was reviewed in light of the topics as perceived by professionals in the financial services industry.

Fit for intended use : a manufacturing metaphor applied to international business students and learning outcomes

It is no secret that contemporary tertiary education in Australia is significantly reliant on international student fee income in a competitive market. Accordingly, the need to attract fee paying students involves strategies for increasing competitive advantage, new course structures, flexible learning initiatives and marketing. However Jackling (1) has found that employers are reluctant to employ graduate international students in the accounting field as they consider them to lack the skills required to effectively meet employment needs. This paper seeks to focus the spotlight on the role of academics/universities in ensuring that graduates have the skills necessary for employment as part of the education process.

Doing it again, thoughtfully : using feedback on draft reports to improve learning outcomes

A lack of communication skills is one of the graduate skill deficiencies most commonly cited by employers. Although the writing of laboratory and other scientific reports is a common teaching-and-learning task in university chemistry curricula, this skill is not fully developed in most Australian undergraduate science degrees. Usually, there is no opportunity for practice and feedback before the reports are used for summative assessment. This paper describes a system whereby students are given the opportunity to submit draft reports. After feedback, they can “do it again, thoughtfully”: only the final report is assessed. Student evaluations of the effectiveness of this approach are reported.

Good practice guidelines for curriculum, supervision and assessment of final year engineering projects and AQF8 learning outcomes

Undergraduate engineering programs require final year students to complete capstone final year projects and demonstrate that they can integrate knowledge, skills and professional graduate attributes developed during the program at Australian Qualification Framework, level 8 (AQF8) outcomes. Literature shows that currently there is no guarantee of consistency for curriculum, supervision and assessment practices of FYEPs. Practices differ greatly between universities and littlework has been initiated that seeks to identify good practice, highlighting the need for the development of guidelines for curriculum, supervision and assessment of FYEPs. This workshop is designed to share and disseminate the good practice guidelines that have been developed on curriculum, supervision and assessment of Final Year Engineering Projects as a part of phase 2 of the project ‘Assessing Final Year Engineering Projects (FYEPs): Ensuring Learning and Teaching Standards and AQF8 Outcomes’ funded by the Australian Office for Learning and Teaching (OLT) with people working in the area of FYEPs. The guidelines typically apply to four year undergraduate engineering degrees with embedded Honours and support achievement of AQF8learning outcomes. The project team has 7 partner Universities – Central Queensland University (the lead), University of Technology Sydney, University of Adelaide, Curtin University, Deakin University, University of Tasmania and RMIT University.Participants will be invited to reflect on and evaluate guidelines and findings derived from FYEP coordinators, supervisors and the wider literature and to consider the ways in which these findings might lead to improvements in their practice.

Assessing the structure of learning outcomes from counselling using the SOLO taxonomy: an exploratory study

Thirty-five clients who had received counselling completed a letter to a friend describing in as much detail as possible what they had learned from counselling. The participants' written responses were analysed and classified using the Structure of Learning Outcomes (SOLO) taxonomy. The results suggested that an expanded SOLO offers a promising and exciting way to view the outcomes of counselling within a learning framework. If the SOLO taxonomy is found to be stable in subsequent research, and clients are easily able to be classified using the taxonomy, then this approach may have implications for the process of counselling. To maximise the learning outcomes, counsellors could use strategies and techniques to enhance their clients' learning.

Implementation of an integrated, technology-based, discovery mode assessment item involving an incubation period to enhance learning outcomes for engineering maths students

In this paper we discuss our current efforts to develop and implement an exploratory, discovery mode assessment item into the total learning and assessment profile for a target group of about 100 second level engineering mathematics students. The assessment item under development is composed of 2 parts, namely, a set of "pre-lab" homework problems (which focus on relevant prior mathematical knowledge, concepts and skills), and complementary computing laboratory exercises which are undertaken within a fixed (1 hour) time frame. In particular, the computing exercises exploit the algebraic manipulation and visualisation capabilities of the symbolic algebra package MAPLE, with the aim of promoting understanding of certain mathematical concepts and skills via visual and intuitive reasoning, rather than a formal or rigorous approach. The assessment task we are developing is aimed at providing students with a significant learning experience, in addition to providing feedback on their individual knowledge and skills. To this end, a noteworthy feature of the scheme is that marks awarded for the laboratory work are primarily based on the extent to which reflective, critical thinking is demonstrated, rather than the amount of CBE-style tasks completed by the student within the allowed time. With regard to student learning outcomes, a novel and potentially critical feature of our scheme is that the assessment task is designed to be intimately linked to the overall course content, in that it aims to introduce important concepts and skills (via individual student exploration) which will be revisited somewhat later in the pedagogically more restrictive formal lecture component of the course (typically a large group plenary format). Furthermore, the time delay involved, or "incubation period", is also a deliberate design feature: it is intended to allow students the opportunity to undergo potentially important internal re-adjustments in their understanding, before being exposed to lectures on related course content which are invariably delivered in a more condensed, formal and mathematically rigorous manner. In our presentation, we will discuss in more detail our motivation and rationale for trailing such a scheme for the targeted student group. Some of the advantages and disadvantages of our approach (as we perceived them at the initial stages) will also be enumerated. In a companion paper, the theoretical framework for our approach will be more fully elaborated, and measures of student learning outcomes (as obtained from eg. student provided feedback) will be discussed.

Student perceptions of the impact of online discussion forum participation on learning outcomes

Choosing the “right” type of technology, either synchronous or asynchronous, to facilitate learning outcomes is a new challenge as the pace of emerging technologies increases and diversifies. Teachers are encouraged to design courses that require collaboration in online learning communities to facilitate the development of higher order thinking skills for life long learning. It is therefore important to gather evidence of the kinds of opportunities afforded to students and whether the students themselves endorse collaborative online tools as a legitimate method of assisting in their learning. This paper outlines the way in which one secondary school has used an online discussion forum to support students in the International Baccalaureate (IB) Diploma Program in enhancing research skills and skills for lifelong learning. The paper considers whether online collaborative tools are perceived by students as a positive way to improve learning outcomes.