How should we measure online learning activity?

The proliferation of Web-based learning objects makes finding and evaluating resources a considerable hurdle for learners to overcome. While established learning analytics methods provide feedback that can aid learner evaluation of learning resources, the adequacy and reliability of these methods is questioned. Because engagement with online learning is different from other Web activity, it is important to establish pedagogically relevant measures that can aid the development of distinct, automated analysis systems. Content analysis is often used to examine online discussion in educational settings, but these instruments are rarely compared with each other which leads to uncertainty regarding their validity and reliability. In this study, participation in Massive Open Online Course (MOOC) comment forums was evaluated using four different analytical approaches: the Digital Artefacts for Learning Engagement (DiAL-e) framework, Bloom's Taxonomy, Structure of Observed Learning Outcomes (SOLO) and Community of Inquiry (CoI). Results from this study indicate that different approaches to measuring cognitive activity are closely correlated and are distinct from typical interaction measures. This suggests that computational approaches to pedagogical analysis may provide useful insights into learning processes.

Fostering postgraduate student engagement: online resources supporting self-directed learning in a diverse cohort

The research question for this study was: ‘Can the provision of online resources help to engage and motivate students to become self-directed learners?’ This study presents the results of an action research project to answer this question for a postgraduate module at a research-intensive university in the United Kingdom. The analysis of results from the study was conducted dividing the students according to their programme degree – Masters or PhD – and according to their language skills. The study indicated that the online resources embedded in the module were consistently used, and that the measures put in place to support self-directed learning (SDL) were both perceived and valued by the students, irrespective of their programme or native language. Nevertheless, a difference was observed in how students viewed SDL: doctoral students seemed to prefer the approach and were more receptive to it than students pursuing their Masters degree. Some students reported that the SDL activity helped them to achieve more independence than did traditional approaches to teaching. Students who engaged with the online resources were rewarded with higher marks and claimed that they were all the more motivated within the module. Despite the different learning experiences of the diverse cohort, the study found that the blended nature of the course and its resources in support of SDL created a learning environment which positively affected student learning.

Best-practice model for technology enhanced learning in the creative arts

This paper presents a best-practice model for the redesign of virtual learning environments (VLEs) within creative arts to augment blended learning. In considering a blended learning best-practice model, three factors should be considered: the conscious and active human intervention, good learning design and pedagogical input, and the sensitive handling of the process by trained professionals. This study is based on a comprehensive VLE content analysis conducted across two academic schools within the creative arts at one Post-92 higher education (HE) institution. It was found that four main barriers affect the use of the VLE within creative arts: lack of flexibility in relation to navigation and interface, time in developing resources, competency level of tutors (confidence in developing online resources balanced against other flexible open resources) and factors affecting the engagement of ‘digital residents’. The experimental approach adopted in this study involved a partnership between the learning technology advisor and academic staff, which resulted in a VLE best-practice model that focused directly on improving aesthetics and navigation. The approach adopted in this study allowed a purposive sample of academic staff to engage as participants, stepping back cognitively from their routine practices in relation to their use of the VLE and questioning approaches to how they embed the VLE to support teaching and learning. The model presented in this paper identified a potential solution to overcome the challenges of integrating the VLE within creative arts. The findings of this study demonstrate positive impact on staff and student experience and provide a sustainable model of good practice for the redesign of the VLE within creative disciplines.

Shaping the future of learning using the student voice: we’re listening but are we hearing clearly?

Student voice data is a key factor as Manchester Metropolitan University strives to continually improve institutional technology enhanced learning (TEL) infrastructure. A bi-annual Institutional Student Survey enables students to communicate their experience of learning, teaching and assessment on programmes and specific units studied. Each cycle of the survey contains approximately 40–50,000 free text comments from students pertaining to what they appreciate and what they would like to see improved. A detailed thematic analysis of this data has identified 18 themes, arranged into six categories relating to the ‘Best’ aspects of courses, and 25 themes, arranged in seven categories in relation to aspects of courses considered to be ‘in need of improvement’. This student data was then used as a basis for semi-structured interviews with staff. Anecdotally, evidence suggested that student expectations and staff expectations around TEL and the virtual learning environment (VLE) differed. On-going evaluation of this work has highlighted a disconnect. In significant instances, academic colleagues seemingly misinterpret the student voice analysis and consequently struggle to respond effectively. In response to the analysis, the learning technologist's role has been to re-interpret the analysis and redevelop TEL staff development and training activities. The changes implemented have focused on: contextualising resources in VLE; making lectures more interactive; enriching the curriculum with audio–visual resources; and setting expectations around communications.

University students and faculty have positive perceptions of open/alternative resources and their utilization in a textbook replacement initiative

This is contribution no. 16-114-J from the Kansas Agricultural Experiment Station. The Kansas State University Open/Alternative Textbook Initiative provides grants to faculty members to replace textbooks with open/alternative educational resources (OAERs) that are available at no cost to students. Open educational resources are available for anyone to access, while alternative educational resources are not open. The objective of this study was to determine the perceptions towards OAERs and the initiative, of students enrolled in, and faculty members teaching, courses using OAERs. A survey was sent out to 2,074 students in 13 courses using the OAERs. A total of 524 (25.3%) students completed the survey and a faculty member from each of the 13 courses using OAERs was interviewed. Students rated the OAERs as good quality, preferred using them instead of buying textbooks for their courses, and agreed that they would like OAERs used in other courses. Faculty felt that student learning was somewhat better and it was somewhat easier to teach using OAERs than when they used the traditional textbooks. Nearly all faculty members preferred teaching with OAERs and planned to continue to do so after the funding period. These results, combined with the tremendous savings to students, support the continued funding of the initiative and similar approaches at other institutions.

Assuring best practice in technology-enhanced learning environments

This paper documents the development and findings of the Good Practice Report on Technology-Enhanced Learning and Teaching funded by the Australian Learning and Teaching Council (ALTC). Developing the Good Practice Report required a meta-analysis of 33 ALTC learning and teaching projects relating to technology funded between 2006 and 2010. This report forms one of 12 completed Good Practice Reports on a range of different topics commissioned by the ALTC and Australian Government Office for Learning and Teaching (OLT). The reports aim to reduce issues relating to dissemination that projects face within the sector by providing educators with an efficient and accessible way of engaging with and filtering through the resources and experiences of numerous learning and teaching projects funded by the ALTC and OLT. The Technology-Enhanced Learning and Teaching Report highlights examples of good practice and provides outcomes and recommendations based on the meta-analysis of the relevant learning and teaching projects. However, in order to ensure the value of these reports is realised, educators need to engage with the reports and integrate the information and findings into their practice. The paper concludes by detailing how educational networks can be utilised to support dissemination.