Learning analytics from distance learning

Presentation at M25 Learning Technology Group, FutureLearn, 15 November 2017

How should we measure online learning activity?

The proliferation of Web-based learning objects makes finding and evaluating resources a considerable hurdle for learners to overcome. While established learning analytics methods provide feedback that can aid learner evaluation of learning resources, the adequacy and reliability of these methods is questioned. Because engagement with online learning is different from other Web activity, it is important to establish pedagogically relevant measures that can aid the development of distinct, automated analysis systems. Content analysis is often used to examine online discussion in educational settings, but these instruments are rarely compared with each other which leads to uncertainty regarding their validity and reliability. In this study, participation in Massive Open Online Course (MOOC) comment forums was evaluated using four different analytical approaches: the Digital Artefacts for Learning Engagement (DiAL-e) framework, Bloom's Taxonomy, Structure of Observed Learning Outcomes (SOLO) and Community of Inquiry (CoI). Results from this study indicate that different approaches to measuring cognitive activity are closely correlated and are distinct from typical interaction measures. This suggests that computational approaches to pedagogical analysis may provide useful insights into learning processes.

The educational problem that MOOCs could solve: professional development for teachers of disadvantaged students

The demographics of massive open online course (MOOC) analytics show that the great majority of learners are highly qualified professionals, and not, as originally envisaged, the global community of disadvantaged learners who have no access to good higher education. MOOC pedagogy fits well with the combination of instruction and peer community learning found in most professional development. A UNESCO study therefore set out to test the efficacy of an experimental course for teachers who need but do not receive high-quality continuing professional development, as a way of exploiting what MOOCs can do indirectly to serve disadvantaged students. The course was based on case studies around the world of information and communication technology (ICT) in primary education and was carried out to contribute to the UNESCO “Education For All” goal. It used a co-learning approach to engage the primary teaching community in exploring ways of using ICT in primary education. Course analytics, forums and participant surveys demonstrated that it worked well. The paper concludes by arguing that this technology has the power to tackle the large-scale educational problem of developing the primary-level teachers needed to meet the goal of universal education.