A case study of the effectiveness of the electronic survey as a research tool for measuring students' experience of e-learning

This paper uses a case study approach to consider the effectiveness of the electronic survey as a research tool to measure the learner voice about experiences of e-learning in a particular institutional case. Two large scale electronic surveys were carried out for the Student Experience of e-Learning (SEEL) project at the University of Greenwich in 2007 and 2008, funded by the UK Higher Education Academy (HEA). The paper considers this case to argue that, although the electronic web-based survey is a convenient method of quantitative and qualitative data collection, enabling higher education institutions swiftly to capture multiple views of large numbers of students regarding experiences of e-learning, for more robust analysis, electronic survey research is best combined with other methods of in-depth qualitative data collection. The advantages and disadvantages of the electronic survey as a research method to capture student experiences of e-learning are the focus of analysis in this short paper, which reports an overview of large-scale data collection (1,000+ responses) from two electronic surveys administered to students using surveymonkey as a web-based survey tool as part of the SEEL research project. Advantages of web-based electronic survey design include flexibility, ease of design, high degree of designer control, convenience, low costs, data security, ease of access and guarantee of confidentiality combined with researcher ability to identify users through email addresses. Disadvantages of electronic survey design include the self-selecting nature of web-enabled respondent participation, which tends to skew data collection towards students who respond effectively to email invitations. The relative inadequacy of electronic surveys to capture in-depth qualitative views of students is discussed with regard to prior recommendations from the JISC-funded Learners' Experiences of e-Learning (LEX) project, in consideration of the results from SEEL in-depth interviews with students. The paper considers the literature on web-based and email electronic survey design, summing up the relative advantages and disadvantages of electronic surveys as a tool for student experience of e-learning research. The paper concludes with a range of recommendations for designing future electronic surveys to capture the learner voice on e-learning, contributing to evidence-based learning technology research development in higher education.

The eLIDA CAMEL nomadic model of collaborative partnership for a community of practice in design for learning

A nomadic collaborative partnership model for a community of practice (CoP) in Design for Learning (D4L) can facilitate successful innovation and continuing appraisals of effective professional practice, stimulated by a 'critical friend' assigned to the project. This paper reports on e-learning case studies collected by the JISC-funded UK eLIDA CAMEL Design for Learning Project. The project implemented and evaluated learning design (LD) tools in higher and further education within the JISC Design for Learning pedagogic programme (2006-07). Project partners trialled professional user evaluations of innovative e-learning tools with learning design function, collecting D4L case studies and LD sequences in post-16/HE contexts using LAMS and Moodle. The project brought together learning activity sequences within a collaborative e-learning community of practice based on the CAMEL (Collaborative Approaches to the Management of e-Learning) model, contributing to international D4L developments. This paper provides an overview of project outputs in e-learning innovations, including evaluations from teachers and students. The paper explores intentionality in the development of a CoP in design for learning, reporting on trials of LD and social software that bridged tensions between formalised intra-institutional e-learning relationships and inter-institutional professional project team dynamic D4L practitioner interactions. Following a brief report of D4L case studies and feedback, the catalytic role of the 'critical friend' is highlighted and recommended as a key ingredient in the successful development of a nomadic model of communities of practice for managing professional e-learning projects. eLIDA CAMEL Partners included the Association of Learning Technology (ALT), JISC infoNet, three universities and five FE/Sixth Form Colleges. Results reported to JISC demonstrated D4L e-learning innovations by practitioners, illuminated by the role of the 'critical friend'. The project also benefited from formal case study evaluations and the leading work of ALT and JISC infoNet in the development of the CAMEL model.

Student Experience of E-Learning (SEEL)

The Student Experience of E-Learning project (SEEL) was an institutional response to the university’s HEA/JISC Benchmarking exercise (Ryan and Kandler, 2007). The study had a social constructivist approach which recognised the importance of listening to the student voice (JISC 2007) within the University of Greenwich context, to interpret the student experience of e-learning. Nearly 1000 students responded to an online survey on their approaches to, and their use of, learning technology. The quantitative and qualitative questions used included identifying study patterns, using specific online tools, within the context of learning and beyond, and student’s attitudes towards using e-learning in their studies. Initially, individual responses to questions were analysed in depth, giving a general indication of the student experience. Further depth was applied through a filtering mechanism, beginning with a cross-slicing of individual student responses to produce cameos. Audio logs and individual interviews were drawn from these cameos. Analysis of the cameos is in progress but has already revealed some unexpected results. There was a mismatch between students’ expectations of the university’s use of technology and their experiences and awareness of its possible use in other contexts. Students recognised the importance of social interaction as a vehicle for learning (Vygotsky 1978, Bruner 2006) but expressed polarised views on the use of social networking sites such as Facebook for e-learning. Their experiences in commercial contexts led them to see the university VLE as unimaginative and the tutors’ use of it as lacking in vision. Whereas analysis of the individual questions provided a limited picture, the cameos gave a truer reflection of the students lived experiences and identified a gulf between the university’s provision and the students’ expectation of e-learning and their customary use of technology. However it is recognised that the very nature of an online survey necessarily excludes students who chose not to engage, either through lack of skills or through disillusionment and this would constitute a separate area for study.