A case study of the effectiveness of the electronic survey as a research tool for measuring students' experience of e-learning

This paper uses a case study approach to consider the effectiveness of the electronic survey as a research tool to measure the learner voice about experiences of e-learning in a particular institutional case. Two large scale electronic surveys were carried out for the Student Experience of e-Learning (SEEL) project at the University of Greenwich in 2007 and 2008, funded by the UK Higher Education Academy (HEA). The paper considers this case to argue that, although the electronic web-based survey is a convenient method of quantitative and qualitative data collection, enabling higher education institutions swiftly to capture multiple views of large numbers of students regarding experiences of e-learning, for more robust analysis, electronic survey research is best combined with other methods of in-depth qualitative data collection. The advantages and disadvantages of the electronic survey as a research method to capture student experiences of e-learning are the focus of analysis in this short paper, which reports an overview of large-scale data collection (1,000+ responses) from two electronic surveys administered to students using surveymonkey as a web-based survey tool as part of the SEEL research project. Advantages of web-based electronic survey design include flexibility, ease of design, high degree of designer control, convenience, low costs, data security, ease of access and guarantee of confidentiality combined with researcher ability to identify users through email addresses. Disadvantages of electronic survey design include the self-selecting nature of web-enabled respondent participation, which tends to skew data collection towards students who respond effectively to email invitations. The relative inadequacy of electronic surveys to capture in-depth qualitative views of students is discussed with regard to prior recommendations from the JISC-funded Learners' Experiences of e-Learning (LEX) project, in consideration of the results from SEEL in-depth interviews with students. The paper considers the literature on web-based and email electronic survey design, summing up the relative advantages and disadvantages of electronic surveys as a tool for student experience of e-learning research. The paper concludes with a range of recommendations for designing future electronic surveys to capture the learner voice on e-learning, contributing to evidence-based learning technology research development in higher education.

Exploration of the student experience of e-learning: beginning to bridge the learning divide

Introduction This proposal aims, through debate within symposium to explore the student experience of e-learning. The team facilitating the discussion will draw upon their experience of an HEA funded pathfinder project, the main aim of which was to collect learner stories about their experience of using technology in their everyday learning activities at the University of Greenwich across a range of programmes, levels, locations and student groups. Method The project design responded to the growing body of student voice literature and then utilised and built upon the JISC-funded studies that focussed on understanding the learner perspectives on the role of technology in learning, namely: • the LEX study which investigated a broad spectrum of technology use by eliciting rich data about learners’ feelings, beliefs and intentions towards e-learning (Creanor et al, 2006); • the LXP studies which explored disciplinary differences in uses of technology by university students through a variety of methodologies (Conole et al, 2006). Results The symposium will be organised as a round table discussion that will be structured into three sections: • Designing an online survey tool, and the results of our survey. • Exploring student stories. • What can learned from the project and taking the findings back to enhance learning. To stimulate discussion each section will start by asking the participants to discuss and debate a particular question, this will be followed by an interactive presentation by the respective member of the project team who will share the findings of the project and invite contributions to the resulting discussion from personal perspectives. The questions are: • What is effective learning within a context of digital technology? • What are the myths and truths about the identity of today's learners? • What practical changes need to happen in order to see real change? Conclusion The final section of the symposium will invite contributions from the participants in order to collate the views and perspectives of all the participants in order to focus the discussion on the following: • The issues that have arisen as a result of the round table debates. • New speculative approaches to enhancing the student experience. • A controversial stand to the future of Higher Education teaching and learning and the role and integration of technology within that education. The symposium will provide an opportunity to explore the predictive value of Student Experience of E-Learning Laboratory (SEEL) project.

Rethinking the digital divide

The ALT-C 2008 conference theme of 'Re-thinking the Digital Divide' was highlighted in relation to the Greenwich conference theme of ‘Learning from the Learners' Experience of eLearning’ in a plenary session in which the Co-Chair of ALT-C 2008 introduced the forthcoming ALT-C conference at Leeds.

The eLIDA CAMEL nomadic model of collaborative partnership for a community of practice in design for learning

A nomadic collaborative partnership model for a community of practice (CoP) in Design for Learning (D4L) can facilitate successful innovation and continuing appraisals of effective professional practice, stimulated by a 'critical friend' assigned to the project. This paper reports on e-learning case studies collected by the JISC-funded UK eLIDA CAMEL Design for Learning Project. The project implemented and evaluated learning design (LD) tools in higher and further education within the JISC Design for Learning pedagogic programme (2006-07). Project partners trialled professional user evaluations of innovative e-learning tools with learning design function, collecting D4L case studies and LD sequences in post-16/HE contexts using LAMS and Moodle. The project brought together learning activity sequences within a collaborative e-learning community of practice based on the CAMEL (Collaborative Approaches to the Management of e-Learning) model, contributing to international D4L developments. This paper provides an overview of project outputs in e-learning innovations, including evaluations from teachers and students. The paper explores intentionality in the development of a CoP in design for learning, reporting on trials of LD and social software that bridged tensions between formalised intra-institutional e-learning relationships and inter-institutional professional project team dynamic D4L practitioner interactions. Following a brief report of D4L case studies and feedback, the catalytic role of the 'critical friend' is highlighted and recommended as a key ingredient in the successful development of a nomadic model of communities of practice for managing professional e-learning projects. eLIDA CAMEL Partners included the Association of Learning Technology (ALT), JISC infoNet, three universities and five FE/Sixth Form Colleges. Results reported to JISC demonstrated D4L e-learning innovations by practitioners, illuminated by the role of the 'critical friend'. The project also benefited from formal case study evaluations and the leading work of ALT and JISC infoNet in the development of the CAMEL model.

Student Experience of E-Learning (SEEL)

The Student Experience of E-Learning project (SEEL) was an institutional response to the university’s HEA/JISC Benchmarking exercise (Ryan and Kandler, 2007). The study had a social constructivist approach which recognised the importance of listening to the student voice (JISC 2007) within the University of Greenwich context, to interpret the student experience of e-learning. Nearly 1000 students responded to an online survey on their approaches to, and their use of, learning technology. The quantitative and qualitative questions used included identifying study patterns, using specific online tools, within the context of learning and beyond, and student’s attitudes towards using e-learning in their studies. Initially, individual responses to questions were analysed in depth, giving a general indication of the student experience. Further depth was applied through a filtering mechanism, beginning with a cross-slicing of individual student responses to produce cameos. Audio logs and individual interviews were drawn from these cameos. Analysis of the cameos is in progress but has already revealed some unexpected results. There was a mismatch between students’ expectations of the university’s use of technology and their experiences and awareness of its possible use in other contexts. Students recognised the importance of social interaction as a vehicle for learning (Vygotsky 1978, Bruner 2006) but expressed polarised views on the use of social networking sites such as Facebook for e-learning. Their experiences in commercial contexts led them to see the university VLE as unimaginative and the tutors’ use of it as lacking in vision. Whereas analysis of the individual questions provided a limited picture, the cameos gave a truer reflection of the students lived experiences and identified a gulf between the university’s provision and the students’ expectation of e-learning and their customary use of technology. However it is recognised that the very nature of an online survey necessarily excludes students who chose not to engage, either through lack of skills or through disillusionment and this would constitute a separate area for study.