JUSTEIS: JISC Usage Surveys: Trends in Electronic Information Services Final report 2003/2004 Cycle Five

Urquhart, C., Spink, S., Thomas, R., Yeoman, A., Durbin, J., Turner, J., Fenton, R. & Armstrong, C. (2004). JUSTEIS: JISC Usage Surveys: Trends in Electronic Information Services Final report 2003/2004 Cycle Five. Aberystwyth: Department of Information Studies, University of Wales Aberystwyth. Sponsorship: JISC

Executive summary Aims and objectives of JUSTEIS Cycle Five The aims of JUSTEIS 5 were to: ? consolidate the monitoring of user behaviour undertaken during the previous four cycles, in accordance with changes agreed by the JCALT committee ? extend the monitoring and evaluation framework to new communities which are now a concern of the JISC ? respond, through a variety of activities, to issues identified in discussions with other JISC committees, taking into account the review of JISC evaluation activities, the changes in government policies for formal education, lifelong learning and workplace learning. The approach agreed with JCALT was to: ? consolidate the monitoring and evaluation (M&E) framework for studying user behaviour AND include new audiences (e.g. Adult and Community Learning (ACL), sixth form colleges (SFC) ? build on the in-depth qualitative data analysis developed in Cycle 4, working with JUBILEE, and provide a data archive to help the audience for the M&E framework find answers to questions they may have (on staff development and student skills, for example) ? focus on dissemination activities, ensuring that the lessons learned are targeted to particular audiences. Scope of project Number of students interviewed = 226 (146FE, 80HE), across all types of disciplines Number of questionnaires obtained from students = 142 (28 SFC, 82 FE, 32 HE) Number of participating institutions = 28 (18FE , 10HE) (Russell to sixth form colleges Number of participating departments = 28 (18FE, 10HE) Number of senior library managers interviewed = 11 Number of academic staff interviewed = 17 (9FE, 8HE) Key messages Virtual/Managed Learning Environments FE colleges are making more use of Virtual Learning Environments now. There is slow progress towards greater use of VLE functionality in HEIs, but students comment on the piecemeal approach in some institutions. One possible lever for greater use of VLEs is the ?drop box?, an electronic post box facility for electronic submission of assignments. Receiving assignments in this format makes checking for plagiarism and incorrect Web citation easier, but there are huge implications for administrative and examination systems within institutions in moving from a paper-based system to an electronic system. Information services need to know more about the ways in which staff read and assess coursework assignments where they do this, for example. There needs to be far more emphasis on mobile computing support for academic staff if electronic submission is the way forward. ?Copy and paste? plagiarism from the Internet This is not a new problem by any means students from the first cycle of JUSTEIS in 1999/2000 talked about copying and pasting from the Internet into their assignments. The availability of diagrams, and other graphics was often the key attraction as a way of making their assignments more attractive, and different from those of other students. A secondary motivation was saving on printing costs when working on campus. In copying and pasting in the way they described they often lost the URL, unless they were very methodical in their approach. Among the sites visited this year, the same message occurs as last year. FE students appear to have more reinforcement from teaching staff on what constitutes plagiarism, and what must be done to avoid the charge of plagiarism. A more active approach is now required by academic staff in HE and some staff are starting to do this. The evidence from the literature is not comforting, with around half the science students in one survey indicating that if they were pressured for time, and felt at risk of failing a module, they would be willing to copy/paste from Web held materials without modification or referencing if this would save them from failing a module. Estimates of the actual frequency of such practice suggest that anything from 10% to 20% of students actually do pass off other peoples? work as their own knowingly. How could Library and Information Services staff respond? Apart from promoting the JISC Plagiarism service the conclusions from JUSTEIS are: ? Students need guidance from teaching staff on what counts as plagiarism verbal guidance could be backed up by clear and succinct guides ? Web citation practice needs to be better, and students need to know that the citations will be checked, and that their text will be checked. ? Electronic submission of assignments may make such checking much easier but some academic staff will need convinced of this. Open access journals and institutional repositories In 2000 the concept of a page charge model for journal publishing (to provide more open access) was novel to most of the senior librarians interviewed for JUSTEIS. There was just the glimmer of interest in institutional repositories. Developments during 2003 and 2004, particularly in the biomedical arena, have forced a change of pace. In the UK, the House of Commons Science and Technology committee has recommended that universities make their research papers free online, and that government funded research grants ought to make free access to the research findings a condition of the grant. JISC is doing work on institutional repositories and encouragement of open access journals but there needs to be greater awareness of this work among senior librarians generally. Performance measurement and benchmarking There is not a shortage of work on performance measurement for digital libraries probably too much guidance exists. The case study work for JUSTEIS in this cycle indicated that the important aspects are still hard to measure simply and reliably. Libraries need to know whether the provision of EIS has had a beneficial impact on student learning and whether information skills have improved. Objective measurement is almost impossible, proxy indicators are maybe unreliable or not sufficiently valid. The case study work, combined with evidence from the literature on electronic library evaluation, suggests that a simple balanced scorecard approach to assessing the impact on e-learning would include these key performance indicators, for common situations where HE and FE institutions are working together: ? Percentage of tailored electronic resources available off campus as well as on campus (student(customer) benefit in saving time) ? Policy link: widening participation ? Percentage of modules/departments for which library staff have authorised access to relevant assignments for checking information skills development (Library staff need feedback on their contribution to skills training and student learning) ? Policy link: e-learning, employability, and role changes of library staff in academic support ? Number of meetings and visits to departments AND/OR Development of specialist Web sites/pages for VLEs (or incorporation of Regional Support Centre subject guides (or equivalent) in course materials) AND/OR Percentage of modules to which library staff have access on the VLE to provide students information skills support (Evidence shows that academic and library staff need to work together, measures required of internal processes to assess whether joint working is increasing) ? Policy link: ?Byte? size e-learning, role changes of library staff in academic support, role changes for academic staff in e-learning ? Percentage of HE licensed resources available to FE students and tutors on franchised courses (Evidence points to this as a key financial barrier) ? Policy link: open access initiatives, licensing ? Percentage of library and academic staff with recognised competencies in e-learning support (An indicator of learning and growth, requires input from the Association of Learning Technologists) ? Policy link: role changes for library and academic staff in e-learning development The impact of training on use of EIS JUSTEIS evidence shows that use and awareness of electronic journals has increased since 1999 and the number of students reporting that they received LIS training has increased. LIS input to training has a statistically significant effect on e-journal use and awareness, and joint training. Joint working, LIS staff and academic staff working together, appears the most effective route to promoting uptake of e-journals. Gender differences in taking advice There is no major difference in habitual information seeking routines between male and female students. However, female students in JUSTEIS were significantly more likely than men to have relied partly on course materials or tutor advice. Female students are significantly more likely to have approached LIS staff for advice or to have used training materials supplied by LIS staff. Further Education and Higher Education and lifelong learning Most students, whether in FE or HE, explained how the need for money from part-time work shaped their approach to preparing coursework. If electronic information services provide useful information quickly and reliably they will be used, and if not, they won?t. Information on the move? Results this year show a shift in popularity towards the use of texting and mobile phones as the electronic information services most used by students. The Internet and search engines are in second place, followed by email. But the message is clear to reach students you should use their mobile phone number.