Developing a robust self evaluation framework for active learning:the first stage of an ERASMUS+ project (QAEMarketPlace4HEI)

In ensuring the quality of learning and teaching in Higher Education, self-evaluation is an important component of the process. An example would be the approach taken within the CDIO community whereby self-evaluation against the CDIO standards is part of the quality assurance process. Eight European universities (Reykjavik University, Iceland; Turku University of Applied Sciences, Finland; Aarhus University, Denmark; Helsinki Metropolia University of Applied Sciences, Finland; Ume? University, Sweden; Telecom Bretagne, France; Aston University, United Kingdom; Queens University Belfast, United Kingdom) are engaged in an EU funded Erasmus + project that is exploring the quality assurance process associated with active learning. The development of a new self-evaluation framework that feeds into a ?Marketplace? where participating institutions can be paired up and then engage in peer evaluations and sharing around each institutions approach to and implementation of active learning. All of the partner institutions are engaged in the application of CDIO within their engineering programmes and this has provided a common starting point for the partnership to form and the project to be developed. Although the initial focus will be CDIO, the longer term aim is that the approach could be of value beyond CDIO and within other disciplines. The focus of this paper is the process by which the self-evaluation framework is being developed and the form of the draft framework. In today?s Higher Education environment, the need to comply with Quality Assurance standards is an ever present feature of programme development and review. When engaging in a project that spans several countries, the wealth of applicable standards and guidelines is significant. In working towards the development of a robust Self Evaluation Framework for this project, the project team decided to take a wide view of the available resources to ensure a full consideration of different requirements and practices. The approach to developing the framework considered: a) institutional standards and processes b) national standards and processes e.g. QAA in the UK c) documents relating to regional / global accreditation schemes e.g. ABET d) requirements / guidelines relating to particular learning and teaching frameworks e.g. CDIO. The resulting draft self-evaluation framework is to be implemented within the project team to start with to support the initial ?Marketplace? pairing process. Following this initial work, changes will be considered before a final version is made available as part of the project outputs. Particular consideration has been paid to the extent of the framework, as a key objective of the project is to ensure that the approach to quality assurance has impact but is not overly demanding in terms of time or paperwork. In other words that it is focused on action and value added to staff, students and the programmes being considered.

Controlled study into the effectiveness of individual client-centred counselling for young offenders in residential care

This study is toe first documented account in the British Isles of an evaluation of the effectiveness of client-centred counselling with young offenders in secure residential care. It is a test of Rogers' (1957) position on the 'necessary and sufficient' conditions of therapeutic personality change within a counselling relationship. Forty teenage male offenders, the subjects of Training School Orders, were randomly allocated in equal numbers to either an experimental or control group. Boys in the experimental group received weekly individual sessions of client-centred counselling over a seven month period. Boys in the control group received no formal counselling but were shown to have similar intellectual, personality, socio-economic and criminal backgrounds to those in the experimental group. It was hypothesised that counselled subjects would show more positive outcomes than control subjects over a range of measures relating to criminal behaviour and self-conception. The results indicated that the counselled subjects had a significantly lower rate of offending and a srnaller range of offences over a mean follow-up period of 2.5 years. They were also licensed from the institution significantly earlier and spent less time in custody during a one year follow-up after counselling was completed. Self-conception measures gave less clear-cut results. The direction of change towards better adjustment favoured the counselled subjects but the magnitude was often small. Those counselled subjects with most positive behaviour change tended to have significantly improved self-evaluation, less self/ideal self discrepancy and more variation on 'actual' self concept compared to pre-counselling. The results are discussed in the context of client-centred theory, methodological adequacy of the experimental design, and their application to the future treatment of young offenders in secure residential care.

Provision of pharmaceutical care by community pharmacists:a comparison across Europe

Objective To investigate the provision of pharmaceutical care by community pharmacists across Europe and to examine the various factors that could affect its implementation. Methods A questionnaire-based survey of community pharmacies was conducted within 13 European countries. The questionnaire consisted of two sections. The first section focussed on demographic data and services provided in the pharmacy. The second section was a slightly adapted version of the Behavioral Pharmaceutical Care Scale (BPCS) which consists of three main dimensions (direct patient care activities, referral and consultation activities and instrumental activities). Results Response rates ranged from 10–71% between countries. The mean total score achieved by community pharmacists, expressed as a percentage of the total score achievable, ranged from 31.6 (Denmark) to 52.2% (Ireland). Even though different aspects of pharmaceutical care were implemented to different extents across Europe, it was noted that the lowest scores were consistently achieved in the direct patient care dimension (particularly those related to documentation, patient assessment and implementation of therapeutic objectives and monitoring plans) followed by performance evaluation and evaluation of patient satisfaction. Pharmacists who dispensed higher daily numbers of prescriptions in Ireland, Germany and Switzerland had significantly higher total BPCS scores. In addition, pharmacists in England and Ireland who were supported in their place of work by other pharmacists scored significantly higher on referral and consultation and had a higher overall provision of pharmaceutical care. Conclusion The present findings suggest that the provision of pharmaceutical care in community pharmacy is still limited within Europe. Pharmacists were routinely engaged in general activities such as patient record screening but were infrequently involved in patient centred professional activities such as the implementation of therapeutic objectives and monitoring plans, or in self-evaluation of performance.

The UK MPharm (1) - are we over reliant on traditional teaching methods?

Introduction-The design of the UK MPharm curriculum is driven by the Royal Pharmaceutical Society of Great Britain (RPSGB) accreditation process and the EU directive (85/432/EEC).[1] Although the RPSGB is informed about teaching activity in UK Schools of Pharmacy (SOPs), there is no database which aggregates information to provide the whole picture of pharmacy education within the UK. The aim of the teaching, learning and assessment study [2] was to document and map current programmes in the 16 established SOPs. Recent developments in programme delivery have resulted in a focus on deep learning (for example, through problem based learning approaches) and on being more student centred and less didactic through lectures. The specific objectives of this part of the study were (a) to quantify the content and modes of delivery of material as described in course documentation and (b) having categorised the range of teaching methods, ask students to rate how important they perceived each one for their own learning (using a three point Likert scale: very important, fairly important or not important). Material and methods-The study design compared three datasets: (1) quantitative course document review, (2) qualitative staff interview and (3) quantitative student self completion survey. All 16 SOPs provided a set of their undergraduate course documentation for the year 2003/4. The documentation variables were entered into Excel tables. A self-completion questionnaire was administered to all year four undergraduates, using a pragmatic mixture of methods, (n=1847) in 15 SOPs within Great Britain. The survey data were analysed (n=741) using SPSS, excluding non-UK students who may have undertaken part of their studies within a non-UK university. Results and discussion-Interviews showed that individual teachers and course module leaders determine the choice of teaching methods used. Content review of the documentary evidence showed that 51% of the taught element of the course was delivered using lectures, 31% using practicals (includes computer aided learning) and 18% small group or interactive teaching. There was high uniformity across the schools for the first three years; variation in the final year was due to the project. The average number of hours per year across 15 schools (data for one school were not available) was: year 1: 408 hours; year 2: 401 hours; year 3: 387 hours; year 4: 401 hours. The survey showed that students perceived lectures to be the most important method of teaching after dispensing or clinical practicals. Taking the very important rating only: 94% (n=694) dispensing or clinical practicals; 75% (n=558) lectures; 52% (n=386) workshops, 50% (n=369) tutorials, 43% (n=318) directed study. Scientific laboratory practices were rated very important by only 31% (n=227). The study shows that teaching of pharmacy to undergraduates in the UK is still essentially didactic through a high proportion of formal lectures and with high levels of staff-student contact. Schools consider lectures still to be the most cost effective means of delivering the core syllabus to large cohorts of students. However, this does limit the scope for any optionality within teaching, the scope for small group work is reduced as is the opportunity to develop multi-professional learning or practice placements. Although novel teaching and learning techniques such as e-learning have expanded considerably over the past decade, schools of pharmacy have concentrated on lectures as the best way of coping with the huge expansion in student numbers. References [1] Council Directive. Concerning the coordination of provisions laid down by law, regulation or administrative action in respect of certain activities in the field of pharmacy. Official Journal of the European Communities 1985;85/432/EEC. [2] Wilson K, Jesson J, Langley C, Clarke L, Hatfield K. MPharm Programmes: Where are we now? Report commissioned by the Pharmacy Practice Research Trust., 2005.

The UK MPharm (2) - Do we over assess?

Introduction-The pace of structural change in the UK health economies, the new focus on regulation and the breaking down of professional boundaries means that the Royal Pharmaceutical Society of Great Britain (RPSGB) has to continually review the scope, range and outputs of education provided by schools of pharmacy (SOPs). In SOPs, the focus is on equipping students with the knowledge, skills and attitudes necessary to successfully engage with the pre-registration year. The aim of this study [1] was to map current programmes and undergraduate experiences to inform the RPSGB debate. The specific objectives of this paper are to describe elements of the survey of final year undergraduates, to explore student opinions and experiences of their workload, teaching, learning and assessment. Material and methods-The three main research techniques were: (1) quantitative course document review, (2) qualitative staff interview and (3) quantitative student self completion survey. The questions in the survey were based on findings from exploratory focus group work with BPSA (British Pharmaceutical Students’ Association) members and were designed to ascertain if views expressed in the focus groups on the volume and format of assessments were held by the general student cohort. The student self completion questionnaire consisting of 31 questions, was administered in 2005 to all (n=1847) final year undergraduates, using a pragmatic mixture of methods. The sample was 15 SOPs within the UK (1 SOP opted out). The total response rate was 50.62% (n=935): it varied by SOP from 14.42% to 84.62%. The survey data were analysed (n=741) using SPSS, excluding non-UK students who may have undertaken part of their studies within a non-UK university. Results and discussion • 76% (n=562) respondents considered that the amount of formal assessment was about right, 21% (n=158) thought it was too much. • There was agreement that the MPharm seems to have more assessment than other courses, with 63% (n=463) strongly agreeing or agreeing. • The majority considered the balance between examinations and coursework was about right (67%, n=498), with 27% (n=198) agreeing that the balance was too far weighted towards examinations. • 57% (n=421) agreed that the focus of MPharm assessment was too much towards memorised knowledge, 40% (n=290) that it was about right. • 78% (n= 575) agreed with the statement “Assessments don’t measure the skills for being a pharmacist they just measure your knowledge base”. Only 10% (n=77) disagreed. • Similarly 49% (n=358) disagreed with, and 35% (n=256) were not sure about the statement “I consider that the assessments used in the MPharm course adequately measure the skills necessary to be a pharmacist”. Only 17% (n=124) agreed. Experience from this study shows the difficulty of administering survey instruments through UK Schools of Pharmacy. It is heavily dependent on timing, goodwill and finding the right person. The variability of the response rate between SOPs precluded any detailed analysis by School. Nevertheless, there are some interesting results. Issues raised in the exploratory focus group work about amount of assessment and over reliance on knowledge have been confirmed. There is a real debate to be had about the extent to which the undergraduate course, which must instil scientific knowledge, can provide students with the requisite qualities, skills, attitudes and behaviour that are more easily acquired in the pre-registration year. References [1] Wilson K, Jesson J, Langley C, Clarke L, Hatfield K. MPharm Programmes: Where are we now? Report commissioned by the Pharmacy Practice Research Trust., 2005.

Using accreditation criteria for collaborative quality enhancement

This paper describes a process to enhance the quality of higher education. At the heart of the process is a cross-sparring collaborative model, whereby institutions are critical friends. This is based on a prior self-evaluation, where the institution / programme identifies quality criteria it wants to improve. Part of the process is to ensure the documentation of best practices so that they can be shared with others in a so called market place. Linking the best practices to a criterion makes them searchable on a large scale. Optimal pairings of institutions can then take place for the cross-sparring activities.

Antecedents and consequences of service quality in consumer evaluation of self-service internet technologies

Advances in technology coupled with increasing labour costs have caused service firms to explore self-service delivery options. Although some studies have focused on self-service and use of technology in service delivery, few have explored the role of service quality in consumer evaluation of technology-based self-service options. By integrating and extending the self-service quality framework the service evaluation model and the Technology Acceptance Model the authors address this emerging issue by empirically testing a comprehensive model that captures the antecedents and consequences of perceived service quality to predict continued customer interaction in the technology-based self-service context of Internet banking. Important service evaluation constructs like perceived risk, perceived value and perceived satisfaction are modelled in this framework. The results show that perceived control has the strongest influence on service quality evaluations. Perceived speed of delivery, reliability and enjoyment also have a significant impact on service quality perceptions. The study also found that even though perceived service quality, perceived risk and satisfaction are important predictors of continued interaction, perceived customer value plays a pivotal role in influencing continued interaction.

Peer mentoring works! How peer mentoring enhances student success in Higher Education:evaluation toolkit

This toolkit, published by the HEA, provides colleagues across the Sector with the practical and methodological tools to empirically evaluate peer mentoring and tutoring. This evaluation kit provides two data collection tools that may be adopted and adapted to meet institutional requirements. The first of these is a survey, developed out of the original survey used in the Peer Mentoring Works Project. Some questions have been added as a result of reflexive application of colleagues and students input as the project has progressed. The second part of the toolkit comprises a qualitative interview guide. This guide is similar to the one used in the research, but again it has been further developed as a result of the project. It may be adapted for use in focus groups or one-to-one interviews. The final document within the kit is a sample consent form.

Run-time model evaluation for requirements model-driven self-adaptation

A self-adaptive system adjusts its configuration to tolerate changes in its operating environment. To date, requirements modeling methodologies for self-adaptive systems have necessitated analysis of all potential system configurations, and the circumstances under which each is to be adopted. We argue that, by explicitly capturing and modelling uncertainty in the operating environment, and by verifying and analysing this model at runtime, it is possible for a system to adapt to tolerate some conditions that were not fully considered at design time. We showcase in this paper our tools and research results. © 2012 IEEE.