Development, evaluation and use of a student experience survey in undergraduate science laboratories: the advancing science by enhancing learning in the laboratory student laboratory learning experience survey

Student experience surveys have become increasingly popular to probe various aspects of processes and outcomes in higher education, such as measuring student perceptions of the learning environment and identifying aspects that could be improved. This paper reports on a particular survey for evaluating individual experiments that has been developed over some 15 years as part of a large national Australian study pertaining to the area of undergraduate laboratories—Advancing Science by Enhancing Learning in the Laboratory. This paper reports on the development of the survey instrument and the evaluation of the survey using student responses to experiments from different institutions in Australia, New Zealand and the USA. A total of 3153 student responses have been analysed using factor analysis. Three factors, motivation, assessment and resources, have been identified as contributing to improved student attitudes to laboratory activities. A central focus of the survey is to provide feedback to practitioners to iteratively improve experiments. Implications for practitioners and researchers are also discussed.

Innovating engineering education: the RVS approach:an empirically grounded method to improved practice and enhanced student experience

This paper builds on previous work (Clark, 2009; Clark & Andrews 2011, 2014) to continue the debate around a seemingly universal question…“How can educational theory be applied to engineering education in such a way so as to make the subject more accessible and attractive to students? It argues that there are three key elements to student success; Relationships, Variety & Synergy (RVS). By further examining the purposefully developed bespoke learning and teaching approach constructed around these three elements (RVS) the discourse in this paper links educational theory to engineering education and in doing so further develops arguments for the introduction of a purposefully designed pedagogic approach for use in engineering education.

The relationship between participation in unethical behaviors during the nursing student experience and participation in unethical behaviors as a registered nurse

The effect of unethical behaviors in health care settings is an important issue in the safe care of clients and has been a concern of the nursing profession for some time. The purpose of this study was to examine the relationship between use of unethical behaviors in the nursing student experience and the use of unethical behaviors in the workplace as a registered nurse. In addition, the relationship between the severity of unethical behaviors utilized in the classroom, clinical setting and those in the workplace was examined. To insure greater honesty in self-report, only a limited number of demographic variables were requested from participants.^ During the summer of 1997, a 56 item questionnaire was distributed to registered nurses enrolled in either undergraduate or graduate courses in a public or private institution. The participants were asked to self-report their own use of unethical behaviors as well as their peers use of unethical behaviors. In order to assign a severity score for each item, nursing school faculty were asked to rate severity of unethical behaviors which could be used during the nursing student experience and nursing administrators were asked to rate unethical behaviors which could be used in the workplace.^ A significant positive relationship was found between individuals' use of unethical behaviors during nursing school and those used in the workplace $r=.630.$ A significant positive relationship was found between the severity of unethical behaviors used in the nursing student experience and the severity of unethical behaviors used in the workplace $r=.637.$ No relationship was found between years of practice, type of initial nursing education and whether or not the participant was raised inside or outside the United States and the use of unethical behaviors. ^

The relationship between participation in unethical behaviors during the nursing student experience and participation in unethical behaviors as a registered nurse

The effect of unethical behaviors in health care settings is an important issue in the safe care of clients and has been a concern of the nursing profession for some time. The purpose of this study was to examine the relationship between use of unethical behaviors in the nursing student experience and the use of unethical behaviors in the workplace as a registered nurse. In addition, the relationship between the severity of unethical behaviors utilized in the classroom, clinical setting and those in the workplace was examined. To insure greater honesty in self-report, only a limited umber of demographic variables were requested from participants. During the summer of 1997, a 56 item questionnaire was distributed to registered nurses enrolled in either undergraduate or graduate courses in a public or private institution. The participants were asked to self-report their own use of unethical behaviors as well as their peers use of unethical behaviors. In order to assign a severity score for each item, nursing school faculty were asked to rate severity of unethical behaviors which could be used during the nursing student experience and nursing administrators were asked to rate unethical behaviors which could be used in the workplace. A significant positive relationship was found between individuals' use of unethical behaviors during nursing school and those used in the workplace r = .630. A significant positive relationship was found between the severity of unethical behaviors used in the nursing student experience and the severity of unethical behaviors used in the workplace r = .637. No relationship was found between years of practice, type of initial nursing education and whether or not the participant was raised inside or outside the United States and the use of unethical behaviors.

Ethical reasoning through simulation: a phenomenological analysis of student experience

Background
Medical students transitioning into professional practice feel underprepared to deal with the emotional complexities of real-life ethical situations. Simulation-based learning (SBL) may provide a safe environment for students to probe the boundaries of ethical encounters. Published studies of ethics simulation have not generated sufficiently deep accounts of student experience to inform pedagogy. The aim of this study was to understand students’ lived experiences as they engaged with the emotional challenges of managing clinical ethical dilemmas within a SBL environment.

Methods
This qualitative study was underpinned by an interpretivist epistemology. Eight senior medical students participated in an interprofessional ward-based SBL activity incorporating a series of ethically challenging encounters. Each student wore digital video glasses to capture point-of-view (PoV) film footage. Students were interviewed immediately after the simulation and the PoV footage played back to them. Interviews were transcribed verbatim. An interpretative phenomenological approach, using an established template analysis approach, was used to iteratively analyse the data.

Results
Four main themes emerged from the analysis: (1) ‘Authentic on all levels?’, (2)‘Letting the emotions flow’, (3) ‘Ethical alarm bells’ and (4) ‘Voices of children and ghosts’. Students recognised many explicit ethical dilemmas during the SBL activity but had difficulty navigating more subtle ethical and professional boundaries. In emotionally complex situations, instances of moral compromise were observed (such as telling an untruth). Some participants felt unable to raise concerns or challenge unethical behaviour within the scenarios due to prior negative undergraduate experiences.

Conclusions
This study provided deep insights into medical students’ immersive and embodied experiences of ethical reasoning during an authentic SBL activity. By layering on the human dimensions of ethical decision-making, students can understand their personal responses to emotion, complexity and interprofessional working. This could assist them in framing and observing appropriate ethical and professional boundaries and help smooth the transition into clinical practice.

‘Computer games can get your brain working’: student experience and perceptions of digital games in the classroom

There is considerable enthusiasm in many quarters for the incorporation of digital games into the classroom, and the capacity of games to engage and challenge players, present complex representations and experiences, foster collaborative learning, and promote deep learning. But while there is increasing research documenting the progress and outcomes of game-based learning, relatively little attention is paid to student perceptions and voice. In order to effectively target game-based learning pedagogy, it is important to understand students' previous experience, if any, of the use of games in the classroom, and what they made of these. In this paper, we present findings from a survey of 270 primary and secondary school students in Year Levels 4–9 (aged 9–14) in 6 Queensland schools at the start of a 3-year Australian Research Council project researching the use of digital games in school to promote literacy and learning.

Voices of dissent: Unpacking Vietnamese international student experience

Purpose – This study examines Vietnamese international students’ experiences with thecampus learning environment by analysing differences in staff and student perceptions.Design/methodology/approach – Two focus groups (N = 12) and ten (10) in-depthinterviews were conducted with Vietnamese students and four (4) in-depth interviews withthe university staff (totalling 26).Findings – The findings show a greater divergence of views between students and staff onteaching and learning than English language proficiency and student support services.These key differences were influenced by students’ prior expectations of their learningenvironment.Research limitations/implications – This study is limited to one group of internationalstudents and to one Australian university.Practical implications – The academics and administrators must have a goodunderstanding of international students’ learning backgrounds and expectations to enhancetheir positive experience; Appropriate teaching skills and practices are essential foracademics to meet the current needs of students. More effective training for internationalstudents to understand the multicultural nature of Australia is also essential.Originality/value – This study contributes to the literature by identifying the experienceof Vietnamese international students in a western university, which is, a relatively underresearchednationality compared to other Asian nationalities such as Chinese and Indian.

Utilising the virtual learning environment to encourage faculty reflection and improve the student learning experience

Student evaluations of teaching (SETs) are the most frequent form of faculty performance in the classroom, though they tend to be used as summative rather than formative evaluations. In this chapter, a project involving the use of a virtual learning environment for formative, weekly SETs is explored from both the student and faculty point of view at a rural university college in the United Kingdom. This project encouraged student participation in creating the learning environment and faculty reflection on how to improve the student experience. From the student perspective, the weekly anonymous evaluations were useful for providing feedback; however, students tended to only respond if they were not satisfied with the faculty member. The exception to this was that some students were more motivated to complete the evaluation forms if they believed the faculty member was utilising their feedback. From the faculty perspective, the feedback was not as detailed as they had expected, and some questioned whether it was worth the effort of conducting formative evaluations if the response rate was so low. Others used the feedback for reflective purposes, and it was found that those that reflected on their work at higher levels tended to receive a greater year-on-year increase in their end of year teaching evaluations.

Experience of problem-based learning (PBL) in virtual space : a phenomenographical study

This thesis reports the outcomes of an investigation into students’ experience of Problem-based learning (PBL) in virtual space. PBL is increasingly being used in many fields including engineering education. At the same time many engineering education providers are turning to online distance education. Unfortunately there is a dearth of research into what constitutes an effective learning experience for adult learners who undertake PBL instruction through online distance education. Research was therefore focussed on discovering the qualitatively different ways that students experience PBL in virtual space. Data was collected in an electronic environment from a course, which adopted the PBL strategy and was delivered entirely in virtual space. Students in this course were asked to respond to open-ended questions designed to elicit their learning experience in the course. Data was analysed using the phenomenographical approach. This interpretative research method concentrated on mapping the qualitative differences in students’ interpretations of their experience in the course. Five qualitatively different ways of experiencing were discovered: Conception 1: ‘A necessary evil for program progression’; Conception 2: ‘Developing skills to understand, evaluate, and solve technical Engineering and Surveying problems’; Conception 3: ‘Developing skills to work effectively in teams in virtual space’; Conception 4: ‘A unique approach to learning how to learn’; Conception 5: ‘Enhancing personal growth’. Each conception reveals variation in how students attend to learning by PBL in virtual space. Results indicate that the design of students’ online learning experience was responsible for making students aware of deeper ways of experiencing PBL in virtual space. Results also suggest that the quality and quantity of interaction with the team facilitator may have a significant impact on the student experience in virtual PBL courses. The outcomes imply pedagogical strategies can be devised for shifting students’ focus as they engage in the virtual PBL experience to effectively manage the student learning experience and thereby ensure that they gain maximum benefit. The results from this research hold important ramifications for graduates with respect to their ease of transition into professional work as well as their later professional competence in terms of problem solving, ability to transfer basic knowledge to real-life engineering scenarios, ability to adapt to changes and apply knowledge in unusual situations, ability to think critically and creatively, and a commitment to continuous life-long learning and self-improvement.

International students in Australian universities : gap between student expectations and realities

Australian Universities are very successful in attracting large number of international students. A large proportion of University revenue comes from the full fee paying international students. However, there have been many reports that international students face numerous problems when they arrive in Australia. The common management practice is to provide support staff services to deal with the orientation and welfare of international students. Such service units act as intermediaries between the students and the teaching and learning community of the university. However, the actual experience of international students may be difficult for support staff, counsellors, advisers and academic staff to anticipate. There is little information on the actual experience of students relative to their expectations. This study aimed at securing a deeper understanding of the contextually relevant issues facing by international students in Australian universities in order to develop management strategies aimed at improved teaching and learning outcomes for international students. Using a highly reliable survey questionnaire, a questionnaire survey was conducted among the international students at Queensland University of Technology (QUT), Brisbane, Australia. About 180 engineering students responded in the survey resulting in a response rate of 81%. Results indicate that international students face many difficulties including understanding colloquial language, Australian accent, cost of tuition, feelin isolation, safety, security, health services, accommodation and part time jobs. They also face difficulty in coping with learning methods in Australia, particularly in research report writing. However, they are happy with their lecturers and find them very helpful. Many of the students lacked the information regarding various community groups, recreational and sports facilities in Australia before arriving. Findings of the study show that there is a significant gap between the expectation of the students before coming to Australia and actual experience they experience here. Importantly, there is a lack of coordination between international students, international student services (ISS) and university management and as a consequence there have been little improvement in conditions. There is no direct link between student experience and University management. Many important suggestions arisen from this study and most important suggestion is that the student information system should be integrated with the University enterprise resource planning (ERP) to reduce the huge gap between international student expectation and actual experiences.

Moving beyond transition pedagogy: Maturity models and student engagement

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). This paper proposes that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities. We will describe the Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012) we are currently investigating. We will discuss if our research may address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. The first aim of our research is to extend the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). The second aim of this research is to move beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf

Verifying a Student Engagement, Success and Retention Maturity Model (SESR-MM): A reality check of the model using data derived from three universities

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). We have proposed that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities and we are currently articulating a Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012). Our research aims to address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. Our research extends the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). Our research also moves beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. The issues explored in this workshop are (i) whether the maturity model concept can be usefully applied to provide a measure of institutional capability for SESR; (ii) whether the SESR-MM can be used to assess the maturity of a particular set of institutional practices; and (iii) whether a collective assessment of an institution’s SESR capabilities can provide an indication of the maturity of the institution’s SESR activities. The workshop will be approached in three stages. Firstly, participants will be introduced to the key characteristics of maturity models, followed by a discussion of the SESR-MM and the processes involved in its development. Secondly, participants will be provided with resources to facilitate the development of a maturity model and an assessment instrument for a range of institutional processes and related practices. In the final stage of the workshop, participants will “assess” the capability of these practices to provide a collective assessment of the maturity of these processes. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf