Supporting active learning in an undergraduate geotechnical engineering course using group-based audience response systems quizzes

The use of audience response systems (ARSs) or ‘clickers’ in higher education has increased over the recent years, predominantly owing to their ability to actively engage students, for promoting individual and group learning, and for providing instantaneous feedback to students and teachers. This paper describes how group-basedARSquizzes have been integrated into an undergraduate civil engineering course on foundation design. Overall, theARSsummary quizzes were very well received by the students. Feedback obtained from the students indicates that the majority believed the group-based quizzes were useful activities, which helped to improve their understanding of course materials, encouraged self-assessment, and assisted preparation for their summative examination. Providing students with clickers does not, however, necessarily guarantee the class will be engaged with the activity. If an ARS activity is to be successful, careful planning and design must be carried out and modifications adopted where necessary, which should be informed by the literature and relevant student feedback.

Trajectories of Learning Dispositions and Attitudes in Upper Primary Schools in Northern Ireland: What exactly is going on?

There is increasing research and policy interest in the importance of attitudes to learning, learning orientations and learning dispositions (however they are labelled), not only because they influence traditional measures of school achievement but also because they facilitate how well children function at school, with implications for their future learning. This paper reports the findings on pupils’ learning dispositions and attitudes from two separate cohorts of pupils as they progress through upper primary school (Key Stage 2) in 50 schools in Northern Ireland. (These data are drawn from two different longitudinal studies and the data collection period predates the introduction of the new Northern Ireland Curriculum.) Approximately 1200 pupils completed seven scales from the Assessment of Learner-Centred Practices, ALCPs (McCombs and Lauer, 1997) at three time points, at the end of P5 (9 year olds), at the end of P6 (10 years olds) and at the end of P7 (11 year olds). ALCPs draws on an extensive research base that has identified cognitive and motivational dispositions and attitudes that are associated with a positive orientation to learning, and ultimately with positive progress in school (Alexander and Murphy, 1998). Although each scale can be considered separately, the seven scales cluster into two groups: self-efficacy, mastery orientation, active learning strategies and curiosity are all predicted to be pro-learning; and challenge avoidance, work avoidance, and – to a lesser extent – performance orientation, are predicted to be negatively associated with learning. The general trajectory in the children’s self-evaluations shows that they are becoming less pro-learning over time, with significant decreases in their self-ratings of active learning, curiosity, mastery orientation and self-efficacy. At the same time, there is some evidence that they work harder and put more effort into their work but this is not accompanied by maintaining their previous pro-learning motivations and strategies. The pattern is consistently more negative for boys than for girls. There are very few differences between the two cohorts indicating that the pattern is not confined to a specific cohort. These findings are challenging and will be interrogated with regard to two questions – are the changes related to the influence of the children’s school experiences per se or are they more related to developmental differences as children adopt more critical appraisals of their personal attributes and efforts as they get older? Whatever the reason, these learning dispositions and attitudes are important as they contribute significantly to school achievement even when the more traditional predictors like gender and ability are taken into account.

Evaluation of sampling methods for learning from imbalanced data

The problem of learning from imbalanced data is of critical importance in a large number of application domains and can be a bottleneck in the performance of various conventional learning methods that assume the data distribution to be balanced. The class imbalance problem corresponds to dealing with the situation where one class massively outnumbers the other. The imbalance between majority and minority would lead machine learning to be biased and produce unreliable outcomes if the imbalanced data is used directly. There has been increasing interest in this research area and a number of algorithms have been developed. However, independent evaluation of the algorithms is limited. This paper aims at evaluating the performance of five representative data sampling methods namely SMOTE, ADASYN, BorderlineSMOTE, SMOTETomek and RUSBoost that deal with class imbalance problems. A comparative study is conducted and the performance of each method is critically analysed in terms of assessment metrics. © 2013 Springer-Verlag.

What is the learning in games-based learning?

This chapter explores the nature of “learning” in games-based learning and the cognitive and motivational processes that might underpin that learning by drawing on psychological theories and perspectives. Firstly, changing conceptions of learning over the last few decades are reviewed. This is described in relation to the changes in formal learning theories and connections made between learning theory and GBL. Secondly, the chapter reviews empirical research on the learning outcomes that have been identified for GBL, with specific focus on cognitive benefits, school attainment, collaborative working, and the motivational and engaging appeal of games. Finally, an overview of the dominant theoretical perspectives/findings mostly associated with GBL is presented in an attempt to broaden understanding of the potential for GBL in the classroom.

Assessment matters-mentors need support in their role

BACKGROUND:
In the UK the Nursing and Midwifery Council (NMC) standards to support learning and assessment in practice state that mentors are responsible and accountable for the assessment of pre-registration nursing students in practice. This study was undertaken to explore mentors' experience of assessing nursing students in practice post implementation of the NMC standards.
METHOD:
Five focus groups were conducted with mentors (N=35) who had assessed adult pre-registration nursing students in the previous 12 months. The focus groups were recorded, transcribed and analysed to generate categories.
RESULTS:
Five categories were identified from the data: Changing roles and responsibilities; Exploring the past to understand the present; Just knowing; The odds; Time to mentor. The findings highlighted that mentors were aware of their role and responsibility for the assessment of students in practice. However, many felt this was a new responsibility and role in which they lacked experience. Some existing mentors felt that they may not have had the necessary preparation to effectively assess students in practice and identified their need for support.
CONCLUSION:
Given that mentors are expected to be competent assessors of students in practice and protect the public through gate-keeping professional registration, this study suggests that serious consideration should be given to how mentors are prepared and the ongoing support and education they receive in assessment.

Time Efficient Assessment and Feedback Methods for Large Computer-Aided-Design Cohorts

Timely and individualized feedback on coursework is desirable from a student perspective as it facilitates formative development and encourages reflective learning practice. Faculty however are faced with a significant and potentially time consuming challenge when teaching larger cohorts if they are to provide feedback which is timely, individualized and detailed. Additionally, for subjects which assess non-traditional submissions, such as Computer-Aided-Design (CAD), the methods for assessment and feedback tend not to be so well developed or optimized. Issues can also arise over the consistency of the feedback provided. Evaluations of Computer-Assisted feedback in other disciplines (Denton et al, 2008), (Croft et al, 2001) have shown students prefer this method of feedback to traditional “red pen” marking and also that such methods can be more time efficient for faculty.
Herein, approaches are described which make use of technology and additional software tools to speed up, simplify and automate assessment and the provision of feedback for large cohorts of first and second year engineering students studying modules where CAD files are submitted electronically. A range of automated methods are described and compared with more “manual” approaches. Specifically one method uses an application programming interface (API) to interrogate SolidWorks models and extract information into an Excel spreadsheet, which is then used to automatically send feedback emails. Another method describes the use of audio recordings made during model interrogation which reduces the amount of time while increasing the level of detail provided as feedback.
Limitations found with these methods and problems encountered are discussed along with a quantified assessment of time saving efficiencies made.

Patience, Persistence and Pragmatism: Experiences and Lessons Learnt from the Implementation of Clinically Integrated Teaching and Learning of Evidence-Based Health Care - A Qualitative Study

Background

Clinically integrated teaching and learning are regarded as the best options for improving evidence-based healthcare (EBHC) knowledge, skills and attitudes. To inform implementation of such strategies, we assessed experiences and opinions on lessons learnt of those involved in such programmes.

Methods and Findings

We conducted semi-structured interviews with 24 EBHC programme coordinators from around the world, selected through purposive sampling. Following data transcription, a multidisciplinary group of investigators carried out analysis and data interpretation, using thematic content analysis. Successful implementation of clinically integrated teaching and learning of EBHC takes much time. Student learning needs to start in pre-clinical years with consolidation, application and assessment following in clinical years. Learning is supported through partnerships between various types of staff including the core EBHC team, clinical lecturers and clinicians working in the clinical setting. While full integration of EBHC learning into all clinical rotations is considered necessary, this was not always achieved. Critical success factors were pragmatism and readiness to use opportunities for engagement and including EBHC learning in the curriculum; patience; and a critical mass of the right teachers who have EBHC knowledge and skills and are confident in facilitating learning. Role modelling of EBHC within the clinical setting emerged as an important facilitator. The institutional context exerts an important influence; with faculty buy-in, endorsement by institutional leaders, and an EBHC-friendly culture, together with a supportive community of practice, all acting as key enablers. The most common challenges identified were lack of teaching time within the clinical curriculum, misconceptions about EBHC, resistance of staff, lack of confidence of tutors, lack of time, and negative role modelling.

Conclusions

Implementing clinically integrated EBHC curricula requires institutional support, a critical mass of the right teachers and role models in the clinical setting combined with patience, persistence and pragmatism on the part of teachers.

Standardizing the assessment of clinical competence: an overview of intensive care course design.

Rationale for the development of the Certificate in Health Studies: Intensive Care and High Dependency for Adults course developed at Queens University Belfast, Northern Ireland. Structure and content of clinical module reviewed. Clinical assessment strategy discussed. Focus on the utilization of a standardized portfolio, individualized learning contract and objective structured clinical examination (OSCE) to evaluate clinical competence. Evaluation of OSCE as an assessment tool and of the course provision.

Current understanding of learning psychomotor skills and the impact on teaching laparoscopic surgical skills

Key content
- Trainees face many challenges in learning the skill set required to perform laparoscopic surgery.
- The time spent in the operating room has been detrimentally impacted upon since the implementation of the European Working Time Directive. In order to address the deficit, surgical educators have looked to the benefits enjoyed in the aviation and sports industries in using simulation training.

Learning objectives
- To summarise the current understanding of the neuropsychological basis of learning a psychomotor skill.
- To clarify factors that influence the acquisition of these skills.
- To summarise how this information can be used in teaching and assessment of laparoscopic skills.

Ethical issues
- The use of virtual reality simulators may be able to form a part of the aptitude assessment in the selection process, in order to identify trainees with the desired attributes to progress into the training programmes. However, as skill improves with practice, is it ethical to exclude novices with poor initial performance assessment before allowing them the opportunities to improve?

Improving E-Assessment Systems With A focus On Adaptive Feedback And Marking

Institutions involved in the provision of tertiary education across Europe are feeling the pinch. European universities, and other higher education (HE) institutions, must operate in a climate where the pressure of government spending cuts (Garben, 2012) is in stark juxtaposition to the EU’s strategy to drive forward and maintain a growth of student numbers in the sector (eurostat, 2015).

In order to remain competitive, universities and HE institutions are making ever-greater use of electronic assessment (E-Assessment) systems (Chatzigavriil et all, 2015; Ferrell, 2012). These systems are attractive primarily because they offer a cost-effect and scalable approach for assessment. In addition to scalability, they also offer reliability, consistency and impartiality; furthermore, from the perspective of a student they are most popular because they can offer instant feedback (Walet, 2012).

There are disadvantages, though.

First, feedback is often returned to a student immediately on competition of their assessment. While it is possible to disable the instant feedback option (this is often the case during an end of semester exam period when assessment scores must be can be ratified before release), however, this option tends to be a global ‘all on’ or ‘all off’ configuration option which is controlled centrally rather than configurable on a per-assessment basis.

If a formative in-term assessment is to be taken by multiple groups of
students, each at different times, this restriction means that answers to each question will be disclosed to the first group of students undertaking the assessment. As soon as the answers are released “into the wild” the academic integrity of the assessment is lost for subsequent student groups.

Second, the style of feedback provided to a student for each question is often limited to a simple ‘correct’ or ‘incorrect’ indicator. While this type of feedback has its place, it often does not provide a student with enough insight to improve their understanding of a topic that they did not answer correctly.

Most E-Assessment systems boast a wide range of question types including Multiple Choice, Multiple Response, Free Text Entry/Text Matching and Numerical questions. The design of these types of questions is often quite restrictive and formulaic, which has a knock-on effect on the quality of feedback that can be provided in each case.

Multiple Choice Questions (MCQs) are most prevalent as they are the most prescriptive and therefore most the straightforward to mark consistently. They are also the most amenable question types, which allow easy provision of meaningful, relevant feedback to each possible outcome chosen.
Text matching questions tend to be more problematic due to their free text entry nature. Common misspellings or case-sensitivity errors can often be accounted for by the software but they are by no means fool proof, as it is very difficult to predict in advance the range of possible variations on an answer that would be considered worthy of marks by a manual marker of a paper based equivalent of the same question.

Numerical questions are similarly restricted. An answer can be checked for accuracy or whether it is within a certain range of the correct answer, but unless it is a special purpose-built mathematical E-Assessment system the system is unlikely to have computational capability and so cannot, for example, account for “method marks” which are commonly awarded in paper-based marking.

From a pedagogical perspective, the importance of providing useful formative feedback to students at a point in their learning when they can benefit from the feedback and put it to use must not be understated (Grieve et all, 2015; Ferrell, 2012).

In this work, we propose a number of software-based solutions, which will overcome the limitations and inflexibilities of existing E-Assessment systems.

Learning from Outdoor Webcams: Surveillance of Physical Activity Across Environments

Abstract
Publicly available, outdoor webcams continuously view the world and share images. These cameras include traffic cams, campus cams, ski-resort cams, etc. The Archive of Many Outdoor Scenes (AMOS) is a project aiming to geolocate, annotate, archive, and visualize these cameras and images to serve as a resource for a wide variety of scientific applications. The AMOS dataset has archived over 750 million images of outdoor environments from 27,000 webcams since 2006. Our goal is to utilize the AMOS image dataset and crowdsourcing to develop reliable and valid tools to improve physical activity assessment via online, outdoor webcam capture of global physical activity patterns and urban built environment characteristics.
This project’s grand scale-up of capturing physical activity patterns and built environments is a methodological step forward in advancing a real-time, non-labor intensive assessment using webcams, crowdsourcing, and eventually machine learning. The combined use of webcams capturing outdoor scenes every 30 min and crowdsources providing the labor of annotating the scenes allows for accelerated public health surveillance related to physical activity across numerous built environments. The ultimate goal of this public health and computer vision collaboration is to develop machine learning algorithms that will automatically identify and calculate physical activity patterns.

Learning outcomes tracking system (L.O.T.S):seeing student progress as term progresses

At QUB we have constructed a system that allows students to self-assess their capability on the fine grained learning outcomes for a module and to update their record as the term progresses. In the system each of the learning outcomes are linked to the relevant teaching session (lectures and labs) and to [online] resources that students can access at any time. Students can structure their own learning experience to their needs to attain the learning outcomes. The system keeps a history of the student’s record, allowing the lecturer to observe how the students’ abilities progress over the term and to compare it to assessment results. The system also keeps of any of the resource links that student has clicked on.

An evaluation of the Socrative (personal response system) app for increasing student engagement and learning in an undergraduate psychology curriculum.

Personal response systems using hardware such as 'clickers' have been around for some time, however their use is often restricted to multiple choice questions (MCQs) and they are therefore used as a summative assessment tool for the individual student. More recent innovations such as 'Socrative' have removed the need for specialist hardware, instead utilising web-based technology and devices common to students, such as smartphones, tablets and laptops. While improving the potential for use in larger classrooms, this also creates the opportunity to pose more engaging open-response questions to students who can 'text in' their thoughts on questions posed in class. This poster will present two applications of the Socrative system in an undergraduate psychology curriculum which aimed to encourage interactive engagement with course content using real-time student responses and lecturer feedback. Data is currently being collected and result will be presented at the conference.
The first application used Socrative to pose MCQs at the end of two modules (a level one Statistics module and level two Individual Differences Psychology module, class size N≈100), with the intention of helping students assess their knowledge of the course. They were asked to rate their self-perceived knowledge of the course on a five-point Likert scale before and after completing the MCQs, as well as their views on the value of the revision session and any issues that had with using the app. The online MCQs remained open between the lecture and the exam, allowing students to revisit the questions at any time during their revision.
This poster will present data regarding the usefulness of the revision MCQs, the metacognitive effect of the MCQs on student's judgements of learning (pre vs post MCQ testing), as well as student engagement with the MCQs between the revision session and the examination. Student opinions on the use of the Socrative system in class will also be discussed.
The second application used Socrative to facilitate a flipped classroom lecture on a level two 'Conceptual Issues in Psychology' module, class size N≈100). The content of this module requires students to think critically about historical and contemporary conceptual issues in psychology and the philosophy of science. Students traditionally struggle with this module due to the emphasis on critical thinking skills, rather than simply the retention of concrete knowledge. To prepare students for the written examination, a flipped classroom lecture was held at the end of the semester. Students were asked to revise their knowledge of a particular area of Psychology by assigned reading, and were told that the flipped lecture would involve them thinking critically about the conceptual issues found in this area. They were informed that questions would be posed by the lecturer in class, and that they would be asked to post their thoughts using the Socrative app for a class discussion. The level of preparation students engaged in for the flipped lecture was measured, as well as qualitative opinions on the usefulness of the session. This poster will discuss the level of student engagement with the flipped lecture, both in terms of preparation for the lecture, and engagement with questions posed during the lecture, as well as the lecturer's experience in facilitating the flipped classroom using the Socrative platform.