Effects of training peer tutors in content knowledge versus tutoring skills on giving feedback to help tutees’ complex tasks

This study aims to investigate the effectiveness of training tutors in content knowledge of a particular domain versus training them in tutoring skills of pedagogical knowledge when tutoring on a complex tutee task. Forty-seven tutor-tutee pairs of fourth year secondary school students were created and assigned to one of two treatments. Twenty-two tutors received training in content knowledge and the other twenty-five tutors in tutoring skills. Tutors formulated written feedback immediately after the training. Tutees first interpreted the tutor feedback and then used it to revise their research questions. The results showed that tutors trained in tutoring skills formulated more effective feedback than tutors trained in content knowledge. In addition, tutees helped by tutoring-skills tutors found the feedback more motivating than those helped by content- knowledge tutors. However, no differences were found in tutee performance on revision. The findings are discussed in terms of the set-up of this study and implications for improving the effectiveness of peer tutoring.

What's in it for us? A case study research about micro sense making processes through the eyes of front line employees, during culture change implementation

Abstract How employees make sense of change is a very complex process. Recently, academics have neglected to research sense making activities in a micro culture implementation context, through the eyes of front line employees. In contrast to a macro view, a micro perspective limits researchers to only look at an individual, departmental or group level. By doing so, we can zoom in on the details of sense making processes that employees use in their daily work life. A macro (organisational) view is based on the notion that there is a general integrated culture that can be found in all organisational units and departments. It is assumed that culture can be researched by using the entire organisation as one single research entity. This thesis challenges this assumption. In case of planned change it is usually the management community who are in charge of the change intervention. Because of their formal hierarchical position, they have the power to abort or initiate change programs. It is perhaps therefore that researchers tend to be focused on the management community rather than on lower level organisational members, such as front line employees. Apart from the micro view, scholars also neglected to research culture change implementation through the eyes of front line employees. This thesis is an attempt to fill these two gaps that currently exists in academic change management publications. The main research question is therefore: From a micro point of view how do front-line employees make sense of the impact of culture change, during the implementation phase? This thesis starts with a literature review which exposes the two main gaps. The most important outcome of this review is that only 2% of the research articles dealt with culture implementation, through the eyes of front line employees. A conceptual research model is built on the integrated sense making theory of Weber and Manning (2001) and the micro variables of Raelin and Cataldo (2011). These theories emphasize elements of sense making in a daily working context. It is likely that front line employees can identify themselves with research elements such as tasks, skills practices, involvement and behaviour. Front line employees were selected, because as lower level organisational members they are usually the change recipients. They are further away from the change initiating scene (usually the management of an organisation) and form a potential sense making ‘hotspot’ that could provide new academic insights. In order to carry out the primary research, two case organisations were selected in the leisure industry. A participative case study research method was chosen. This meant that the researcher worked in the concerning departments of the case organisations. The goal was to observe and interview front line employees, while they were performing their jobs. The most important advantage of this approach is that the researcher temporarily becomes one with the organisation and is therefore able to acquire both formal and informal narratives that front line employees use during sense making activities. It was found that front line employees make sense of organisational change by using a practical approach. They make sense of the change program by carrying out new tasks, developing new skills and sharing best practices. The most noticeable conclusion was that sense making activities predominantly take place at an individual level in relation to change acceptance. Organisational members tend to create a mental equation in order to weigh the advantages against the disadvantages. They evaluate whether the concerning change program is beneficial to them or not. For future research a sense making scheme model is suggested that is based on two methods: an introspection and an action method.

A First Step Towards Synthesising Rubrics and Video for the Formative Assessment of Complex Skills

Abstract. The performance objectives used for the formative assessment of com- plex skills are generally set through text-based analytic rubrics[1]. Moreover, video modeling examples are a widely applied method of observational learning, providing students with context-rich modeling examples of complex skills that act as an analogy for problem solving [1]. The purpose of this theoretical paper is to synthesize the components of video modeling and rubrics to support the formative assessment of complex skills. Based on theory, we argue that application of the developed Video Enhanced Rubrics (VER) fosters learners’ development of mental models, quality of provided feedback by various actors and finally, the learners mastery of complex skills.

Enhancing public speaking skills - an evaluation of the Presentation Trainer in the wild

Full paper presented at EC-TEL 2016

Viewbrics: Formative Assessment of Complex Skills with Video-Enhanced Rubrics (VER) in Dutch Secondary Education

To learn complex skills, like collaboration, learners need to acquire a concrete and consistent mental model of what it means to master this skill. If learners know their current mastery level and know their targeted mastery level, they can better determine their subsequent learning activities. Rubrics support learners in judging their skill performance as they provide textual descriptions of skills’ mastery levels with performance indicators for all constituent subskills. However, text-based rubrics have a limited capacity to support the formation of mental models with contextualized, time-related and observable behavioral aspects of a complex skill. This paper outlines the design of a study that intends to investigate the effect of rubrics with video modelling examples compared to text-based rubrics on skills acquisition and feedback provisioning. The hypothesis is that video-enhanced rubrics, compared to text based rubrics, will improve mental model formation of a complex skill and improve the feedback quality a learner receives (from e.g. teachers, peers) while practicing a skill, hence positively effecting final mastery of a skill.

A First Step Towards Video Enhanced Rubrics for the Formative Assessment of Complex Skills : A literature review and a plea

Paper presentation at the TEA2016 conference, Tallinn, Estonia.