Work-based assessment : qualitative perspectives of novice nutrition and dietetics educators

Background The assessment of competence for health professionals including nutrition and dietetics professionals in work-based settings is challenging. The present study aimed to explore the experiences of educators involved in the assessment of nutrition and dietetics students in the practice setting and to identify barriers and enablers to effective assessment. Methods A qualitative research approach using in-depth interviews was employed with a convenience sample of inexperienced dietitian assessors. Interviews explored assessment practices and challenges. Data were analysed using a thematic approach within a phenomenological framework. Twelve relatively inexperienced practice educators were purposefully sampled to take part in the present study. Results Three themes emerged from these data. (i) Student learning and thus assessment is hindered by a number of barriers, including workload demands and case-mix. Some workplaces are challenged to provide appropriate learning opportunities and environment. Adequate support for placement educators from the university, managers and their peers and planning are enablers to effective assessment. (ii) The role of the assessor and their relationship with students impacts on competence assessment. (iii) There is a lack of clarity in the tasks and responsibilities of competency-based assessment. Conclusions The present study provides perspectives on barriers and enablers to effective assessment. It highlights the importance of reflective practice and feedback in assessment practices that are synonymous with evidence from other disciplines, which can be used to better support a work-based competency assessment of student performance.

Learner-focused evaluation cycles : facilitating learning using feedforward, concurrent and feedback evaluation

There is a growing trend to offer students learning opportunities that are flexible, innovative and engaging. As educators embrace student-centred agile teaching and learning methodologies, which require continuous reflection and adaptation, the need to evaluate students’ learning in a timely manner has become more pressing. Conventional evaluation surveys currently dominate the evaluation landscape internationally, despite recognition that they are insufficient to effectively evaluate curriculum and teaching quality. Surveys often: (1) fail to address the issues for which educators need feedback, (2) constrain student voice, (3) have low response rates and (4) occur too late to benefit current students. Consequently, this paper explores principles of effective feedback to propose a framework for learner-focused evaluation. We apply a three-stage control model, involving feedforward, concurrent and feedback evaluation, to investigate the intersection of assessment and evaluation in agile learning environments. We conclude that learner-focused evaluation cycles can be used to guide action so that evaluation is not undertaken simply for the benefit of future offerings, but rather to benefit current students by allowing ‘real-time’ learning activities to be adapted in the moment. As a result, students become co-producers of learning and evaluation becomes a meaningful, responsive dialogue between students and their instructors.

Time, place and space – student group collaboration using Google Drive

Virtual working environments are intrinsic to the contemporary workplace and collaborative skills are a vital graduate capability. To develop students’ collaborative skills, first year medical laboratory science students undertake a group poster project, based on a blended learning model. Learning is scaffolded in lectures, workshops in collaborative learning spaces, practitioner mentoring sessions, and online resources. Google Drive provides an online collaborative space for students to realise tangible outcomes from this learning. A Google Drive document is created for each group and shared with members. In this space, students assign tasks and plan workflow, share research, progressively develop poster content, reflect and comment on peer contributions and use the messaging functions to ‘talk’ to group members. This provides a readily accessible, transparent record of group work, crucial in peer assessment, and a communication channel for group members and the lecturer, who can support groups if required. This knowledge creation space also augments productivity and effectiveness of face-to-face collaboration. As members are randomly allocated to groups and are often of diverse backgrounds and unknown to each other, resilience is built as students navigate the uncertainties and complexities of group dynamics, learning to focus on the goal of the team task as they constructively and professionally engage in team dialogue. Students are responsible and accountable for individual and group work. The use of Google Drive was evaluated in a survey including Likert scale and open ended qualitative questions. Statistical analysis was carried out. Results show students (79%) valued the inclusion of online space in collaborative work and highly appreciated (78%) the flexibility provided by Google Drive, while recognising the need for improved notification functionality. Teaching staff recognised the advantages in monitoring and moderating collaborative group work, and the transformational progression in student collaborative as well as technological skill acquisition, including professional dialogue.

The right to a voice and the fight to be heard: The experience of being an ESL user in Australia

This paper focuses on the fundamental right to be heard, that is, the right to have one’s voice heard and listened to – to impose reception (Bourdieu, 1977). It focuses on the ways that non-mainstream English is heard and received in Australia, where despite public policy initiatives around equal opportunity, language continues to socially disadvantage people (Burridge & Mulder, 1998). English is the language of the mainstream and most people are monolingually English (Ozolins, 1993). English has no official status yet it remains dominant and its centrality is rarely challenged (Smolicz, 1995). This paper takes the position that the lack of language engagement in mainstream Australia leads to linguistic desensitisation. Writing in the US context where English is also the unofficial norm, Lippi-Green (1997) maintains that discrimination based on speech features or accent is commonly accepted and widely perceived as appropriate. In Australia, non-standard forms of English are often disparaged or devalued because they do not conform to the ‘standard’ (Burridge & Mulder, 1998). This paper argues that talk cannot be taken for granted: ‘spoken voices’ are critical tools for representing the self and negotiating and manifesting legitimacy within social groups (Miller, 2003). In multicultural, multilingual countries like Australia, the impact of the spoken voice, its message and how it is heard are critical tools for people seeking settlement, inclusion and access to facilities and services. Too often these rights are denied because of the way a person sounds. This paper reports a study conducted with a group that has been particularly vulnerable to ongoing ‘panics’ about language – international students. International education is the third largest revenue source for Australia (AEI, 2010) but has been beset by concerns from academics (Auditor-General, 2002) and the media about student language levels and falling work standards (e.g. Livingstone, 2004). Much of the focus has been high-stakes writing but with the ascendancy of project work in university assessment and the increasing emphasis on oracy, there is a call to recognise the salience of talk, especially among students using English as a second language (ESL) (Kettle & May, 2012). The study investigated the experiences of six international students in a Master of Education course at a large metropolitan university. It utilised data from student interviews, classroom observations, course materials, university policy documents and media reports to examine the ways that speaking and being heard impacted on the students’ learning and legitimacy in the course. The analysis drew on Fairclough’s (2003) model of the dialectical-relational Critical Discourse Analysis (CDA) to analyse the linguistic, discursive and social relations between the data texts and their conditions of production and interpretation, including the wider socio-political discourses on English, language difference, and second language use. The interests of the study were if and how discourses of marginalisation and discrimination manifested and if and how students recognised and responded to them pragmatically. Also how they juxtaposed with and/or contradicted the official rhetoric about diversity and inclusion. The underpinning rationale was that international students’ experiences can provide insights into the hidden politics and practices of being heard and afforded speaking rights as a second language speaker in Australia.

Writing to learn and learning to write across the disciplines: Peer-to-peer writing in introductory-level MOOCs

© Comer, Clark, Canelas.This study aimed to evaluate how peer-to-peer interactions through writing impact student learning in introductory-level massive open online courses (MOOCs) across disciplines. This article presents the results of a qualitative coding analysis of peer-to-peer interactions in two introductory level MOOCs: English Composition I: Achieving Expertise and Introduction to Chemistry. Results indicate that peer-to-peer interactions in writing through the forums and through peer assessment enhance learner understanding, link to course learning objectives, and generally contribute positively to the learning environment. Moreover, because forum interactions and peer review occur in written form, our research contributes to open distance learning (ODL) scholarship by highlighting the importance of writing to learn as a significant pedagogical practice that should be encouraged more in MOOCs across disciplines.

Students’ engagement in school and peer relations: A literature review

Este E-Book reúne um conjunto de investigações apresentadas no “I Congresso Internacional Envolvimento dos Alunos na Escola: Perspetivas da Psicologia e Educação” (ICIEAE), organizado no âmbito do “Projeto PTDC/CPE-CED/114362/2009 - Envolvimento dos Alunos na Escola: Diferenciação e Promoção” (EAE-DP), financiado pela Fundação para a Ciência e a Tecnologia (FCT).

Design, development and assessment of the Java Intelligent Tutoring System

The "Java Intelligent Tutoring System" (JITS) research project focused on designing, constructing, and determining the effectiveness of an Intelligent Tutoring System for beginner Java programming students at the postsecondary level. The participants in this research were students in the School of Applied Computing and Engineering Sciences at Sheridan College. This research involved consistently gathering input from students and instructors using JITS as it developed. The cyclic process involving designing, developing, testing, and refinement was used for the construction of JITS to ensure that it adequately meets the needs of students and instructors. The second objective in this dissertation determined the effectiveness of learning within this environment. The main findings indicate that JITS is a richly interactive ITS that engages students on Java programming problems. JITS is equipped with a sophisticated personalized feedback mechanism that models and supports each student in his/her learning style. The assessment component involved 2 main quantitative experiments to determine the effectiveness of JITS in terms of student performance. In both experiments it was determined that a statistically significant difference was achieved between the control group and the experimental group (i.e., JITS group). The main effect for Test (i.e., pre- and postiest), F( l , 35) == 119.43,p < .001, was qualified by a Test by Group interaction, F( l , 35) == 4.98,p < .05, and a Test by Time interaction, F( l , 35) == 43.82, p < .001. Similar findings were found for the second experiment; Test by Group interaction revealed F( 1 , 92) == 5.36, p < .025. In both experiments the JITS groups outperformed the corresponding control groups at posttest.

Writing and science connections : integrated instruction and assessment

Learning to write is a daunting task for many young children. The purpose of this study was to examine the impact of a combined approach to writing instruction and assessment on the writing performance of students in two grade 3 classes. Five forms and traits of writing were purposefully connected during writing lessons while exhibiting links to the four strands of the grade 3 Ontario science curriculum. Students then had opportunities to engage in the writing process and to self-assess their compositions using either student-developed (experimental group/teacher-researcher's class) or teachercreated (control group/teacher-participant's class) rubrics. Paired samples t-tests revealed that both the experimental and control groups exhibited statistically significant growth from pretest to posttest on all five integrated writing units. Independent samples t-tests showed that the experimental group outperformed the control group on the persuasive + sentence fluency and procedure + word choice writing tasks. Pearson product-moment correlation r tests revealed significant correlations between the experimental group and the teacher-researcher on the recount + ideas and report + organization tasks, while students in the control group showed significant correlations with the teacher-researcher on the narrative + voice and procedure + word choice tasks. Significant correlations between the control group and the teacher-participant were evident on the persuasive + sentence fluency and procedure + word choice tasks. Qualitative analyses revealed five themes that highlighted how students' self-assessments and reflections can be used to guide teachers in their instructional decision making. These findings suggest that educators should adopt an integrated writing program in their classrooms, while working with students to create and utilize purposeful writing assessment tools.

Children's and adolescent's self - assessment of metabolic control versus professional judgment: a cross-sectional retrospective and prospective cohort study

BACKGROUND Morbidity and mortality in T1DM depend on metabolic control, which is assessed by HbA1c measurements every 3-4 months. Patients' self-perception of glycemic control depends on daily blood glucose monitoring. Little is known about the congruence of patients' and professionals' perception of metabolic control in T1DM. OBJECTIVE To assess the actual patients' self-perception and objective assessment (HbA1c) of metabolic control in T1DM children and adolescents and to investigate the possible factors involved in any difference. METHODS Patients with T1DM aged 8 - 18 years were recruited in a cross-sectional, retrospective and prospective cohort study. Data collection consisted of clinical details, measured HbA1c, self-monitored blood glucose values and questionnaires assessing self and professionals' judgment of metabolic control. RESULTS 91 patients participated. Mean HbA1c was 8.03%. HbA1c was higher in patients with a diabetes duration > 2 years (p = 0.025) and in patients of lower socioeconomic level (p = 0.032). No significant correlation was found for self-perception of metabolic control in well and poorly controlled patients. We found a trend towards false-positive memory of the last HbA1c in patients with a HbA1c > 8.5% (p = 0.069) but no difference in patients' knowledge on target HbA1c between well and poorly controlled patients. CONCLUSIONS T1DM patients are aware of a target HbA1c representing good metabolic control. Ill controlled patients appear to have a poorer recollection of their HbA1c. Self-perception of actual metabolic control is similar in well and poorly controlled T1DM children and adolescents. Therefore, professionals should pay special attention that ill controlled T1DM patients perceive their HbA1c correctly.

Survey on student motivation and approval of educational environment at Universidad Politécnica de Madrid within the European Higher Education area implementation

A recent study elaborated by Vicerrectorado de Ordenación Académica y Planificación Estratégica of Technical University of Madrid (UPM) defines the satisfaction of the university student body as "the response that the University offers to the expectations and demands of service of the students, considered in a general way ". Besides an indicator of academic and institutional insertion of the student, the assessment of student engagement allows us to adapt the academic offer and the extension services of the University to the real needs of the students. The process of convergence towards the European Higher Education Area (EHEA) raises the need to form in competitions, that is to say, of developing in our students capacities and knowledge beyond the purely theoretical-practical thing. Therefore, the perception and experience of the educational process and environment by the students is an important issue to be addressed to accomplish their expectations and achieve a curriculum accordingly to EHEA expectations. The present study aims to explore the student motivation and approval of the educational environment at the UPM. To this end a total of 97 students enrolled in the undergraduate program of Civil Engineering, Computer Engineering and Agronomic Engineering at UPM were surveyed. The survey consisted of 40 questions divided in three blocks. The first one of 20 questions of personal character in that they were gathering, besides the sex and the age, the degree of fulfilment, implication and dedication with the institution and the academic tasks. In the second block we identify 10 questions related to the perception of the student on the teaching quality, and finally a block of 10 questions regarding the Bologna Process. The students personal motivation was moderately high, with a score of 3.6 (all scores are provided on a 5-point scale), being the most valuable items obtaining a university degree (4,3) and the friendship between students (4,2). Any significant difference was shown between sexes (P=0.23) since the averages for this block of questions were of 3.7±0.3 and 3.5±0.4 for women and men respectively. The students are moderately satisfied with their graduate studies with an average score of 3,2, being the questions that reflect a minor satisfaction the research profile of the teachers (2,8) and the organization of the Schools (2,9). The best valued questions are related to the usefulness and quality of the degrees, with 3,5 and 3,4 respectively, and to the interest of the courses within the degree (3,4). For sexes, the results of this block of questions are similar (3.1±0.3 and 3.2±0.3 for men and women respectively=0.79). Also, there were no differences (P=0.39) between the students who arrange work and studies or do not work (3.1±0.2 and 3.2±0.3 respectively). In conclusion, students at UPM present an acceptable degree of motivation and satisfaction with regard to the studies and services that offer their respective Schools. Both characteristics receive the same value both for men and for women and so much for students who arrange work and studies as for those who devote themselves only to studying. In a significant way, students who are more engaged and are in-class attendants present the major degree of satisfaction.Overall, there is a great lack of information regarding the Bologna Process. In fact to the majority, they would like to know more on what it is, what it means and what changes will involve its implementation.

Digital Classics Outside the Echo-Chamber : Teaching, Knowledge Exchange and Public Engagement /

The international perspectives on these issues are especially valuable in an increasingly connected, but still institutionally and administratively diverse world. The research addressed in several chapters in this volume includes issues around technical standards bodies like EpiDoc and the TEI, engaging with ways these standards are implemented, documented, taught, used in the process of transcribing and annotating texts, and used to generate publications and as the basis for advanced textual or corpus research. Other chapters focus on various aspects of philological research and content creation, including collaborative or community driven efforts, and the issues surrounding editorial oversight, curation, maintenance and sustainability of these resources. Research into the ancient languages and linguistics, in particular Greek, and the language teaching that is a staple of our discipline, are also discussed in several chapters, in particular for ways in which advanced research methods can lead into language technologies and vice versa and ways in which the skills around teaching can be used for public engagement, and vice versa. A common thread through much of the volume is the importance of open access publication or open source development and distribution of texts, materials, tools and standards, both because of the public good provided by such models (circulating materials often already paid for out of the public purse), and the ability to reach non-standard audiences, those who cannot access rich university libraries or afford expensive print volumes. Linked Open Data is another technology that results in wide and free distribution of structured information both within and outside academic circles, and several chapters present academic work that includes ontologies and RDF, either as a direct research output or as essential part of the communication and knowledge representation. Several chapters focus not on the literary and philological side of classics, but on the study of cultural heritage, archaeology, and the material supports on which original textual and artistic material are engraved or otherwise inscribed, addressing both the capture and analysis of artefacts in both 2D and 3D, the representation of data through archaeological standards, and the importance of sharing information and expertise between the several domains both within and without academia that study, record and conserve ancient objects. Almost without exception, the authors reflect on the issues of interdisciplinarity and collaboration, the relationship between their research practice and teaching and/or communication with a wider public, and the importance of the role of the academic researcher in contemporary society and in the context of cutting edge technologies. How research is communicated in a world of instant- access blogging and 140-character micromessaging, and how our expectations of the media affect not only how we publish but how we conduct our research, are questions about which all scholars need to be aware and self-critical.

Student response to a pub quiz style first year psychology assessment

The deterioration in staff-student ratios in UK higher education has had a disproportionate impact on assessment and feedback, meaning that contemporary students may have fewer assessments and much less feedback than a generation ago (Gibbs, 2006). Early use of a quiz assessment may offer a blend of social benefits (social comparison, shared problem solving leading to engagement, belonging and continuation), academic benefits (early formative assessment, immediate feedback) and administrative benefits (on-the-spot verbal marking and feedback to 230 students simultaneously). This study sought student views on the acceptability and contribution to learning of the quiz. Social benefits were apparent but difficulties in creating questions to elicit deeper reasoning and problem solving are discussed and the quiz had limited pedagogic value in the eyes of participants. The use of assertion-reason questions are considered as a way of taking the table quiz to a higher level and extending its pedagogic value.

Self-other rating discrepancy and personality antecedents

This study explored individual difference factors to help explain the discrepancy that has been found to exist between self and other ratings in prior research. Particularly, personality characteristics of the self-rater were researched in the current study as a potential antecedent for self-other rating agreement. Self, peer, and supervisor ratings were provided for global performance as well as five competencies specific to the organization being examined. Four rating tendency categories, over-raters, under-raters, in-agreement (good), and in-agreement (poor), established in research by Atwater and Yammarino were used as the basis of the current research. The sample for rating comparisons within the current study consisted of 283 self and supervisor dyads and 275 for self and peer dyads from a large financial organization. Measures included a custom multi-rater performance instrument and the personality survey instrument, ASSESS, which measures 20 specific personality characteristics. MANCOVAs were then performed on this data to examine if specific personality characteristics significantly distinguished the four rating tendency groups. Examination of all personality dimensions and overall performance uncovered significant findings among rating groups for self-supervisor rating comparisons but not for self-peer rating comparisons. Examination of specific personality dimensions for self-supervisory ratings group comparisons and overall performance showed Detail Interest to be an important characteristic among the hypothesized variables. For self-supervisor rating comparisons and specific competencies, support was found for the hypothesized personality dimension of Fact-based Thinking which distinguished the four rating groups for the competency, Builds Relationships. For both self-supervisor and self-peer rating comparisons, the competencies, Builds Relationships and Leads in a Learning Environment, were found to have significant relationship with several personality characteristics, however, these relationships were not consistent with the hypotheses in the current study. Several unhypothesized personality dimensions were also found to distinguish rating groups for both self-supervisor and self-peer comparisons on overall performance and various competencies. Results of the current study hold implications for the training and development session that occur after a 360-degree evaluation process. Particularly, it is suggested that feedback sessions may be designed according to particular rating tendencies to maximize the interpretation, acceptance and use of evaluation information. ^

Toward an understanding of how individuals overcome challenges to development: A phenomenological study of an inner -city student's attitude toward academic orientation and life

The purpose of this qualitative case study was to gain an understanding of the phenomenon of academic orientation by seeking the insights into an inner-city Haitian-American middle school student's attitudes and world view toward education and life. A phenomenological approach was used in order to explore the way in which Cindy, a minority student, gives meaning to her lived-experiences in terms of her desire to meet academic expectations and her ability to overcome social adversity and/or other risk factors.^ The study attempted to answer the following two research questions: (1) What provides the focus for Cindy's (the subject's) approach to her school work and/or life? (2) What are the processes that give meaning and direction to academic orientation and life for Cindy? In-depth interviewing was the primary method of data collection. In addition, journal and sketchbook entries and school district records were used and classroom observations made.^ The nature of the study to understand lived-experience facilitated the use of the case study method and a phenomenological method of description. Data analysis was conducted by means of an adapted form of the constant comparative approach. Patterns in the data which emerged were coded and categorized according to underlying generative themes. Phenomenological reflection and analysis were used to grasp the experiential structures of Cindy's experience. The following textural themes were identified and confirmed to be essential themes to Cindy's experience: personal challenge to do her best, personal challenge to want to learn, having a sense of determination, being able to think for self, having a disposition to like self, achieving self-respect through performance, seeing a need to help others, being intrinsically motivated, being an independent learner, attending more to academic pressure and less to peer pressure, having motivational catalysts in her life, learning and support opportunities, and having a self-culture. Using Mahrer's humanistic theory of experiencing, Cindy's development was interpreted in terms of her progression through a sequence of developmental plateaus: externalized self, internalized self, and integrating and actualizing self.^ The findings of this study were that Cindy's desire to meet academic expectations is guided by a meaning construction internal frame of reference. High expectations of self in conjunction with other protective factors found in Cindy's home and school environments were also found to be linked to her educational resilience and success. Cindy's lived-experiences were also found to be related to Mahrer's theory of human development. In addition, it was concluded that "minority" students do not all fit into social categories and labels. ^

Role of assessment components and recent adverse outcomes in risk estimation and prediction: use of the Short Term Assessment of Risk and Treatability (START) in an adult secure inpatient mental health service

The Short Term Assessment of Risk and Treatability is a structured judgement tool used to inform risk estimation for multiple adverse outcomes. In research, risk estimates outperform the tool's strength and vulnerability scales for violence prediction. Little is known about what its’component parts contribute to the assignment of risk estimates and how those estimates fare in prediction of non-violent adverse outcomes compared with the structured components. START assessment and outcomes data from a secure mental health service (N=84) was collected. Binomial and multinomial regression analyses determined the contribution of selected elements of the START structured domain and recent adverse risk events to risk estimates and outcomes prediction for violence, self-harm/suicidality, victimisation, and self-neglect. START vulnerabilities and lifetime history of violence, predicted the violence risk estimate; self-harm and victimisation estimates were predicted only by corresponding recent adverse events. Recent adverse events uniquely predicted all corresponding outcomes, with the exception of self-neglect which was predicted by the strength scale. Only for victimisation did the risk estimate outperform prediction based on the START components and recent adverse events. In the absence of recent corresponding risk behaviour, restrictions imposed on the basis of START-informed risk estimates could be unwarranted and may be unethical.