De "quantofrenias" y "artefactos" o la "doxa" en el reino de la "episteme" : Uso y abusos de las escalas de estimaciones sumadas en la evaluación del desempeño docente en contextos de educación superior universitaria

Este trabajo presenta una reflexión metodológica relativa al uso de escalas de estimaciones sumadas o Likert en la evaluación del desempeño docente en el contexto universitario. Se presentan antecedentes en el marco de las prescripciones técnicas para este tipo de escalamientos, así como un conjunto de observaciones referidas a la pertinencia de su aplicación con fines evaluativos y a sus limitaciones en tanto herramienta para la generación de conocimiento. Se concluye que la escala de Likert puede ser utilizada en contexto evaluativos, atendiendo al conjunto de requerimientos ligados a su aplicación y tratamiento analítico-interpretativo, reconociendo los problemas insalvables que presenta, de manera de sopesar y relativizar la construcción del dato numeral. De este modo, se hace explícita la crítica al carácter "quantofrénico" y "artefactual" que acompaña su aplicación, y que contradictoriamente se inscribe en un discurso que sitúa la evaluación docente en el marco de políticas de calidad en educación superior

The refractive folio : exegesis and assessment in the open studio

Folio submission is universally regarded as the most appropriate means for measuring a student’s performance in the studio. However, developing meaningful and defensible assessment criteria is persistent challenge for all tertiary art educators. In discipline-based studios, the parameters provided by medium and technique provide useful points of reference for assessing creative performance. But how can student performance be evaluated when there is no discipline-based framework to act as a point of reference? The ‘open’ studio approach to undergraduate teaching presents these and other pedagogical challenges. This paper discusses the innovative approaches to studio-based teaching and assessment at QUT. Vital to the QUT open studio model is the studio rationale – an exegetical document that establishes an individualised theoretical framework through which a student’s understandings can be, in part, evaluated. This paper argues that the exegetical folio effectively reconciles the frequently divergent imperatives of creative, professional and academic skills, while retaining the centrality of the studio as a site for the production of new material, processual and conceptual understandings.

Curriculum design innovation in flexible science teaching

In this paper you will be introduced to a number of principles which can be used to inform good teaching practice and rigorous curriculum design. Principles relate to: * Application of a common sequence of events for how learners learn; * Accommodating different learning styles; * Adopting a purposeful approach to teaching and learning; * Using assessment as a central driving force in the curriculum and as an organising structure leading to coherence of teaching and learning approach; and * The increasing emphasis that is being placed on the development of generic graduate competencies over and above discipline content knowledge. The principles are particularly significant in relation to adult learning. The paper will use three specific applications as illustrations to help you to learn how these principles can be applied. The illustrations are taken from a second year subject in supercomputing that uses scientific case studies. The subject has been developed (with support from Silicon Graphics Inc. and Intel) to be taught entirely via the Internet.

Integration of CENA practice standards for postgraduate emergency nursing assessment

The use of professional competency standards to assess postgraduate nursing student’s clinical performance has been in place since 2004, at the Queensland University of Technology, School of Nursing & Midwifery (SONAM) when the Graduate Certificate in Emergency Nursing degree commenced. Emergency nursing students were assessed in their workplace, using a Clinical Performance Appraisal Tool or CPAT which was based on the Australian College of Critical Care Nurses (ACCCN) Competency Standards. With the subsequent formation of a separate Emergency Nursing Course advisory group in 2007, there was a review of clinical assessment course component. The release of the 2008 CENA revised Practice Standards for the Emergency Nursing Specialist’s, led to the emergency nursing course advisory committee supporting the integration of the CENA practice standards for assessment of emergency nurses in preference to the less relevant ACCCN competency standards. The SONAM emergency nursing study area team commenced the phasing in and progression of the CENA practice standards across the two Graduate Certificate units, and Graduate Diploma and Master of Nursing (emergency) clinical major options in 2009. As some units undertaken in the degree are available to nurses in other disciplines a separate CPAT was devised for the clinical assessments according to speciality context. The team has had to carefully consider how the professional standards are integrated into the teaching and assessment of the unit and not just applied instead of the ACCCN competency standards. Professional standards for the emergency context has also helped tailor course content and learning outcomes to be relevant across a number of emergency nursing contexts in Australia. The assessment of the CPAT is undertaken at the workplace by QUT appointed clinical lecturers. Clinical lecturers need to apply and have suitable postgraduate qualification to undertake the position. The clinical lecturer support role is well established at QUT. The integration of the new CENA practice standards has necessitated a review of the postgraduate assessment of emergency nurses. A clinical lecturer workshop has been organised to review role, scope and how to utilise the new look CENA based CPAT, clinical assessment format.

A model for enhancing assessment and teaching practice at the coalface : Insights from a Fellow-In-Residence engagement program

Australian universities are currently engaging with new governmental policies and regulations that require them to demonstrate enhanced quality and accountability in teaching and research. The development of national academic standards for learning outcomes in higher education is one such instance of this drive for excellence. These discipline-specific standards articulate the minimum, or Threshold Learning Outcomes, to be addressed by higher education institutions so that graduating students can demonstrate their achievement to their institutions, accreditation agencies, and industry recruiters. This impacts not only on the design of Engineering courses (with particular emphasis on pedagogy and assessment), but also on the preparation of academics to engage with these standards and implement them in their day-to-day teaching practice on a micro level. This imperative for enhanced quality and accountability in teaching is also significant at a meso level, for according to the Australian Bureau of Statistics, about 25 per cent of teachers in Australian universities are aged 55 and above and more than 54 per cent are aged 45 and above (ABS, 2006). A number of institutions have undertaken recruitment drives to regenerate and enrich their academic workforce by appointing capacity-building research professors and increasing the numbers of early- and mid-career academics. This nationally driven agenda for quality and accountability in teaching permeates also the micro level of engineering education, since the demand for enhanced academic standards and learning outcomes requires both a strong advocacy for a shift to an authentic, collaborative, outcomes-focused education and the mechanisms to support academics in transforming their professional thinking and practice. Outcomes-focused education means giving greater attention to the ways in which the curriculum design, pedagogy, assessment approaches and teaching activities can most effectively make a positive, verifiable difference to students’ learning. Such education is authentic when it is couched firmly in the realities of learning environments, student and academic staff characteristics, and trustworthy educational research. That education will be richer and more efficient when staff works collaboratively, contributing their knowledge, experience and skills to achieve learning outcomes based on agreed objectives. We know that the school or departmental levels of universities are the most effective loci of changes in approaches to teaching and learning practices in higher education (Knight & Trowler, 2000). Heads of Schools are being increasingly entrusted with more responsibilities - in addition to setting strategic directions and managing the operational and sometimes financial aspects of their school, they are also expected to lead the development and delivery of the teaching, research and other academic activities. Guiding and mentoring individuals and groups of academics is one critical aspect of the Head of School’s role. Yet they do not always have the resources or support to help them mentor staff, especially the more junior academics. In summary, the international trend in undergraduate engineering course accreditation towards the demonstration of attainment of graduate attributes poses new challenges in addressing academic staff development needs and the assessment of learning. This paper will give some insights into the conceptual design, implementation and empirical effectiveness to date, of a Fellow-In-Residence Engagement (FIRE) program. The program is proposed as a model for achieving better engagement of academics with contemporary issues and effectively enhancing their teaching and assessment practices. It will also report on the program’s collaborative approach to working with Heads of Schools to better support academics, especially early-career ones, by utilizing formal and informal mentoring. Further, the paper will discuss possible factors that may assist the achievement of the intended outcomes of such a model, and will examine its contributions to engendering an outcomes-focussed thinking in engineering education.

Creating efficiencies in teaching : where are the right resources when you need them?

Early career engineering academics are encouraged to join and contribute to established research groups at the leading edge of their discipline. This is often facilitated by various staff development and support programs. Given that academics are often appointed primarily on the basis of their research skills and outputs, such an approach is justified and is likely to result in advancing the individual academic’s career. It also enhances their capacity to attract competitive research funding, while contributing to the overall research performance of their institution, with further potential for an increased share of government funding. In contrast, there is much less clarity of direction or availability of support mechanisms for those academics in their role as teachers. Following a general induction to teaching and learning at their institution, they would commonly think about preparing some lecture materials, whether for delivery in a face-to-face or on-line modality. Typically they would look for new references and textbooks to act as a guide for preparing the content. They would probably find out how the course has been taught before, and what laboratory facilities and experiments have been used. In all of these and other related tasks, the majority of newly appointed academics are guided strongly by their own experiences as students, rather than any firm knowledge of pedagogical principles. At a time of increased demands on academics’ time, and high expectations of performance and productivity in both research and teaching, it is essential to examine possible actions to support academics in enhancing their teaching performance in effective and efficient ways. Many resources have been produced over the years in engineering schools around the world, with very high intellectual and monetary costs. In Australia, the last few years have seen a surge in the number of ALTC/OLT projects and fellowships addressing a range of engineering education issues and providing many resources. There are concerns however regarding the extent to which these resources are being effectively utilised. Why are academics still re-inventing the wheel and creating their own version of teaching resources and pedagogical practice? Why do they spend so much of their precious time in such an inefficient way? A symposium examining the above issues was conducted at the AAEE2012 conference, and some pointers to possible responses to the above questions were obtained. These are explored in this paper and supplemented by the responses to a survey of a group of engineering education leaders on some of the aspects of these research questions. The outcomes of the workshop and survey results have been analysed in view of the literature and the ALTC/OLT sponsored learning and teaching projects and resources. Other factors are discussed, including how such resources can be found, how their quality might be evaluated, and how assessment may be appropriately incorporated, again using readily available resources. This study found a strong resonance between resources reuse with work on technology acceptance (Davis, 1989), suggesting that technology adoption models could be used to encourage resource sharing. Efficient use of outstanding learning materials is an enabling approach. The paper provides some insights on the factors affecting the re-use of available resources, and makes some recommendations and suggestions on how the issue of resources re-use might be incorporated in the process of applying and completing engineering education projects.

The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates

Objective: To evaluate the feasibility, reliability and acceptability of the mini clinical evaluation exercise (mini-CEX) for performance assessment among international medical graduates (IMGs). Design, setting and participants: Observational study of 209 patient encounters involving 28 IMGs and 35 examiners at three metropolitan teaching hospitals in New South Wales, Victoria and Queensland, September-December 2006. Main outcome measures: The reliability of the mini-CEX was estimated using generatisability (G) analysis, and its acceptability was evaluated by a written survey of the examiners and IMGs. Results: The G coefficient for eight encounters was 0.88, suggesting that the reliability of the mini-CEX was 0.90 for 10 encounters. Almost half of the IMGs (7/16) and most examiners (14/18) were satisfied with the mini-CEX as a learning tool. Most of the IMGs and examiners enjoyed the immediate feedback, which is a strong component of the tool. Conclusion: The mini-CEX is a reliable tool for performance assessment of IMGs, and is acceptable to and well received by both learners and supervisors.

Current understanding of learning psychomotor skills and the impact on teaching laparoscopic surgical skills

Key content
- Trainees face many challenges in learning the skill set required to perform laparoscopic surgery.
- The time spent in the operating room has been detrimentally impacted upon since the implementation of the European Working Time Directive. In order to address the deficit, surgical educators have looked to the benefits enjoyed in the aviation and sports industries in using simulation training.

Learning objectives
- To summarise the current understanding of the neuropsychological basis of learning a psychomotor skill.
- To clarify factors that influence the acquisition of these skills.
- To summarise how this information can be used in teaching and assessment of laparoscopic skills.

Ethical issues
- The use of virtual reality simulators may be able to form a part of the aptitude assessment in the selection process, in order to identify trainees with the desired attributes to progress into the training programmes. However, as skill improves with practice, is it ethical to exclude novices with poor initial performance assessment before allowing them the opportunities to improve?

A three-stage DEA model to evaluate learning-teaching technical efficiency: Key performance indicators and contextual variables

This study evaluates the technical efficiency of the learning-teaching process in higher education using a three-stage procedure that offers advances in comparison to previous studies and improves the quality of the results. First, it utilizes a multiple stage Data Envelopment Analysis (DEA) with contextual variables. Second, the levels of super efficiency are calculated in order to prioritize the efficiency units. And finally, through sensitivity analysis, the contribution of each key performance indicator (KPI) is established with respect to the efficiency levels without omission of variables. The analytical data was collected from a survey completed by 633 tourism students during the 2011/12, 2012/13 and 2013/14 academic course years. The results suggest that level of satisfaction with the course, diversity of materials and satisfaction with the teacher were the most important factors affecting teaching performance. Furthermore, the effect of the contextual variables was found to be significant.

An online assessment and development tool for business incubators:optimising business-incubation management practice, performance and impact, Project Report

This project is focused on exchanging knowledge between ABS, UKBI and managers of business incubators in the UK. The project relates to exploitation of extant knowledge-base on assessing and improving business incubation management practice and performance and builds on two earlier studies. It addresses a pressing need for assessing and benchmarking business incubation input, process and outcome performance and highlighting best practice. The overarching aim of this project was to obtain proof-of-concept for a business incubation performance assessment and benchmarking online tool, fine-tune it and put it in use by nurturing a community of business incubation management practice, aligned by the resultant tool. The purpose was to offer an appropriate set of measures, in areas identified by relevant research on business incubation performance management and impact as critical, against which: 1.The input and process performance of business incubation management practice can be assessed and benchmarked within the auspices of a community of incubator managers concerned with best practice 2.The outcome performance and impact of business incubators can be assessed longitudinally. As such, the developed online assessment framework is geared towards the needs of researchers, policy makers and practitioners concerned with business incubation performance, added value and impact.

Does video feedback analysis improve CPR performance in phase 5 medical students?

Background The use of simulation in medical education is increasing, with students taught and assessed using simulated patients and manikins. Medical students at Queen’s University of Belfast are taught advanced life support cardiopulmonary resuscitation as part of the undergraduate curriculum. Teaching and feedback in these skills have been developed in Queen’s University with high-fidelity manikins. This study aimed to evaluate the effectiveness of video compared to verbal feedback in assessment of student cardiopulmonary resuscitation performance Methods Final year students participated in this study using a high-fidelity manikin, in the Clinical Skills Centre, Queen’s University Belfast. Cohort A received verbal feedback only on their performance and cohort B received video feedback only. Video analysis using ‘StudioCode’ software was distributed to students. Each group returned for a second scenario and evaluation 4 weeks later. An assessment tool was created for performance assessment, which included individual skill and global score evaluation. Results One hundred thirty eight final year medical students completed the study. 62 % were female and the mean age was 23.9 years. Students having video feedback had significantly greater improvement in overall scores compared to those receiving verbal feedback (p = 0.006, 95 % CI: 2.8–15.8). Individual skills, including ventilation quality and global score were significantly better with video feedback (p = 0.002 and p < 0.001, respectively) when compared with cohort A. There was a positive change in overall score for cohort B from session one to session two (p < 0.001, 95 % CI: 6.3–15.8) indicating video feedback significantly benefited skill retention. In addition, using video feedback showed a significant improvement in the global score (p < 0.001, 95 % CI: 3.3–7.2) and drug administration timing (p = 0.004, 95 % CI: 0.7–3.8) of cohort B participants, from session one to session two. Conclusions There is increased use of simulation in medicine but a paucity of published data comparing feedback methods in cardiopulmonary resuscitation training. Our study shows the use of video feedback when teaching cardiopulmonary resuscitation is more effective than verbal feedback, and enhances skill retention. This is one of the first studies to demonstrate the benefit of video feedback in cardiopulmonary resuscitation teaching.

Orals ain't orals : how instruction and assessment practices affect delivery choices with prepared student oral presentations.

Despite an ostensibly technology-driven society, the ability to communicate orally is still seen as an essential ability for students at school and university, as it is for graduates in the workplace. The need to develop effective oral communication skills is often tied to future work-related tasks. One tangible way that educators have assessed proficiency in this area is through prepared oral presentations. While some use the terms oral communication and oral presentation interchangeably, other writers question the role more formal presentations play in the overall development of oral communication skills. Adding to the discussion, this paper is part of a larger study examining the knowledge and skills students bring into the academy from previous educational experiences. The study examines some of the teaching and assessment methods used in secondary schools to develop oral communication skills through the use of formal oral presentations. Specifically, it will look at assessment models and how these are used as a form of instruction as well as how they contribute to an accurate evaluation of student abilities. The purpose of this paper is to explore key terms and identify tensions between expectations and practice. Placing the emphasis on the ‘oral’ aspect of this form of communication this paper will particularly look at the ‘delivery’ element of the process.

Assessment in tertiary mathematics

There is a growing consensus among many educators that the goals of teaching and learning mathematics are to help students solve real-life problems, participate intelligently in daily affairs, and prepare them for jobs (Gardiner, 1994; Roeber, 1995). These goals suggest that the role of routine procedural skills should be diminished while more emphasis ought to be placed on learners gaining conceptual insights and analytical skills that appear essential in real-life mathematical problem solving (Schoenfeld, 1993; Stenmark, 1989).

Assessment by interview and portfolio in a Graduate School Program

Assessment plays an integral role in teaching and learning in Higher Education and teachers have a strong interest in debates and commentaries on assessment as and for learning. In a one-year graduate entry teacher preparation program, the temptation is to emphasize assessment in an attempt to ensure students “cover” everything as part of a robust preparation for the profession. The risk is that, for students, assessment drives curriculum, and time spent in the completion of assignments is no guarantee of either effective learning or authentic preparation for teaching. Interviews as assessment provide an opportunity for a learning experience as well as an authentic task, since students will shortly be interviewing for employment in a “real world” situation. This paper reports on a project experimenting with interview panels as authentic assessment with pre-service early childhood teachers. At the end of their first semester of study, students enrolled in the Graduate Diploma of Education program at the Queensland University of Technology in Australia were required to participate in a panel interview where they were graded by a panel made up of three faculty staff and one undergraduate student enrolled in the four-year Bachelor of Education program. Students and panel members completed a questionnaire on their experience after the interview. Results indicated that both students and staff valued the experience and felt it was authentic. Results are discussed in terms of how the assessment interview and portfolio presentation supports graduating students in their preparation for employment interviews, and how this authentic assessment task has benefits for both students and teaching staff.