PEER-REVIEWING, FEEDBACK & ASSESSMENT IN ENGINEERING TEACHING

Presentation

WEBS6203 Peer reviewing your reports

These notes contain a workflow, guidance notes, and supporting forms

[Study protocol of the VISEP study. Response of the SepNet study group]

In the commentary by Zander et al. the authors appear concerned about the methods and results of our, at that time, unpublished sepsis trial evaluating hydroxyethyl starch (HES) and insulin therapy. Unfortunately, the authors' concerns are based on false assumptions about the design, conduct and modes of action of the compounds under investigation. For instance, in our study the HES solution was not used for maintenance of daily fluid requirements, so that the assumption of the authors that this colloid was used "exclusively" is wrong. Moreover, the manufacturer of Hemohes, the HES product we used, gives no cut-off value for creatinine, thus the assumption that this cut-off value was "doubled" in our study is also incorrect. Other claims by the authors such as that lactated solutions cause elevated lactate levels, iatrogenic hyperglycemia and increase O(2) consumption are unfounded. There is no randomized controlled trial supporting such a claim - this claim is neither consistent with our study data nor with any credible published sepsis guidelines or with routine practice worldwide. We fully support open scientific debate. Our study methods and results have now been published after a strict peer-reviewing process and this data is now open to critical and constructive reviewing. However, in our opinion this premature action based on wrong assumptions and containing comments by representatives of pharmaceutical companies does not contribute to a serious, unbiased scientific discourse.

Nuevo método de evaluación mediante revisiones cruzadas de los estudiantes

Actualmente nos enfrentamos a nuevos desafíos en el mundo de la educación. Entre las cuestiones más importantes se encuentra el grado de participación y compromiso de los estudiantes en su propio proceso de aprendizaje y esto implica que deban participar en su evaluación de manera activa. Así, en este artículo se presenta un nuevo método de evaluación empleado en la asignatura “Arquitecturas y Sistemas Operativos para Tiempo Real”, del título de Ingeniería Informática de la Universidad de Alicante. En este novedoso método, los alumnos participan en el proceso de evaluación de sus propios trabajos por medio de revisiones cruzadas (peer-reviewing) que pretenden identificar las carencias o errores en los trabajos presentados por sus compañeros de asignatura. En pocas semanas los estudiantes pueden entender la innovadora visión del proceso de enseñanza-aprendizaje empleado en nuestra asignatura y se involucran activamente en dicho proceso; con todo ello, sus conocimientos han de actualizarse de manera continua, por lo que son capaces de entender y asimilar los nuevos conceptos explicados por el profesor.

Development of multidisciplinary workshops between different subjects as an innovative teaching methodology for the acquisition of professional skills of the future architect

The implantation of the new Architecture Degree and the important normative changes in the building sector imply the need to use new teaching methodologies that enhance skills and competences in order to response to the increasing requirements demanded by society to the future architect. The aim of this paper is to present, analyze and discuss the development of multidisciplinary workshops as a new teaching methodology used in several Construction subjects of the Architecture Degree in the University of Alicante. Workshops conceived with the aim to synthesize and complement the technical knowledge acquired by the students during the Degree and to enhance the skills and competencies necessary for the professional practice. With that purpose, we decided to experiment on current subjects of the degree during this academic year, by applying the requirements defined in the future Architecture Degree in a practical way, through workshops between different subjects, superposing the technical knowledge with the resolution of constructive problems in the development of an architectural project. Developing these workshops between subjects we can dissolve the traditional boundaries between different areas of the Degree. This multidisciplinary workshop methodology allows the use of all the global knowledge acquired by students during their studies and at the same time, it enhances students’ ability to communicate and discuss their ideas and solutions in public. It also increases their capacity of self-criticism, and it foments their ability to undertake learning strategies and research in an autonomous way. The used methodology is based on the development of a practical work common to several subjects of different knowledge areas within the "Technology Block" of the future Architecture Degree. Thus, students work approaching the problem in a global way discussing simultaneously with teachers from different areas. By using these new workshops we stimulate an interactive class versus a traditional lecture. Work is evaluated continuously, valuing the participative pupil´s attitude, working in groups in class time, reaching weekly objectives and stimulating the individual responsibility and positive interdependence of the pupil inside the working group. The exercises are designed to improve students’ ability to transmit their ideas and solutions in public, knowing how to discuss and defend their technical resolutions to peers and teachers (Peer Reviewing), their capacity for self-criticism and their capacity to undertake strategies and autonomous learning processes at the same time they develop a personal research into new technologies, systems and materials. Students have shown their majority preference for this teaching methodology by the multidisciplinary workshops offered in the last years, with very satisfactory academic results. In conclusion, it can be verified nowadays the viability of the introduction of new contents and new teaching methodologies necessary for the acquisition of the skills in the future Architecture Degree, through workshops between several subjects that have had a great acceptance in students and positive contrasted academic results.

What can we learn from the Hwang and Sudbo affairs?

* The recent publication, in prestigious scientific journals, of two major studies that were subsequently shown to contain fabricated data may compel reviewers and editors to adopt a more rigorous policy in accepting articles for publication. * The current manner of peer reviewing research articles provides no assurance that the proffered work is not the result of fraud. * The present guidelines for contributors in large team investigations may need to be updated to avoid giving credit to co-authors who may have made little, if any, contribution to the work.

Seeking good peer review in geomorphology

This paper provides an extended guide to reviewing for ESPL in particular and geomorphology in general. After a brief consideration of both how we choose reviewers and why we hope that reviewers will accept, I consider what makes a fair and constructive review. I note that we aim to publish papers with the rigour (r) necessary to sustain an original and significant contribution (q). I note that judging q is increasingly difficult because of the ever-growing size of the discipline (the Q). This is the sense in which we rarely have a full appreciation of Q, and our reviews are inevitably going to contain some bias. It is this bias that cannot be avoided (cf. Nicholas and Gordon, 2011) and makes the job of ESPL's Editors of critical importance. With this in mind, I identify six elements of a good review: (1) an introductory statement that explains your assessment of your competences in relation to the manuscript (r and Q); (2) a summative view of the originality and significance of the manuscript (q) in relation to Q: (3) a summative view of the methodological rigour of the manuscript (r); (4) identification and justification of any major concerns; (5) identification of any minor issues to be corrected if you think the manuscript merits eventual publication; and (6) note of any typographical or presentation issues to be addressed although this latter activity is also an editorial responsibility. In addition, I note the importance of a constructive review, grounded in what is written in the manuscript, justified where appropriate and avoiding reference to personal views as far as is possible. I conclude with a discussion of whether or not you should sign your review openly and the importance of reviewer confidentiality. Copyright (C) 2012 John Wiley & Sons, Ltd.

Being a successful and valuable peer reviewer

Purpose – The purpose of this editorial is to announce the winners of the EJM Reviewer of the Year awards, and also to make some observations about successful and effective practice in reviewing scholarly work. Design/methodology/approach – The authors draw from their own experience as reviewers, authors, and editors to provide a set of considerations for those involved in the review process. Findings – The authors propose a set of guidelines for reviewers who wish to make the most of their role in the scientific method. Research limitations/implications – These thoughts are preliminary, and are drawn from personal experiences rather than a wide-ranging survey of stakeholders. Practical implications – Marketing researchers should ensure that they understand their role in the scholarly reviewing process, and that they provide timely and constructive comments on their areas of expertise. Originality/value – This piece should offer marketing researchers considerable value in the context of their own efforts, both as authors and reviewer.

Does reviewing lead to better learning and decision making? Answers from a randomized stock market experiment

Background - The literature is not univocal about the effects of Peer Review (PR) within the context of constructivist learning. Due to the predominant focus on using PR as an assessment tool, rather than a constructivist learning activity, and because most studies implicitly assume that the benefits of PR are limited to the reviewee, little is known about the effects upon students who are required to review their peers. Much of the theoretical debate in the literature is focused on explaining how and why constructivist learning is beneficial. At the same time these discussions are marked by an underlying presupposition of a causal relationship between reviewing and deep learning. Objectives - The purpose of the study is to investigate whether the writing of PR feedback causes students to benefit in terms of: perceived utility about statistics, actual use of statistics, better understanding of statistical concepts and associated methods, changed attitudes towards market risks, and outcomes of decisions that were made. Methods - We conducted a randomized experiment, assigning students randomly to receive PR or non–PR treatments and used two cohorts with a different time span. The paper discusses the experimental design and all the software components that we used to support the learning process: Reproducible Computing technology which allows students to reproduce or re–use statistical results from peers, Collaborative PR, and an AI–enhanced Stock Market Engine. Results - The results establish that the writing of PR feedback messages causes students to experience benefits in terms of Behavior, Non–Rote Learning, and Attitudes, provided the sequence of PR activities are maintained for a period that is sufficiently long.

Scotland's Marine Protected Area network : Reviewing progress towards achieving commitments for marine conservation

Acknowledgements This research was supported and funded by Climate XChange (reference no: A10431853). Climate XChange is a collaborative initiative between Scottish research and higher education institutes and is funded by the Scottish Government. The authors would like to thank Marine Scotland, JNCC and SNH for their permission to reproduce their figures of the Scottish MPA process and maps of the Scottish MPA network.

Reviewing mHealth in developing countries: A stakeholder perspective

Infrastructural deficiencies, limited access to medicare, and shortage of health care workers are just a few of the barriers to health care in developing countries. As a consequence, the burden of disease and its impact on the livelihoods and the economic productivity of people are staggering. mHealth has been extolled as one possible solution to overcoming these challenges, yet discussion of mHealth systems is often limited to specific tasks and user groups. To address this, we adopt a stakeholder perspective and analyze existing research on the mHealth process in developing countries. Specifically, we focus on three key stakeholder groups, i.e. healthcare workers, patients, and system developers. We perform an in-depth analysis of 60 peer-reviewed studies to determine the extent to which different mHealth stakeholder interactions are researched, and to identify high-level themes emerging within these interactions. This analysis illustrates two key gaps in existing mHealth research. First, while interactions involving healthcare workers and/or patients have received significant attention, relatively little research has looked at the role of patient-to-patient interactions. Second, the interactions between system developers and the other stakeholder groups are strikingly under-represented. We conclude by calling for more mHealth research that explicitly addresses these stakeholder interactions.

A Taxonomy for Locality Algorithms on Peer-to-Peer Networks

The continuous growth of peer-to-peer networks has made them responsible for a considerable portion of the current Internet traffic. For this reason, improvements in P2P network resources usage are of central importance. One effective approach for addressing this issue is the deployment of locality algorithms, which allow the system to optimize the peers` selection policy for different network situations and, thus, maximize performance. To date, several locality algorithms have been proposed for use in P2P networks. However, they usually adopt heterogeneous criteria for measuring the proximity between peers, which hinders a coherent comparison between the different solutions. In this paper, we develop a thoroughly review of popular locality algorithms, based on three main characteristics: the adopted network architecture, distance metric, and resulting peer selection algorithm. As result of this study, we propose a novel and generic taxonomy for locality algorithms in peer-to-peer networks, aiming to enable a better and more coherent evaluation of any individual locality algorithm.