Enhancing fuzzy inference system based criterion-referenced assessment with an application

An important and difficult issue in designing a Fuzzy Inference System (FIS) is the specification of fuzzy sets, and fuzzy rules. The aim of this paper is to demonstrate how an additional qualitative information, i.e., monotonicity property, can be exploited and extended to be part of an FIS designing procedure (i.e., fuzzy sets and fuzzy rules design). In this paper, the FIS is employed as an alternative to the use of addition in aggregating the scores from test items/tasks in a Criterion-Referenced Assessment (CRA) model. In order to preserve the monotonicity property, the sufficient conditions of the FIS is proposed. Our proposed FIS based CRA procedure can be viewed as an enhancement for the FIS based CRA procedure, where monotonicity property is preserved. We demonstrate the applicability of the proposed approach with a case study related to a laboratory project assessment task at a university, and the results indicate the usefulness of the proposed approach in the CRA domain.

Does calibration reduce variability in the assessment of accounting learning outcomes?

Reliable, consistent assessment process that produces comparable assessment grades between assessors and institutions is a core activity and an ongoing challenge with which universities have failed to come to terms. In this paper, we report results from an experiment that tests the impact of an intervention designed to reduce grader variability and develop a shared understanding of national threshold learning standards by a cohort of reviewers. The intervention involved consensus moderation of samples of accounting students’ work, with a focus on three research questions. First, what is the quantifiable difference in grader variability on the assessment of learning outcomes in ‘application skills’ and ‘judgement’? Second, does participation in the workshops lead to reduced disparity in the assessment of the students’ learning outcomes in ‘application skills’ and ‘judgement’? Third, does participation in the workshops lead to greater confidence by reviewers in their ability to assess students’ skills in application skills and judgement? Our findings suggest consensus moderation does reduce variability across graders and also builds grader confidence.

Patterns of resistance in managing assessment change

Achieving change in assessment practices in higher education is difficult. One of the reasons for this is resistance among those responsible for teaching and assessing. This paper seeks to explore this resistance through an analysis of staff dialogue during a major attempt to change the assessment practices at one institution. An institution-wide intervention to pilot new assessment practices was initiated, involving 35 academics across 12 departments. This paper reports on patterns emerging in dialogue about this among academics, and between academics and educational development personnel. Findings suggest that resistance is not a unitary concept, but that different stakeholders under varied circumstances express it differently; and that resistance to assessment change is particularly resilient. Implications are discussed in terms of relevance to current theories of higher education assessment and learning change management, as well as the practical considerations of attempts by institutions to engage in assessment change. © 2013 © 2013 Taylor & Francis.

Teamwork benefits in tertiary education

Purpose – The purpose of this paper is to contribute to the understanding of the development ofstudents’ skills in the context of team-based learning. Academics have heeded the call to incorporateteam learning activities into the curricula, yet little is known of student perception of teamworkand whether they view it as beneficial to them and their future professional career. Further, this studypresents an instructional framework to guide best practice in higher education practitioners withregard to the design of teamwork assessment.Design/methodology/approach – The paper adopts a qualitative approach utilising 190 students’reflections to examine their perception of the benefits of teamwork and whether it will contribute totheir future professional work.Findings – Results indicate students perceive team-based assessment tasks require them to adopt adeep approach to learning together with a deep approach to study, as well as improving their skillsin the areas of collaboration, team unity and cultural diversity. Further, the study identified a bestpractice approach that higher education practitioners should adopt in teamwork assessment designgiving this study both national and international significance and aids fellow educators in theirpractices.Research limitations/implications – Because of the chosen research approach, the results maylack generalisability. Therefore, researchers are encouraged to test the proposed propositions further.Practical implications – The paper presents important implications for those involved in thedevelopment of assessment items where objectives include the development of team skills and qualitylearning outcomes. The findings are vital for unit and course planning and design generally, andassessment planning, design and processes, specifically, both nationally and internationally.Originality/value – This paper fulfils an identified need to study students’ perceptions of teamwork,whether they view it as beneficial to them and their future professional career, and presents a bestpractice approach for teamwork assessment design.

Evaluating assessment practices in design based learning environment

This investigation is focused on evaluating assessment practices in design based learning environment. The School of Engineering at Deakin University practices project/design based learning as one of its learning and teaching approach. When identifying graduate attributes particularly for undergraduate engineering programs in Australia, the program accrediting body Engineers Australia (EA) initiates a set of graduate attribute elements which was mentioned in “Stage1 competencies and elements of competency”. Stage1 competencies state that one of the important engineering application ability for graduates is ‘application of systematic engineering synthesis and design processes’. By practicing the design focused learning environment and evaluating students perceptions, This investigation examines students’ experiences of assessment practices in their curriculum through an online survey given to the same cohort of students in third year and fourth year undergraduate engineering.

Assessment of a high school geological field course

Fieldwork is supportive of students’ natural inquiry abilities. Educational research findings suggest that instructors can foster the growth of thinking skills and promote science literacy by incorporating active learning strategies (McConnel et al, 2003). Huntoon (2001) states that there is a need to determine optimal learning strategies and to document the procedure of assessing those optimal geoscience curricula. This study seeks to determine if Earth Space II, a high school geological field course, can increase students’ knowledge of selected educational objectives. This research also seeks to measure any impact Earth Space II has on students’ attitude towards science. Assessment of the Earth Space II course objectives provided data on the impact of field courses on high school students’ scientific literacy, scientific inquiry skills, and understanding of selected course objectives. Knowledge assessment was done using a multiple choice format test, the Geoscience Concept Inventory, and an open-ended format essay test. Attitude assessment occurred by utilizing a preexisting science attitude survey. Both knowledge assessments items showed a positive effect size from the pretest to the posttest. The essay exam effect size was 17 and the Geoscience Concept Inventory effect size was 0.18. A positive impact on students’ attitude toward science was observed by an increase in the overall mean Likert value from the pre-survey to the post-survey.

Assessment of anastomotic leakage: A Novel System for the Training of Surgeons to Perform a Tubular Anastomosis and Objectively Evaluate Anastomotic Leak

Introduction: Dehiscence of the suture line of an anastomosis can lead to reoperation, temporary or permanent stoma, and even sepsis or death. Few techniques for the laboratory training of tubular anastomosis use ex-vivo animal tissues. We describe a novel model that can be used in the laboratory for the training of anastomosis in tubular tissues and objectively assess any anastomotic leak. [See PDF for complete abstract]

Do multiple true-false items beat the commonly used one-best-answers questions regarding to the Ottawa Criteria for Good Assessment? Results of a literature review

Background: Multiple True-False-Items (MTF-Items) might offer some advantages compared to one-best-answer-questions (TypeA) as they allow more than one correct answer and may better represent clinical decisions. However, in medical education assessment MTF-Items are seldom used. Summary of Work: With this literature review existing findings on MTF-items and on TypeA were compared along the Ottawa Criteria for Good Assessment, i.e. (1) reproducibility, (2) feasibility, (3) validity, (4) acceptance, (5) educational effect, (6) catalytic effects, and (7) equivalence. We conducted a literature research on ERIC and Google Scholar including papers from the years 1935 to 2014. We used the search terms “multiple true-false”, “true-false”, “true/false”, and “Kprim” combined with “exam”, “test”, and “assessment”. Summary of Results: We included 29 out of 33 studies. Four of them were carried out in the medical field Compared to TypeA, MTF-Items are associated with (1) higher reproducibility (2) lower feasibility (3) similar validity (4) higher acceptance (5) higher educational effect (6) no studies on catalytic effects or (7) equivalence. Discussion and Conclusions: While studies show overall good characteristics of MTF items according to the Ottawa criteria, this type of question seems to be rather seldom used. One reason might be the reported lower feasibility. Overall the literature base is still weak. Furthermore, only 14 % of literature is from the medical domain. Further studies to better understand the characteristics of MTF-Items in the medical domain are warranted. Take-home messages: Overall the literature base is weak and therefore further studies are needed. Existing studies show that: MTF-Items show higher reliability, acceptance and educational effect; MTF-Items are more difficult to produce

Novel educational assessment for building structures: Automatic evaluation of on-line graphics

This article presents a new automatic evaluation for on-line graphics, its application and the numerous advantages achieved applying this developed correcting method. The software application developed by the Innovation in Education Group “E4”, from the Technical University of Madrid, is oriented for the online self-assessment of the graphic drawings that students carry out as continuous training. The adaptation to the European Higher Educational Area is an important opportunity to research about the possibilities of on-line education assessment. In this way, a new software tool has been developed for continuous self-testing by undergraduates. Using this software it is possible to evaluate the graphical answer of the students. Thus, the drawings made on-line by students are automatically corrected according to the geometry (straight lines, sloping lines or second order curves) and by sizes (depending on the specific values which define the graphics).

The language proficiency handbook : a practitioner's guide to instructional assessment.

"This guide is useful for preK-12 educators who work with second language learners, irrespective of which language, who wish to document their students' language development over time"--P. ii.

Curriculum, content and controversy in higher education

Recent evidence suggest that academic staff face difficulties in applying new technologies as a means of assessing higher order assessment outcomes such as critical thinking, problem solving and creativity. Although higher education institutional mission statements and course unit outlines purport the value of these higher order skills there is still some question about how well academics are equipped to design curricula and, in particular, assessment strategies accordingly. Despite a rhetoric avowing the benefits of these higher order skills, it has been suggested that academics set assessment tasks up in such a way as to inadvertently lead students on the path towards lower order outcomes. This is a controversial claim, and one that this paper seeks to explore and critique in terms of challenging the conceptual basis of assessing higher order skills through new technologies. It is argued that the use of digital media in higher education is leading to a focus on student's ability to use and manipulate of these products as an index of their flexibility and adaptability to the demands of the knowledge economy. This focus mirrors market flexibility and encourages programmes and courses of study to be rhetorically packaged as such. Curricular content has becomes a means to procure more or less elaborate aggregates of attributes. Higher education is now charged with producing graduates who are entrepreneurial and creative in order to drive forward economic sustainability. It is argued that critical independent learning can take place through the democratisation afforded by cultural and knowledge digitization and that assessment needs to acknowledge the changing relations between audience and author, expert and amateur, creator and consumer.

Is peer assessment of asynchronous group discussions fostering skills relevant to our future graduates?

The increasing use of peer assessment in higher education institutions, as well as its benefits in term of students' learning is well documented. Distance education can be fraught with challenges, but creating a community of practice has been proven to increase student engagement and learning. This paper reports on the implementation of peer assessment of online asynchronous group discussions to foster a community of practice and equip future graduates with lifelong skills relevant to their chosen professional path. Through a careful preparation of students, the implementation of the peer assessment process proved beneficial. This paper describes the analysis performed to establish the validity and reliability of the peer assessment process in the context of a 3rd year unit of study of the bachelor of nursing at Charles Darwin University. © 2010 Stéphane Bouchoucha & Helen Wozniak.