Designing online assessment tools for disengaged youth

This article reports on the development of online assessment tools for disengaged youth in flexible learning environments. Sociocultural theories of learning and assessment and Bourdieu’s sociological concepts of capital and exchange were used to design a purpose-built content management system. This design experiment engaged participants in assessment that led to the exchange of self, peer and teacher judgements for credentialing. This collaborative approach required students and teachers to adapt and amend social networking practices for students to submit and judge their own and others’ work using comments, ratings, keywords and tags. Students and teachers refined their evaluative expertise across contexts, and negotiated meanings and values of digital works, which gave rise to revised versions and emergent assessment criteria. By combining social networking tools with sociological models of capital, assessment activities related to students’ digital productions were understood as valuations and judgements within an emergent, negotiable social field of exchange.

Blended delivery and online assessment : scaffolding student reflections in work-integrated learning

This paper documents a teaching innovation addressing the challenges of embedding and assessing reflective practice in work-integrated learning, specifically marketing internships. We identify four issues relating to this problem: lack of knowledge or skill for reflection, limitations of physical journals, facilitation of different forms of reflection, and suitable models for teaching and assessing reflection. The paper outlines a blended approach combining face-to-face workshops and online resources, and using online reflective journals and digital stories as assessment. The approach and assessment tasks can be implemented in a variety of marketing and business units.

A pilot study of online assessment of childhood speech disorders

We investigated the feasibility of assessing childhood speech disorders via an Internet-based telehealth system (eREHAB). The equipment provided videoconferencing through a 128 kbit/s Internet link, and enabled the transfer of pre-recorded video and audio data from the participant to the online clinician. Six children (mean age = 5.3 years) with a speech disorder were studied. Assessments of single-word articulation, intelligibility in conversation, and oro-motor structure and function were conducted for each participant, with simultaneous scoring by a face to face and an online clinician. There were high levels of agreement between the two scoring environments for single-word articulation (92%), speech intelligibility (100%) and oro-motor tasks (91%). High levels of inter- and intra-rater agreement were achieved for the online ratings for most measures. The results suggest that an Internet-based assessment protocol has potential for assessing paediatric speech disorders.

An online assessment and development tool for business incubators:optimising business-incubation management practice, performance and impact, Project Report

This project is focused on exchanging knowledge between ABS, UKBI and managers of business incubators in the UK. The project relates to exploitation of extant knowledge-base on assessing and improving business incubation management practice and performance and builds on two earlier studies. It addresses a pressing need for assessing and benchmarking business incubation input, process and outcome performance and highlighting best practice. The overarching aim of this project was to obtain proof-of-concept for a business incubation performance assessment and benchmarking online tool, fine-tune it and put it in use by nurturing a community of business incubation management practice, aligned by the resultant tool. The purpose was to offer an appropriate set of measures, in areas identified by relevant research on business incubation performance management and impact as critical, against which: 1.The input and process performance of business incubation management practice can be assessed and benchmarked within the auspices of a community of incubator managers concerned with best practice 2.The outcome performance and impact of business incubators can be assessed longitudinally. As such, the developed online assessment framework is geared towards the needs of researchers, policy makers and practitioners concerned with business incubation performance, added value and impact.

Sustainable assessment for large science classes : non-multiple choice, randomised assignments through a Learning Management System

This paper reports on the development of a tool that generates randomised, non-multiple choice assessment within the BlackBoard Learning Management System interface. An accepted weakness of multiple-choice assessment is that it cannot elicit learning outcomes from upper levels of Biggs’ SOLO taxonomy. However, written assessment items require extensive resources for marking, and are susceptible to copying as well as marking inconsistencies for large classes. This project developed an assessment tool which is valid, reliable and sustainable and that addresses the issues identified above. The tool provides each student with an assignment assessing the same learning outcomes, but containing different questions, with responses in the form of words or numbers. Practice questions are available, enabling students to obtain feedback on their approach before submitting their assignment. Thus, the tool incorporates automatic marking (essential for large classes), randomised tasks to each student (reducing copying), the capacity to give credit for working (feedback on the application of theory), and the capacity to target higher order learning outcomes by requiring students to derive their answers rather than choosing them. Results and feedback from students are presented, along with technical implementation details.

Building on the past for the future : assessment for learning as a field of exchange

Standardised testing does not recognise the creativity and skills of marginalised youth. This paper presents the development of an innovative approach to assessment designed for the re-engagement of at risk youth who have left formal schooling and are now in an alternative education institution. An electronic portfolio system (EPS) has been developed to capture, record and build on the broad range of students’ cultural and social capital. The assessment as a field of exchange model draws on categories from sociological fields of capital and reconceptualises an eportfolio and social networking hybrid system as a sociocultural zone of learning and development. The EPS, and assessment for learning more generally, are conceptualised as social fields for the exchange of capital (Bourdieu 1977, 1990). The research is underpinned by a sociocultural theoretical perspective that focuses on how students and teachers at the Flexible Learning Centre (FLC) develop and learn, within the zone of proximal development (Vygotsky, 1978). The EPS is seen to be highly effective in the engagement and social interaction between students, teachers and institutions. It is argued throughout this paper that the EPS provides a structurally identifiable space, an arena of social activity, or a field of exchange. The students, teachers and the FLC within this field are producing cultural capital exchanges. The term efield (exchange field) has been coined to refer to this constructed abstract space. Initial results from the trial show a general tendency towards engagement with the EPS and potential for the attainment of socially valued cultural capital in the form of school credentials.

E-assessment em ensino a distância on-line: estudo exploratório no curso de licenciatura em gestão de negócios da Universidade Eduardo Mondlane

A evolução acelerada da tecnologia, com um grande impacto em quase todos os segmentos da humanidade, coloca o homem perante novos desafios, com vários graus de complexidade. Cada dia que passa, estes desafios exigem novas competências e estratégias que devem ser adequadas a cada contexto. A dinâmica das tecnologias de comunicação e informação impulsionaram a evolução do ensino a distância (EaD), o que de certo modo tem exigido grandemente às instituições provedoras do EaD em geral e, em particular, às provedoras do EaD on-line novas formas de gestão, e novas formas de ensinar, fazer aprender os seus estudantes e avaliar as suas aprendizagens. No sentido de contribuir para a evolução e melhoramento do processo de avaliação das aprendizagens on-line, desenvolveu-se um estudo sobre o tema avaliação das aprendizagens on-line como elemento potenciador da interação em EaD. O desenvolvimento do tema foi orientado por um referencial teórico e contextual e concretizou-se, na sua vertente empírica, num estudo de caso exploratório. Este estudo desenvolveu-se em duas fases. Fase I: de caracterização de conceções e “práticas” vigentes de avaliação das aprendizagens on-line no Curso de licenciatura de Gestão de Negócios (LGN) da Universidade Eduardo Mondlane (UEM) de Moçambique; Fase II: de conceção, implementação e avaliação de uma proposta alternativa de avaliação das aprendizagens on-line no Curso de LGN. Para a consecução do estudo empírico concorreram um conjunto de procedimentos metodológicos que envolveu diversas técnicas e instrumentos de recolha de dados. Na fase I recorreu-se à entrevista a seis atores do Curso de LGN (3 professores, 3 assessores pedagógicos e 3 estudantes) e à análise documental (documentos relativos à conceção do Curso e outros relativos à sua implementação). Na fase II recorreu-se à observação participante, a um questionário respondido pelos estudantes participantes, a um relatório elaborado pelo professor colaborador na implementação da proposta por nós concebido, e a documentos retirados da plataforma usada no Curso de LGN, a plataforma Aulanet, nomeadamente quanto às interações dos estudantes em fóruns e chats. Os dados recolhidos foram sujeitos a análises quantitativas e qualitativas.Dos resultados obtidos na Fase I realça-se a necessidade sentida pelos entrevistados de se mudarem práticas de avaliação on-line, nomeadamente de forma a potenciar a interação e interatividade e permitir que a classificação nos módulos não fosse apenas resultado de um exame final. Dos resultados obtidos na Fase II enfatiza-se a apreciação globalmente positiva da nova metodologia de avaliação das aprendizagens on-line, quer pelos estudantes, quer pelo professor participante. Foram identificados como pontos fortes: o aumento das interações na plataforma, em particular nos fóruns e chats; as interações geraram uma dinâmica na evolução dos estudantes no que diz respeito ao desenvolvimento das suas competências no seu sucesso académico; a avaliação, por explicitar os critérios e indicadores, tornou-se mais transparente e orientadora do trabalho dos estudantes e do professor. Porém foram identificados alguns pontos menos fortes, nomeadamente: a sobrecarga de trabalho do professor e a não adesão total dos estudantes à metodologia proposta. Dada a natureza exploratória do estudo sugere-se a sua continuidade, através do envolvimento de outros professores e estudantes.

Prototype of a Configurable Web-Based Assessment System

The MIT Prototype Educational Assessment System provides subjects and courses at MIT with the ability to perform online assessment. The system includes polices to handle harassment and electronic "flaming" while protecting privacy. Within these frameworks, individual courses and subjects can make their own policy decisions about such matters as to when assessments can occur, who can submit assessments, and how anonymous assessments are. By allowing assessment to take place continually and allowing both students and staff to participate, the system can provide a forum for the online discussion of subjects. Even in the case of scheduled assessments, the system can provide advantages over end-of-term assessment, since the scheduled assessments can occur several times during the semester, allowing subjects to identify and adjust those areas that could use improvement. Subjects can also develop customized questionnaires, perhaps in response to previous assessments, to suit their needs.

Expanding the online math homework system WeBWork to CUNY

Math teachers everywhere agree: the way to learn math is to do math. Effective homework is a key to a successful math course. With this goal in mind, a group of math professors at BCC spent the last year working with the online homework system WeBWork. Our intention is to expand our current implementation, with the hope of working across campuses. We will discuss the advantages of WeBWork and how we might work collaboratively across CUNY.

An Internet-based telerehabilitation system for the assessment of motor speech disorders: A pilot study

Purpose: This pilot study explored the feasibility and effectiveness of an Internet-based telerehabilitation application for the assessment of motor speech disorders in adults with acquired neurological impairment. Method: Using a counterbalanced, repeated measures research design, 2 speech-language pathologists assessed 19 speakers with dysarthria on a battery of perceptual assessments. The assessments included a 19-item version of the Frenchay Dysarthria Assessment (FDA; P. Enderby, 1983), the Assessment of Intelligibility of Dysarthric Speech (K. M. Yorkston & D. R. Beukelman, 1981), perceptual analysis of a speech sample, and an overall rating of severity of the dysarthria. One assessment was conducted in the traditional face-to-face manner, whereas the other assessment was conducted using an online, custom-built telerehabilitation application. This application enabled real-time videoconferencing at 128 kb/s and the transfer of store-and-forward audio and video data between the speaker and speech-language pathologist sites. The assessment methods were compared using the J.M.Bland and D.G.Altman (1986, 1999) limits-of-agreement method and percentage level of agreement between the 2 methods. Results: Measurements of severity of dysarthria, percentage intelligibility in sentences, and most perceptual ratings made in the telerehabilitation environment were found to fall within the clinically acceptable criteria. However, several ratings on the FDA were not comparable between the environments, and explanations for these results were explored. Conclusions: The online assessment of motor speech disorders using an Internet-based telerehabilitation system is feasible. This study suggests that with additional refinement of the technology and assessment protocols, reliable assessment of motor speech disorders over the Internet is possible. Future research methods are outlined.

An assessment of the learning benefits of using a web-based learning environment when teaching accounting

The impact and use of information and communication technology on learning outcomes for accounting students is not well understood. This study investigates the impact of design features of Blackboard 1 used as aWeb-based Learning Environment (WBLE) in teaching undergraduate accounting students. Specifically, this investigation reports on a number of Blackboard design features (e.g. delivery of lecture notes, announcements, online assessment and model answers) used to deliver learning materials regarded as necessary to enhance learning outcomes. Responses from 369 on-campus students provided data to develop a regression model that seeks to explain enhanced participation and mental effort. The final regression shows that student satisfaction with the use of a WBLE is associated with five design features or variables. These include usefulness and availability of lecture notes, online assessment, model answers, and online chat.

How can flexible learning centres use social networking to improve their ongoing educational assessments?

Most online assessment systems now incorporate social networking features, and recent developments in social media spaces include protocols that allow the synchronisation and aggregation of data across multiple user profiles. In light of these advances and the concomitant fear of data sharing in secondary school education this papers provides important research findings about generic features of online social networking, which educators can use to make sound and efficient assessments in collaboration with their students and colleagues. This paper reports on a design experiment in flexible educational settings that challenges the dichotomous legacy of success and failure evident in many assessment activities for at-risk youth. Combining social networking practices with the sociology of education the paper proposes that assessment activities are best understood as a negotiable field of exchange. In this design experiment students, peers and educators engage in explicit, "front-end" assessment (Wyatt-Smith, 2008) to translate digital artefacts into institutional, and potentiality economic capital without continually referring to paper based pre-set criteria. This approach invites students and educators to use social networking functions to assess “work in progress” and final submissions in collaboration, and in doing so assessors refine their evaluative expertise and negotiate the value of student’s work from which new criteria can emerge. The mobile advantages of web-based technologies aggregate, externalise and democratise this transparent assessment model for most, if not all, student work that can be digitally represented.

Writing-to-Learn Using Social Pedagogy and Technology

Accounting faculty from LaGuardia will share the structure and results of an initiative centered on using ePortfolio and social pedagogy to facilitate writing-to-learn for accounting students. Through a series of ePortfolio-based assignments that connected two classes, along with an in-class writing workshop conducted by non-discipline faculty, students worked together to understand the importance of writing in the accounting discipline and strengthened their writing abilities.

The Use of On-Line Computer adaptive Testing to raise Standardized Test Scores

Nursing school graduates are under pressure to pass the RN-NCLEX Exam on the first attempt since New York State monitors the results and uses them to evaluate the school’s nursing programs. Since the RN-NCLEX Exam is a standardized test, we sought a method to make our students better test takers. The use of on-line computer adaptive testing has raised our student’s standardized test scores at the end of the nursing course.