Developing a Process to Create and Validate an Instrument Assessing Student Attainment of Competencies at an Intercultural University in Mexico

This study took place at one of the intercultural universities (IUs) of Mexico that serve primarily indigenous students. The IUs are pioneers in higher education despite their numerous challenges (Bertely, 1998; Dietz, 2008; Pineda & Landorf, 2010; Schmelkes, 2009). To overcome educational inequalities among their students (Ahuja, Berumen, Casillas, Crispín, Delgado et al., 2004; Schmelkes, 2009), the IUs have embraced performance-based assessment (PBA; Casillas & Santini, 2006). PBA allows a shared model of power and control related to learning and evaluation (Anderson, 1998). While conducting a review on PBA strategies of the IUs, the researcher did not find a PBA instrument with valid and reliable estimates. The purpose of this study was to develop a process to create a PBA instrument, an analytic general rubric, with acceptable validity and reliability estimates to assess students’ attainment of competencies in one of the IU’s majors, Intercultural Development Management. The Human Capabilities Approach (HCA) was the theoretical framework and a sequential mixed method (Creswell, 2003; Teddlie & Tashakkori, 2009) was the research design. IU participants created a rubric during two focus groups, and seven Spanish-speaking professors in Mexico and the US piloted using students’ research projects. The evidence that demonstrates the attainment of competencies at the IU is a complex set of actual, potential and/or desired performances or achievements, also conceptualized as “functional capabilities” (FCs; Walker, 2008), that can be used to develop a rubric. Results indicate that the rubric’s validity and reliability estimates reached acceptable estimates of 80% agreement, surpassing minimum requirements (Newman, Newman, & Newman, 2011). Implications for practice involve the use of PBA within a formative assessment framework, and dynamic inclusion of constituencies. Recommendations for further research include introducing this study’s instrument-development process to other IUs, conducting parallel mixed design studies exploring the intersection between HCA and assessment, and conducting a case study exploring assessment in intercultural settings. Education articulated through the HCA empowers students (Unterhalter & Brighouse, 2007; Walker, 2008). This study aimed to contribute to the quality of student learning assessment at the IUs by providing a participatory process to develop a PBA instrument.

Project RAILS: Lessons Learned about Collaborative Rubric Assessment of Information Literacy Skills

Link to article on publisher site: https://www.press.jhu.edu/journals/portal_libraries_and_the_academy/portal_pre_print/articles/belanger.pdf

Three Creative Tensions in Document Interpretation Theory Set as Evidence of the Need fora Descriptive Informatics

Describes three tensions in the theoretical literature of indexing: chief sources of evidence indexing, process of indexing (rubrics and methods), and philosophical position of indexing scholarship. Following this exposition, we argue for a change in perspective in Knowledge Organization research. Using the difference between prescriptive and descriptive linguis- tics as a metaphor, we advocate for a shift to a more descriptive, rather than the customary prescriptive, approach to the theo- retical and empirical study of indexing, and by extension Knowledge Organization.

La importancia de la formación del profesorado en la evaluación de aprendizajes con rúbrica

En esta investigación pretendemos conocer si las emociones que experimenta el alumnado de Primaria al ser evaluado con rúbrica pueden verse afectadas por la formación y el conocimiento que sobre dicho instrumento se les presupone a sus evaluadores. Por ello, se establece una comparativa entre las emociones que sienten los estudiantes al ser evaluados por profesores (a los que se les reconoce una formación sobre rúbrica) y las que sienten aquellos que son coevaluados por sus iguales o autoevaluados. Un total de 66 alumnos del 2º ciclo de Enseñanza Básica (10 años y 11 años) de Évora (Portugal), que habían tenido experiencia de heteroevaluación, autoevaluación o coevaluación con rúbrica, cumplimentaron un amplio cuestionario. Los resultados muestran que cerca de un 62% de los estudiantes que fueron evaluados por pares o autoevaluados a través de una rúbrica experimentan emociones positivas, frente a un 91 % de aquellos cuyo trabajo fue valorado por profesores utilizando dicho instrumento. En consonancia con este dato, existe mayor probabilidad de experimentar emociones positivas cuando se recibe una evaluación con rúbrica por parte de un maestro que cuando la valoración procede de un igual o de uno mismo. También se ha observado que el porcentaje de estudiantes que experimenta emociones negativas es mayor cuando no son evaluados por el profesor. Además, un 76 % de los alumnos evaluados con rúbrica por los compañeros considera que esta acción puede ayudarles, frente a un 81% de los evaluados por el docente que comparten esta opinión. Por último, cabe destacar la posibilidad de que estos resultados estén mediatizados por la percepción de que una evaluación realizada por un igual no posee la misma calidad que la que lleva a cabo un profesor.