How difficult are exams? A framework for assessing the complexity of introductory programming exams


Autoria(s): Sheard, Judy; Simon,; Carbone, Angela; Chinn, Donald; Clear, Tony; Corney, Malcolm W.; D'Souza, Daryl; Fenwick, Joel; Harland, James; Laakso, Mikko-Jussi; Teague, Donna M.
Contribuinte(s)

Carbone, Angela

Whalley, Jacqueline

Data(s)

01/01/2013

Resumo

Student performance on examinations is influenced by the level of difficulty of the questions. It seems reasonable to propose therefore that assessment of the difficulty of exam questions could be used to gauge the level of skills and knowledge expected at the end of a course. This paper reports the results of a study investigating the difficulty of exam questions using a subjective assessment of difficulty and a purpose-built exam question complexity classification scheme. The scheme, devised for exams in introductory programming courses, assesses the complexity of each question using six measures: external domain references, explicitness, linguistic complexity, conceptual complexity, length of code involved in the question and/or answer, and intellectual complexity (Bloom level). We apply the scheme to 20 introductory programming exam papers from five countries, and find substantial variation across the exams for all measures. Most exams include a mix of questions of low, medium, and high difficulty, although seven of the 20 have no questions of high difficulty. All of the complexity measures correlate with assessment of difficulty, indicating that the difficulty of an exam question relates to each of these more specific measures. We discuss the implications of these findings for the development of measures to assess learning standards in programming courses.

Formato

application/pdf

Identificador

http://eprints.qut.edu.au/57540/

Publicador

ACS

Relação

http://eprints.qut.edu.au/57540/1/SheardEtAl2013HowDifficultAreExams_aFrameworkForAssessingTheComplexityOfIntroductoryProgrammingExams.pdf

Sheard, Judy, Simon, , Carbone, Angela, Chinn, Donald, Clear, Tony, Corney, Malcolm W., D'Souza, Daryl, Fenwick, Joel, Harland, James, Laakso, Mikko-Jussi, & Teague, Donna M. (2013) How difficult are exams? A framework for assessing the complexity of introductory programming exams. In Carbone, Angela & Whalley, Jacqueline (Eds.) Proceedings of the 15th Australasian Computing Education Conference, ACS, Adelaide, SA.

Direitos

Copyright 2013 Australian Computer Society, Inc.

Copyright © 2013, Australian Computer Society, Inc. This paper appeared at the 15th Australasian Computing Education Conference (ACE 2013). Conferences in Research and Practice in Information Technology (CRPIT), Vol. 136. Angela Carbone and Jacqueline Whalley, Eds. Reproduction for academic, not-for profit purposes permitted provided this text is included.

Fonte

School of Electrical Engineering & Computer Science; Science & Engineering Faculty

Palavras-Chave #080300 COMPUTER SOFTWARE #130200 CURRICULUM AND PEDAGOGY #standards #quality #examination papers #CS1 #introductory programming #assessment #question complexity #question difficulty #HERN
Tipo

Conference Paper