Is Our Students Learning - QuestionMark: Extending E-Assessment Systems for Adapative Marking and Feedback

Our key contribution is a flexible, automated marking system that adds desirable functionality to existing E-Assessment systems. In our approach, any given E-Assessment system is relegated to a data-collection mechanism, whereas marking and the generation and distribution of personalised per-student feedback is handled separately by our own system. This allows content-rich Microsoft Word feedback documents to be generated and distributed to every student simultaneously according to a per-assessment schedule.

The feedback is adaptive in that it corresponds to the answers given by the student and provides guidance on where they may have gone wrong. It is not limited to simple multiple choice which are the most prescriptive question type offered by most E-Assessment Systems and as such most straightforward to mark consistently and provide individual per-alternative feedback strings. It is also better equipped to handle the use of mathematical symbols and images within the feedback documents which is more flexible than existing E-Assessment systems, which can only handle simple text strings.

As well as MCQs the system reliably and robustly handles Multiple Response, Text Matching and Numeric style questions in a more flexible manner than Questionmark: Perception and other E-Assessment Systems. It can also reliably handle multi-part questions where the response to an earlier question influences the answer to a later one and can adjust both scoring and feedback appropriately.

New question formats can be added at any time provided a corresponding marking method conforming to certain templates can also be programmed. Indeed, any question type for which a programmatic method of marking can be devised may be supported by our system. Furthermore, since the student’s response to each is question is marked programmatically, our system can be set to allow for minor deviations from the correct answer, and if appropriate award partial marks.

Jigsaw Java: Intelligent, Automated Marking

The popularity of Computing degrees in the UK has been increasing significantly over the past number of years. In Northern Ireland, from 2007 to 2015, there has been a 40% increase in acceptances to Computer Science degrees with England seeing a 60% increase over the same period (UCAS, 2016). However, this is tainted as Computer Science degrees also continue to maintain the highest dropout rates.
In Queen’s University Belfast we currently have a Level 1 intake of over 400 students across a number of computing pathways. Our drive as staff is to empower and motivate the students to fully engage with the course content. All students take a Java programming module the aim of which is to provide an understanding of the basic principles of object-oriented design. In order to assess these skills, we have developed Jigsaw Java as an innovative assessment tool offering intelligent, semi-supervised automated marking of code.
Jigsaw Java allows students to answer programming questions using a drag-and-drop interface to place code fragments into position. Their answer is compared to the sample solution and if it matches, marks are allocated accordingly. However, if a match is not found then the corresponding code is executed using sample data to determine if its logic is acceptable. If it is, the solution is flagged to be checked by staff and if satisfactory is saved as an alternative solution. This means that appropriate marks can be allocated and should another student have submitted the same placement of code fragments this does not need to be executed or checked again. Rather the system now knows how to assess it.
Jigsaw Java is also able to consider partial marks dependent on code placement and will “learn” over time. Given the number of students, Jigsaw Java will improve the consistency and timeliness of marking.

Circadian variation with a diurnal bimodal profile on scent-marking behavior in captive common marmosets (callitrhix jacchus)

Scent-marking behavior is associated with different behavioral contexts in callitrichids, including signalizing a territory, location of feeding resources, and social rank. In marmosets and tamarins it is also associated with intersexual communication. Though it appears very important for the daily routine of the individuals, very few researchers have investigated distribution through the 24-h cycle. In a preliminary report, we described a preferential incidence of this behavior 2 h before nocturnal rest in families of common marmosets. We expand the data using 8 family groups (28 subjects), 8 fathers, 6 mothers, 8 nonreproductive adults (4 sons and 4 daughters), and 6 juvenile (3 sons and 3 daughters) offspring that we kept in outdoor cages under natural environmental conditions. We recorded the frequency of anogenital scent marking for each group during the light phase, twice a wk, for 4 consecutive wks, from March 1998 to September 1999. Cosinor test detected 24- and 8-h variations in 89.3% and 85.7% of the subjects, respectively, regardless of sex or reproductive status. The 8-h component is a consequence of the 2 peaks for the behavior, at the beginning and end of the light phase. Daily distribution of scent marking is similar to that others described previously for motor activity in marmosets. The coincident rhythmical patterns for both behaviors seem to be associated with feeding behavior, as described for callitrichids in free-ranging conditions, involving an increase in foraging activities early in the morning and shortly before nocturnal rest

Epistemic Modality in Linguistic and Literature Essays in English : A comparative corpus-based study of modal verbs in student claims

This study is a corpus-based comparison between student essays written in the subject areas of English linguistics and literature at undergraduate level. They are 200 Bachelor degree theses submitted at a variety of university departments (such as English, Language and Literature, Humanities, Social and Intercultural Studies) in Sweden. The comparison concerns frequencies of core modal verbs and how often they occur together with the I, we and it subject pronouns and in the structures this/the [essay, study, project, thesis] when students attempt to communicate their personal claims. Quantitative and qualitative analyses of the essays show few similarities in the ways that core modal verbs appear in both disciplines. The results indicate mainly distinct differences, especially in relation to clusters and variation of performative verbs. Specific patterns in the ways that students use core modal verbs as hedges have also been identified.

Circadian variation with a diurnal bimodal profile on scent-marking behavior in captive common marmosets (callitrhix jacchus)

Scent-marking behavior is associated with different behavioral contexts in callitrichids, including signalizing a territory, location of feeding resources, and social rank. In marmosets and tamarins it is also associated with intersexual communication. Though it appears very important for the daily routine of the individuals, very few researchers have investigated distribution through the 24-h cycle. In a preliminary report, we described a preferential incidence of this behavior 2 h before nocturnal rest in families of common marmosets. We expand the data using 8 family groups (28 subjects), 8 fathers, 6 mothers, 8 nonreproductive adults (4 sons and 4 daughters), and 6 juvenile (3 sons and 3 daughters) offspring that we kept in outdoor cages under natural environmental conditions. We recorded the frequency of anogenital scent marking for each group during the light phase, twice a wk, for 4 consecutive wks, from March 1998 to September 1999. Cosinor test detected 24- and 8-h variations in 89.3% and 85.7% of the subjects, respectively, regardless of sex or reproductive status. The 8-h component is a consequence of the 2 peaks for the behavior, at the beginning and end of the light phase. Daily distribution of scent marking is similar to that others described previously for motor activity in marmosets. The coincident rhythmical patterns for both behaviors seem to be associated with feeding behavior, as described for callitrichids in free-ranging conditions, involving an increase in foraging activities early in the morning and shortly before nocturnal rest

Rule-based and Resource-bounded: A New Look at Epistemic Logic

Syntactic logics do not suffer from the problems of logical omniscience but are often thought to lack interesting properties relating to epistemic notions. By focusing on the case of rule-based agents, I develop a framework for modelling resource-bounded agents and show that the resulting models have a number of interesting properties.

How do servant leaders ignite absorptive capacity?: the role of epistemic motivation and organizational support

This study examines the role of servant leadership in absorptive capacity. Data from manufacturing and service sector organizations found that: a) there was moderation of servant leadership influence on knowledge identification through POS by high need for cognition, b) there was moderation of servant leadership influence on knowledge application through POS by low time pressure, and c) POS mediated relationship between servant leadership and knowledge dissemination. The findings illustrate and support the importance of a comprehensive model integrating servant leadership, POS, and epistemic motivation in determining absorptive capacity.

A case study of epistemic order in mathematics classroom dialogue

We define epistemic order as the way in which the exchange and development of knowledge takes place in the classroom, breaking this down into a system of three components: epistemic initiative relating to who sets the agenda in classroom dialogue, and how; epistemic appraisal relating to who judges contributions to classroom dialogue, and how; and epistemic framing relating to the terms in which development and exchange of knowledge are represented, particularly in reflexive talk. These components are operationalised in terms of various types of structural and semantic analysis of dialogue. It is shown that a lesson segment displays a multi-layered epistemic order differing from that of conventional classroom recitation.

The Weight of Language and Action: Epistemic Stance, Methodological Action,and Theoretical Perspective in Knowledge Organization

Knowledge organization (KO) research is a field of scholarship concerned with the design, study and critique of the processes of organizing and representing documents that societies see as worthy of preserving (Tennis, 2008). In this context we are concerned with the relationship between language and action.On the one hand, we are concerned with what language can and does do for our knowledge organization systems (KOS). For example, how do the words NEGRO or INDIAN work in historical and contemporary indexing languages? In relation to this, we are also concerned with how we know about knowledge organization (KO) and its languages. On the other hand, we are concerned with how to act given this knowledge. That is, how do we carry out research and how do we design, implement, and evaluate KO systems?It is important to consider these questions in the context of our work because we are delegated by society to disseminate cultural memory. We are endowed with a perspective, prepared by an education, and granted positions whereby society asks us to ensure that documentary material is accessible to future generations. There is a social value in our work, and as such there is a social imperative to our work. We must act with good conscience, and use language judiciously, for the memory of the world is a heavy burden.In this paper, I explore these two weights of language and action that bear down on KO researchers. I first summarize what extant literature says about the knowledge claims we make with regard to KO practices and systems. To make it clear what it is that I think we know, I create a schematic that will link claims (language) to actions in advising, implementing, or evaluating information practices and systems.I will then contrast this with what we do not know, that is, what the unanswered questions might be (Gnoli, 2008 ; Dahlberg, 2011), and I will discuss them in relation to the two weights in our field of KO.Further, I will try to provide a systematic overview of possible ways to address these open questions in KO research. I will draw on the concept of elenchus - the forms of epistemology, theory, and methodology in KO (Tennis, 2008), and framework analysis which are structures, work practice, and discourses of KO systems (Tennis, 2006). In so doing, I will argue for a Neopragmatic stance on the weight of language and action in KO (Rorty, 1982 ; 2000). I will close by addressing the lacuna left in Neopragmatic thought – the ethical imperative to use language and action in a particular good and moral way. That is, I will address the ethical imperative of KO given its weights, epistemologies, theories, and methods. To do this, I will review a sample of relevant work on deontology in both western and eastern philosophical schools (e.g., Harvey, 1995).The perspective I want to communicate in this section is that the good in carrying out KO research may begin with epistemic stances (cf., language), but ultimately stands on ethical actions. I will present an analysis describing the micro and the macro ethical concerns in relation to KO research and its advice on practice. I hope this demonstrates that the direction of epistemology, theory, and methodology in KO, while burdened with the dual weights of language and action, is clear when provided an ethical sounding board. We know how to proceed when we understand how our work can benefit the world.KO is an important, if not always understood, division of labor in a society that values its documentary heritage and memory institutions. Being able to do good requires us to understand how to balance the weights of language and action. We must understand where we stand and be able to chart a path forward, one that does not cause harm, but adds value to the world and those that want to access recorded knowledge.

Epistemic, Ontic, Axiologic, and Praxic Constructs in Knowledge Organization Research

The work of knowledge organization requires a particular set of tools. For instance we need standards of content description like Anglo-American Cataloging Rules Edition 2, Resource Description and Access (RDA), Cataloging Cultural Objects, and Describing Archives: A Content Standard. When we intellectualize the process of knowledge organization – that is when we do basic theoretical research in knowledge organization we need another set of tools. For this latter exercise we need constructs. Constructs are ideas with many conceptual elements, largely considered subjective. They allow us to be inventive as well as allow us to see a particular point of view in knowledge organization. For example, Patrick Wilson’s ideas of exploitative control and descriptive control, or S. R. Ranganathan’s fundamental categories are constructs. They allow us to identify functional requirements or operationalizations of functional requirements, or at least come close to them for our systems and schemes. They also allow us to carry out meaningful evaluation.What is even more interesting, from a research point of view, is that constructs once offered to the community can be contested and reinterpreted and this has an affect on how we view knowledge organization systems and processes. Fundamental categories are again a good example in that some members of the Classification Research Group (CRG) argued against Ranganathan’s point of view. The CRG posited more fundamental categories than Ranganathan’s five, Personality, Matter, Energy, Space, and Time (Ranganathan, 1967). The CRG needed significantly more fundamental categories for their work.1 And these are just two voices in this space we can also consider the fundamental categories of Johannes Kaiser (1911), Shera and Egan, Barbara Kyle (Vickery, 1960), and Eric de Grolier (1962). We can also reference contemporary work that continues comparison and analysis of fundamental categories (e.g., Dousa, 2011).In all these cases we are discussing a construct. The fundamental category is not discovered; it is constructed by a classificationist. This is done because it is useful in engaging in the act of classification. And while we are accustomed to using constructs or debating their merit in one knowledge organization activity or another, we have not analyzed their structure, nor have we created a typology. In an effort to probe the epistemological dimension of knowledge organization, we think it would be a fruitful exercise to do this. This is because we might benefit from clarity around not only our terminology, but the manner in which we talk about our terminology. We are all creative workers examining what is available to us, but doing so through particular lenses (constructs) identifying particular constructs. And by knowing these and being able to refer to these we would consider a core competency for knowledge organization researchers.