Statistical Analyses and Reproducible Research

For various reasons, it is important, if not essential, to integrate the computations and code used in data analyses, methodological descriptions, simulations, etc. with the documents that describe and rely on them. This integration allows readers to both verify and adapt the statements in the documents. Authors can easily reproduce them in the future, and they can present the document's contents in a different medium, e.g. with interactive controls. This paper describes a software framework for authoring and distributing these integrated, dynamic documents that contain text, code, data, and any auxiliary content needed to recreate the computations. The documents are dynamic in that the contents, including figures, tables, etc., can be recalculated each time a view of the document is generated. Our model treats a dynamic document as a master or ``source'' document from which one can generate different views in the form of traditional, derived documents for different audiences. We introduce the concept of a compendium as both a container for the different elements that make up the document and its computations (i.e. text, code, data, ...), and as a means for distributing, managing and updating the collection. The step from disseminating analyses via a compendium to reproducible research is a small one. By reproducible research, we mean research papers with accompanying software tools that allow the reader to directly reproduce the results and employ the methods that are presented in the research paper. Some of the issues involved in paradigms for the production, distribution and use of such reproducible research are discussed.

DISTRIBUTED REPRODUCIBLE RESEARCH USING CACHED COMPUTATIONS

The ability to make scientific findings reproducible is increasingly important in areas where substantive results are the product of complex statistical computations. Reproducibility can allow others to verify the published findings and conduct alternate analyses of the same data. A question that arises naturally is how can one conduct and distribute reproducible research? This question is relevant from the point of view of both the authors who want to make their research reproducible and readers who want to reproduce relevant findings reported in the scientific literature. We present a framework in which reproducible research can be conducted and distributed via cached computations and describe specific tools for both authors and readers. As a prototype implementation we introduce three software packages written in the R language. The cacheSweave and stashR packages together provide tools for caching computational results in a key-value style database which can be published to a public repository for readers to download. The SRPM package provides tools for generating and interacting with "shared reproducibility packages" (SRPs) which can facilitate the distribution of the data and code. As a case study we demonstrate the use of the toolkit on a national study of air pollution exposure and mortality.

A REPRODUCIBLE RESEARCH TOOLKIT FOR R

We present a collection of R packages for conducting and distributing reproducible research using R, Sweave, and LaTeX. The collection consists of the cacheSweave, stashR, and SRPM packages which allow for the caching of computations in Sweave documents and the distribution of those cached computations via remotely accessible key-value databases. We describe the caching mechanism used by the cacheSweave package and tools that we have developed for authors and readers for the purposes of creating and interacting with reproducible documents.

An automatic documentation generator for (C)LP - Reference manual

Lpdoc is an automatic program documentation generator for (C)LP systems. Lpdoc generates a reference manual automatically from one or more source files for a logic program (including ISO-Prolog, Ciao, many CLP systems, ...). It is particularly useful for documenting library modules, for which it automatically generates a description of the module interface. However, lpdoc can also be used quite successfully to document full applications and to generate nicely formatted plain ascii "readme" files. A fundamental advantage of using lpdoc to document programs is that it is much easier to maintain a true correspondence between the program and its documentation, and to identify precisely to what version of the program a given printed manual corresponds. The quality of the documentation generated can be greatly enhanced by including within the program text: • assertions (types, modes, etc. ...) for the predicates in the program, and • machine-readable comments (in the "literate programming" style). The assertions and comments included in the source file need to be written using the Ciao system assertion language. A simple compatibility library is available to make traditional (constraint) logic programming systems ignore these assertions and comments allowing normal treatment of programs documented in this way. The documentation is currently generated in HTML or texinf o format. From the texinf o output, printed and on-line manuals in several formats (dvi, ps, info, etc.) can be easily generated automatically, using publicly available tools, lpdoc can also generate 'man' pages (Unix man page format) as well as brief descriptions in html or emacs info formats suitable for inclusion in an on-line index of applications. In particular, lpdoc can create and maintain fully automatically WWW and info sites containing on-line versions of the documents it produces. The lpdoc manual (and the Ciao system manuals) are generated by lpdoc. Lpdoc is distributed under the GNU general public license. Note: lpdoc is fully supported on Linux, Mac OS X, and other Un*x-like systems. Due to the use of several Un*x-related utilities, some documentation back-ends may require Cygwin under Win32. This documentation corresponds to version 3.0 (2011/7/7, 16:33:15 CEST).

Learning to be literate : parental empowerment in early literacy interventions

Considerable research has focused on the success of early intervention programs for children. However, minimal research has focused on the effect these programs have on the parents of targeted children. Many current early intervention programs champion family-focused and inclusive programming, but few have evaluated parent participation in early interventions and fewer still have evaluated the impact of these programs on beliefs and attitudes and parenting practices. Since parents will continue to play a key role in their child's developmental course long after early intervention programs end, it is vital to examine whether these programs empower parents to take action to make changes in the lives of their children. The goal of this study was to understand parental influences on the early development of literacy, and in particular how parental attitudes, beliefs and self efficacy impact parent and child engagement in early literacy intervention activities. A mixed method procedure using quantitative and qualitative strategies was employed. A quasi-experimental research design was used. The research sample, sixty parents who were part of naturally occurring community interventions in at- risk neighbourhoods in a south-western Ontario city participated in the quantitative phase. Largely individuals whose home language was other than English, these participants were divided amongst three early literacy intervention groups, a Prescriptive Interventionist type group, a Participatory Empowering type group and a drop-in parent- child neighbourhood Control group. Measures completed pre and post a six session literacy intervention, on all three literacy and evidence of change in parental empowerment. Parents in all three groups, on average, held beliefs about early literacy that were positive and that were compatible with current approaches to language development and emergent literacy. No significant change in early literacy beliefs and attitudes for pre to post intervention was found. Similarly, there was no significant difference between groups on empowerment scores, but there was a significant change post intervention in one group's empowerment score. There was a drop in the empowerment score for the Prescriptive Interventionist type group, suggesting a drop in empowerment level. The qualitative aspect of this study involved six in-depth interviews completed with a sub-set of the sixty research participants. Four similar themes emerged across the groups: learning takes place across time and place; participation is key; success is achieved by taking small steps; and learning occurs in multiple ways. The research findings have important implications for practitioners and policy makers who target at risk populations with early intervention programming and wish to sustain parental empowerment. Study results show the value parents place on early learning and point to the importance of including parents in the development and delivery of early intervention programs. groups, were analyzed for evidence of change in parental attitudes and beliefs about early literacy and evidence of change in parental empowerment. Parents in all three groups, on average, held beliefs about early literacy that were positive and that were compatible with current approaches to language development and emergent literacy. No significant change in early literacy beliefs and attitudes for pre to post intervention was found. Similarly, there was no significant difference between groups on empowerment scores, but there was a significant change post intervention in one group's empowerment score. There was a drop in the empowerment score for the Prescriptive Interventionist type group, suggesting a drop in empowerment level. The qualitative aspect of this study involved six in-depth interviews completed with a sub-set of the sixty research participants. Four similar themes emerged across the groups: learning takes place across time and place; participation is key; success is achieved by taking small steps; and learning occurs in multiple ways. The research findings have important implications for practitioners and policy makers who target at risk populations with early intervention programming and wish to sustain parental empowerment. Study results show the value parents place on early learning and point to the importance of including parents in the development and delivery of early intervention programs.

Classed femininity, performativity, and camp in British structured reality programming

This article explores the representations and tonal qualities of British “structured reality” programming. Focusing on The Only Way Is Essex and Made in Chelsea, it investigates their glocalizing of the model established by MTV’s Laguna Beach and The Hills. It argues that while they blur boundaries between docusoap, drama, and soap opera, the British programs also recognize and foreground issues of construction for their reality TV-literate youth audience. It suggests the programs play a key role in their respective channel identities and the ideologies of British youth television, connecting to larger issues of class, gender, and taste. This is articulated through their regional and classed femininities, with the article exploring how the programs draw on classed ideologies surrounding “natural” and “excessive” femininities and of the role of this in their engagement with construction and camp play. This play contributes to the tonal shift offered by the British programs, mixing the melodrama of the MTV programs with a knowing, at times comic edge that can tip into mockery. In doing so, the programs offer their audience a combination of performative self-awareness and emotional realism that situates them clearly within British youth television

Automated Feedback for 'Fill in the Gap' Programming Exercises

Timely feedback is a vital component in the learning process. It is especially important for beginner students in Information Technology since many have not yet formed an effective internal model of a computer that they can use to construct viable knowledge. Research has shown that learning efficiency is increased if immediate feedback is provided for students. Automatic analysis of student programs has the potential to provide immediate feedback for students and to assist teaching staff in the marking process. This paper describes a “fill in the gap” programming analysis framework which tests students’ solutions and gives feedback on their correctness, detects logic errors and provides hints on how to fix these errors. Currently, the framework is being used with the Environment for Learning to Programming (ELP) system at Queensland University of Technology (QUT); however, the framework can be integrated into any existing online learning environment or programming Integrated Development Environment (IDE)

Losing their marbles : syntax-free programming for assessing problem-solving skills

Novice programmers have difficulty developing an algorithmic solution while simultaneously obeying the syntactic constraints of the target programming language. To see how students fare in algorithmic problem solving when not burdened by syntax, we conducted an experiment in which a large class of beginning programmers were required to write a solution to a computational problem in structured English, as if instructing a child, without reference to program code at all. The students produced an unexpectedly wide range of correct, and attempted, solutions, some of which had not occurred to their teachers. We also found that many common programming errors were evident in the natural language algorithms, including failure to ensure loop termination, hardwiring of solutions, failure to properly initialise the computation, and use of unnecessary temporary variables, suggesting that these mistakes are caused by inexperience at thinking algorithmically, rather than difficulties in expressing solutions as program code.

Further evidence of a relationship between explaining, tracing and writing skills in introductory programming

This paper reports on a replication of earlier studies into a possible hierarchy of programming skills. In this study, the students from whom data was collected were at a university that had not provided data for earlier studies. Also, the students were taught the programming language Python, which had not been used in earlier studies. Thus this study serves as a test of whether the findings in the earlier studies were specific to certain institutions, student cohorts, and programming languages. Also, we used a non–parametric approach to the analysis, rather than the linear approach of earlier studies. Our results are consistent with the earlier studies. We found that students who cannot trace code usually cannot explain code, and also that students who tend to perform reasonably well at code writing tasks have also usually acquired the ability to both trace code and explain code.

Seeing is understanding : the effect of visualisation in understanding programming concepts

How and why visualisations support learning was the subject of this qualitative instrumental collective case study. Five computer programming languages (PHP, Visual Basic, Alice, GameMaker, and RoboLab) supporting differing degrees of visualisation were used as cases to explore the effectiveness of software visualisation to develop fundamental computer programming concepts (sequence, iteration, selection, and modularity). Cognitive theories of visual and auditory processing, cognitive load, and mental models provided a framework in which student cognitive development was tracked and measured by thirty-one 15-17 year old students drawn from a Queensland metropolitan secondary private girls’ school, as active participants in the research. Seventeen findings in three sections increase our understanding of the effects of visualisation on the learning process. The study extended the use of mental model theory to track the learning process, and demonstrated application of student research based metacognitive analysis on individual and peer cognitive development as a means to support research and as an approach to teaching. The findings also forward an explanation for failures in previous software visualisation studies, in particular the study has demonstrated that for the cases examined, where complex concepts are being developed, the mixing of auditory (or text) and visual elements can result in excessive cognitive load and impede learning. This finding provides a framework for selecting the most appropriate instructional programming language based on the cognitive complexity of the concepts under study.

Engaging students in programming

Poor student engagement and high failure rates in first year units were addressed at the Queensland University of Technology (QUT) with a course restructure involving a fresh approach to introducing programming. Students’ first taste of programming in the new course focused less on the language and syntax, and more on problem solving and design, and the role of programming in relation to other technologies they are likely to encounter in their studies. In effect, several technologies that have historically been compartmentalised and taught in isolation have been brought together as a breadth-first introduction to IT. Incorporating databases and Web development technologies into what used to be a purely programming unit gave students a very short introduction to each technology, with programming acting as the glue between each of them. As a result, students not only had a clearer understanding of the application of programming in the real world, but were able to determine their preference or otherwise for each of the technologies introduced, which will help them when the time comes for choosing a course major. Students engaged well in an intensely collaborative learning environment for this unit which was designed to both support the needs of students and meet industry expectations. Attrition from the unit was low, with computer laboratory practical attendance rates for the first time remaining high throughout semester, and the failure rate falling to a single figure percentage.