An Empirical Study of Fault Localization for End-User Programmers

End users develop more software than any other group of programmers, using software authoring devices such as e-mail filtering editors, by-demonstration macro builders, and spreadsheet environments. Despite this, there has been little research on finding ways to help these programmers with the dependability of their software. We have been addressing this problem in several ways, one of which includes supporting end-user debugging activities through fault localization techniques. This paper presents the results of an empirical study conducted in an end-user programming environment to examine the impact of two separate factors in fault localization techniques that affect technique effectiveness. Our results shed new insights into fault localization techniques for end-user programmers and the factors that affect them, with significant implications for the evaluation of those techniques.

Directed Test Suite Augmentation

As software evolves, engineers use regression testing to evaluate its fitness for release. Such testing typically begins with existing test cases, and many techniques have been proposed for reusing these cost-effectively. After reusing test cases, however, it is also important to consider code or behavior that has not been exercised by existing test cases and generate new test cases to validate these. This process is known as test suite augmentation. In this paper we present a directed test suite augmentation technique, that utilizes results from reuse of existing test cases together with an incremental concolic testing algorithm to augment test suites so that they are coverage-adequate for a modified program. We present results of an empirical study examining the effectiveness of our approach.

A Controlled Experiment Assessing Test Case Prioritization Techniques via Mutation Faults

Regression testing is an important part of software maintenance, but it can also be very expensive. To reduce this expense, software testers may prioritize their test cases so that those that are more important are run earlier in the regression testing process. Previous work has shown that prioritization can improve a test suite’s rate of fault detection, but the assessment of prioritization techniques has been limited to hand-seeded faults, primarily due to the belief that such faults are more realistic than automatically generated (mutation) faults. A recent empirical study, however, suggests that mutation faults can be representative of real faults. We have therefore designed and performed a controlled experiment to assess the ability of prioritization techniques to improve the rate of fault detection techniques, measured relative to mutation faults. Our results show that prioritization can be effective relative to the faults considered, and they expose ways in which that effectiveness can vary with characteristics of faults and test suites. We also compare our results to those collected earlier with respect to the relationship between hand-seeded faults and mutation faults, and the implications this has for researchers performing empirical studies of prioritization.

Helping End-Users “Engineer” Dependable Web Applications

End-user programmers are increasingly relying on web authoring environments to create web applications. Although often consisting primarily of web pages, such applications are increasingly going further, harnessing the content available on the web through “programs” that query other web applications for information to drive other tasks. Unfortunately, errors can be pervasive in web applications, impacting their dependability. This paper reports the results of an exploratory study of end-user web application developers, performed with the aim of exposing prevalent classes of errors. The results suggest that end-users struggle the most with the identification and manipulation of variables when structuring requests to obtain data from other web sites. To address this problem, we present a family of techniques that help end user programmers perform this task, reducing possible sources of error. The techniques focus on simplification and characterization of the data that end-users must analyze while developing their web applications. We report the results of an empirical study in which these techniques are applied to several popular web sites. Our results reveal several potential benefits for end-users who wish to “engineer” dependable web applications.

Protein Hydrolysates Are Avoided by Herbivores but Not by Omnivores in Two-Choice Preference Tests

Background: The negative sensory properties of casein hydrolysates (HC) often limit their usage in products intended for human consumption, despite HC being nutritious and having many functional benefits. Recent, but taxonomically limited, evidence suggests that other animals also avoid consuming HC when alternatives exist. Methodology/Principal Findings: We evaluated ingestive responses of five herbivorous species (guinea pig, mountain beaver, gopher, vole, and rabbit) and five omnivorous species (rat, coyote, house mouse, white-footed mouse, and deer mouse; N = 16–18/species) using solid foods containing 20% HC in a series of two-choice preference tests that used a nonprotein, cellulose-based alternative. Individuals were also tested with collagen hydrolysate (gelatin; GE) to determine whether it would induce similar ingestive responses to those induced by HC. Despite HC and GE having very different nutritional and sensory qualities, both hydrolysates produced similar preference score patterns. We found that the herbivores generally avoided the hydrolysates while the omnivores consumed them at similar levels to the cellulose diet or, more rarely, preferred them (HC by the white-footed mouse; GE by the rat). Follow-up preference tests pairing HC and the nutritionally equivalent intact casein (C) were performed on the three mouse species and the guinea pigs. For the mice, mean HC preference scores were lower in the HC v C compared to the HC v Cel tests, indicating that HC’s sensory qualities negatively affected its consumption. However, responses were species-specific. For the guinea pigs, repeated exposure to HC or C (4.7-h sessions; N = 10) were found to increase subsequent HC preference scores in an HC v C preference test, which was interpreted in the light of conservative foraging strategies thought to typify herbivores. Conclusions/Significance: This is the first empirical study of dietary niche-related taxonomic differences in ingestive responses to protein hydrolysates using multiple species under comparable conditions. Our results provide a basis for future work in sensory, physiological, and behavioral mechanisms of hydrolysate avoidance and on the potential use of hydrolysates for pest management.

Analysis and Transformation of Pipe-like Web Mashups for End User Programmers

Mashups are becoming increasingly popular as end users are able to easily access, manipulate, and compose data from several web sources. To support end users, communities are forming around mashup development environments that facilitate sharing code and knowledge. We have observed, however, that end user mashups tend to suffer from several deficiencies, such as inoperable components or references to invalid data sources, and that those deficiencies are often propagated through the rampant reuse in these end user communities. In this work, we identify and specify ten code smells indicative of deficiencies we observed in a sample of 8,051 pipe-like web mashups developed by thousands of end users in the popular Yahoo! Pipes environment. We show through an empirical study that end users generally prefer pipes that lack those smells, and then present eleven specialized refactorings that we designed to target and remove the smells. Our refactorings reduce the complexity of pipes, increase their abstraction, update broken sources of data and dated components, and standardize pipes to fit the community development patterns. Our assessment on the sample of mashups shows that smells are present in 81% of the pipes, and that the proposed refactorings can reduce that number to 16%, illustrating the potential of refactoring to support thousands of end users developing pipe-like mashups.