Factors associated with prompt difficulty in automated essay scoring

This study explores factors related to the prompt difficulty in Automated Essay Scoring. The sample was composed of 6,924 students. For each student, there were 1-4 essays, across 20 different writing prompts, for a total of 20,243 essays. E-rater® v.2 essay scoring engine developed by the Educational Testing Service was used to score the essays. The scoring engine employs a statistical model that incorporates 10 predictors associated with writing characteristics of which 8 were used. The Rasch partial credit analysis was applied to the scores to determine the difficulty levels of prompts. In addition, the scores were used as outcomes in the series of hierarchical linear models (HLM) in which students and prompts constituted the cross-classification levels. This methodology was used to explore the partitioning of the essay score variance.^ The results indicated significant differences in prompt difficulty levels due to genre. Descriptive prompts, as a group, were found to be more difficult than the persuasive prompts. In addition, the essay score variance was partitioned between students and prompts. The amount of the essay score variance that lies between prompts was found to be relatively small (4 to 7 percent). When the essay-level, student-level-and prompt-level predictors were included in the model, it was able to explain almost all variance that lies between prompts. Since in most high-stakes writing assessments only 1-2 prompts per students are used, the essay score variance that lies between prompts represents an undesirable or "noise" variation. Identifying factors associated with this "noise" variance may prove to be important for prompt writing and for constructing Automated Essay Scoring mechanisms for weighting prompt difficulty when assigning essay score.^

An analysis of student and instructor perceptions of students' writing skills at the university level

This study examined university students' writing skills as perceived by university students and their English instructors. The goal of the study was to provide English instructors with objective, quantified information about writing perceptions from both the students' and instructors' viewpoints. ^ A survey instrument was developed based on a survey instrument created by Newkirk, Cameron, and Selfe (1977) to identify instructors' perceived knowledge of student writing skills. The present study used a descriptive statistical design. It examined five writing skill areas: attitude, content, grammar and mechanics, literary considerations, and the writing process through a questionnaire completed by a convenience sample of summer and fall admitted freshmen who were enrolled in Essay Writing and Freshman Composition courses and English Department instructors at a large South Florida public university. ^ The study consisted of five phases. The first phase was modifying of the Newkirk, Cameron, and Selfe (1977) questionnaire. Two versions of the revised survey were developed - one for instructors and one for students. The second phase was pilot testing the questionnaire for evaluation of administration and scoring. The third phase was administering the questionnaire to 1,280 students and 48 instructors. The fourth phase was analyzing the data. The study found a significant difference in the perceptions of students and instructors in all areas of writing skills examined by the survey. Responses to 29 of 30 questions showed that students felt they had better attitudes toward writing and better writing skills than instructors thought. ^ The final phase was developing recommendations for practice. Based on findings and theory and empirical evidence drawn from the fields of adult education and composition research, learner-centered, self-directed curriculum guidelines are offered. ^ By objectively quantifying student and instructor perceptions of students' writing skills, this study contributes to a growing body of literature that: (a) encourages instructors to acknowledge the perception disparities between instructors and students; (b) gives instructors a better understanding of how to communicate with students; and (c) recommends the development of new curriculum, placement tests, and courses that meet the needs of students and enables English instructors to provide meaningful instruction. ^

An automated method for the determination of electroreception thresholds in weakly electric fish

Weakly electric fish produce a dual function electric signal that makes them ideal models for the study of sensory computation and signal evolution. This signal, the electric organ discharge (EOD), is used for communication and navigation. In some families of gymnotiform electric fish, the EOD is a dynamic signal that increases in amplitude during social interactions. Amplitude increase could facilitate communication by increasing the likelihood of being sensed by others or by impressing prospective mates or rivals. Conversely, by increasing its signal amplitude a fish might increase its sensitivity to objects by lowering its electrolocation detection threshold. To determine how EOD modulations elicited in the social context affect electrolocation, I developed an automated and fast method for measuring electroreception thresholds using a classical conditioning paradigm. This method employs a moving shelter tube, which these fish occupy at rest during the day, paired with an electrical stimulus. A custom built and programmed robotic system presents the electrical stimulus to the fish, slides the shelter tube requiring them to follow, and records video of their movements. I trained the electric fish of the genus Sternopygus was trained to respond to a resistive stimulus on this apparatus in 2 days. The motion detection algorithm correctly identifies the responses 91% of the time, with a false positive rate of only 4%. This system allows for a large number of trials, decreasing the amount of time needed to determine behavioral electroreception thresholds. This novel method enables the evaluation the evolutionary interplay between two conflicting sensory forces, social communication and navigation.

Evaluation of the evidential value of the elemental composition of glass, ink and paper by laser-based microspectrochemical methods

Elemental analysis can become an important piece of evidence to assist the solution of a case. The work presented in this dissertation aims to evaluate the evidential value of the elemental composition of three particular matrices: ink, paper and glass. In the first part of this study, the analytical performance of LIBS and LA-ICP-MS methods was evaluated for paper, writing inks and printing inks. A total of 350 ink specimens were examined including black and blue gel inks, ballpoint inks, inkjets and toners originating from several manufacturing sources and/or batches. The paper collection set consisted of over 200 paper specimens originating from 20 different paper sources produced by 10 different plants. Micro-homogeneity studies show smaller variation of elemental compositions within a single source (i.e., sheet, pen or cartridge) than the observed variation between different sources (i.e., brands, types, batches). Significant and detectable differences in the elemental profile of the inks and paper were observed between samples originating from different sources (discrimination of 87–100% of samples, depending on the sample set under investigation and the method applied). These results support the use of elemental analysis, using LA-ICP-MS and LIBS, for the examination of documents and provide additional discrimination to the currently used techniques in document examination. In the second part of this study, a direct comparison between four analytical methods (µ-XRF, solution-ICP-MS, LA-ICP-MS and LIBS) was conducted for glass analyses using interlaboratory studies. The data provided by 21 participants were used to assess the performance of the analytical methods in associating glass samples from the same source and differentiating different sources, as well as the use of different match criteria (confidence interval (±6s, ±5s, ±4s, ±3s, ±2s), modified confidence interval, t-test (sequential univariate, p=0.05 and p=0.01), t-test with Bonferroni correction (for multivariate comparisons), range overlap, and Hotelling's T2 tests. Error rates (Type 1 and Type 2) are reported for the use of each of these match criteria and depend on the heterogeneity of the glass sources, the repeatability between analytical measurements, and the number of elements that were measured. The study provided recommendations for analytical performance-based parameters for µ-XRF and LA-ICP-MS as well as the best performing match criteria for both analytical techniques, which can be applied now by forensic glass examiners.

An evaluation of an intervention strategy for retention and determination of predictors of success for high -risk students in an associate degree nursing program

The purpose of this study was twofold: (1) to evaluate the effect of a specific instructional Intervention, a Nursing Theory Laboratory, on increasing the retention of high risk students in the associate degree nursing program at Miami-Dade Community College in Miami, Florida; and (2) to identify predictors of success of high risk nursing students in this associate degree nursing program.^ Data were collected from the 195 nursing students enrolled in Nursing Fundamentals during the 1985-1987 academic years, and identified as high risk students. Control and experimental groups were selected based on enrollment in the Nursing Theory Laboratory.^ Results were determined by analyzing several cross-tabulations of selected variables and yielding chi square values, t-tests, and two discriminant analyses. There was no significant relationship between age or ethnic background and enrollment in the Nursing Theory Laboratory. There was no significant relationship between enrollment in the Nursing Theory Laboratory and success in Nursing 1 (Nursing Fundamentals). There was a significant relationship between enrollment in the Nursing Theory Laboratory and success in Nursing 3 (Medical-Surgical Nursing). Writing assessment test scores in two entrance tests and high risk categories, based on the number of enrollments in required science courses, were identified as predictors of success in this program.^ The conclusion was that the Nursing Theory Laboratory does not significantly improve retention of high risk associate degree nursing students if they are enrolled in this intervention at the same time they are enrolled in Nursing Fundamentals. Since those students who were enrolled in the Nursing Theory Laboratory had a significantly higher success rate in Nursing 3, than those students who were not enrolled in the Nursing Theory Laboratory, a recommendation of this study was to offer the Nursing Theory Laboratory to high risk students prior to the beginning of nursing courses. Another recommendation was that students deficient in reading and writing skills should be required to enroll in developmental courses prior to enrollment in the nursing course. ^

Assessing an automated, information -sharing technology in the post "9 -11" era: Do local law enforcement officers think it meets their needs?

In the wake of the “9-11” terrorists' attacks, the U.S. Government has turned to information technology (IT) to address a lack of information sharing among law enforcement agencies. This research determined if and how information-sharing technology helps law enforcement by examining the differences in perception of the value of IT between law enforcement officers who have access to automated regional information sharing and those who do not. It also examined the effect of potential intervening variables such as user characteristics, training, and experience, on the officers' evaluation of IT. The sample was limited to 588 officers from two sheriff's offices; one of them (the study group) uses information sharing technology, the other (the comparison group) does not. Triangulated methodologies included surveys, interviews, direct observation, and a review of agency records. Data analysis involved the following statistical methods: descriptive statistics, Chi-Square, factor analysis, principal component analysis, Cronbach's Alpha, Mann-Whitney tests, analysis of variance (ANOVA), and Scheffe' post hoc analysis. ^ Results indicated a significant difference between groups: the study group perceived information sharing technology as being a greater factor in solving crime and in increasing officer productivity. The study group was more satisfied with the data available to it. As to the number of arrests made, information sharing technology did not make a difference. Analysis of the potential intervening variables revealed several remarkable results. The presence of a strong performance management imperative (in the comparison sheriff's office) appeared to be a factor in case clearances and arrests, technology notwithstanding. As to the influence of user characteristics, level of education did not influence a user's satisfaction with technology, but user-satisfaction scores differed significantly among years of experience as a law enforcement officer and the amount of computer training, suggesting a significant but weak relationship. ^ Therefore, this study finds that information sharing technology assists law enforcement officers in doing their jobs. It also suggests that other variables such as computer training, experience, and management climate should be accounted for when assessing the impact of information technology. ^

What are the effects of the 6 +1 Writing Traits on the Attitudes and Writing Skills of Fifth Grade ELLs?

In the Bahamas, ELLs consist mainly of Haitian descent students. Unfortunately, this demographic of students continuously score below their Bahamian counterparts in Creative Writing. This research examined the affects the 6 + 1 Writing Traits assessment had on the attitudes and writing abilities of fifteen, fifth grade, Haitian descent students.

Evaluation of the Evidential Value of the Elemental Composition of Glass, Ink and Paper by Laser-Based Micro-Spectrochemical Methods

Elemental analysis can become an important piece of evidence to assist the solution of a case. The work presented in this dissertation aims to evaluate the evidential value of the elemental composition of three particular matrices: ink, paper and glass. In the first part of this study, the analytical performance of LIBS and LA-ICP-MS methods was evaluated for paper, writing inks and printing inks. A total of 350 ink specimens were examined including black and blue gel inks, ballpoint inks, inkjets and toners originating from several manufacturing sources and/or batches. The paper collection set consisted of over 200 paper specimens originating from 20 different paper sources produced by 10 different plants. Micro-homogeneity studies show smaller variation of elemental compositions within a single source (i.e., sheet, pen or cartridge) than the observed variation between different sources (i.e., brands, types, batches). Significant and detectable differences in the elemental profile of the inks and paper were observed between samples originating from different sources (discrimination of 87 – 100% of samples, depending on the sample set under investigation and the method applied). These results support the use of elemental analysis, using LA-ICP-MS and LIBS, for the examination of documents and provide additional discrimination to the currently used techniques in document examination. In the second part of this study, a direct comparison between four analytical methods (µ-XRF, solution-ICP-MS, LA-ICP-MS and LIBS) was conducted for glass analyses using interlaboratory studies. The data provided by 21 participants were used to assess the performance of the analytical methods in associating glass samples from the same source and differentiating different sources, as well as the use of different match criteria (confidence interval (±6s, ±5s, ±4s, ±3s, ±2s), modified confidence interval, t-test (sequential univariate, p=0.05 and p=0.01), t-test with Bonferroni correction (for multivariate comparisons), range overlap, and Hotelling’s T2 tests. Error rates (Type 1 and Type 2) are reported for the use of each of these match criteria and depend on the heterogeneity of the glass sources, the repeatability between analytical measurements, and the number of elements that were measured. The study provided recommendations for analytical performance-based parameters for µ-XRF and LA-ICP-MS as well as the best performing match criteria for both analytical techniques, which can be applied now by forensic glass examiners.