Factors associated with prompt difficulty in automated essay scoring

This study explores factors related to the prompt difficulty in Automated Essay Scoring. The sample was composed of 6,924 students. For each student, there were 1-4 essays, across 20 different writing prompts, for a total of 20,243 essays. E-rater® v.2 essay scoring engine developed by the Educational Testing Service was used to score the essays. The scoring engine employs a statistical model that incorporates 10 predictors associated with writing characteristics of which 8 were used. The Rasch partial credit analysis was applied to the scores to determine the difficulty levels of prompts. In addition, the scores were used as outcomes in the series of hierarchical linear models (HLM) in which students and prompts constituted the cross-classification levels. This methodology was used to explore the partitioning of the essay score variance.^ The results indicated significant differences in prompt difficulty levels due to genre. Descriptive prompts, as a group, were found to be more difficult than the persuasive prompts. In addition, the essay score variance was partitioned between students and prompts. The amount of the essay score variance that lies between prompts was found to be relatively small (4 to 7 percent). When the essay-level, student-level-and prompt-level predictors were included in the model, it was able to explain almost all variance that lies between prompts. Since in most high-stakes writing assessments only 1-2 prompts per students are used, the essay score variance that lies between prompts represents an undesirable or "noise" variation. Identifying factors associated with this "noise" variance may prove to be important for prompt writing and for constructing Automated Essay Scoring mechanisms for weighting prompt difficulty when assigning essay score.^

A multi-method approach for the assessment of composite indices and rankings

There is growing popularity in the use of composite indices and rankings for cross-organizational benchmarking. However, little attention has been paid to alternative methods and procedures for the computation of these indices and how the use of such methods may impact the resulting indices and rankings. This dissertation developed an approach for assessing composite indices and rankings based on the integration of a number of methods for aggregation, data transformation and attribute weighting involved in their computation. The integrated model developed is based on the simulation of composite indices using methods and procedures proposed in the area of multi-criteria decision making (MCDM) and knowledge discovery in databases (KDD). The approach developed in this dissertation was automated through an IT artifact that was designed, developed and evaluated based on the framework and guidelines of the design science paradigm of information systems research. This artifact dynamically generates multiple versions of indices and rankings by considering different methodological scenarios according to user specified parameters. The computerized implementation was done in Visual Basic for Excel 2007. Using different performance measures, the artifact produces a number of excel outputs for the comparison and assessment of the indices and rankings. In order to evaluate the efficacy of the artifact and its underlying approach, a full empirical analysis was conducted using the World Bank's Doing Business database for the year 2010, which includes ten sub-indices (each corresponding to different areas of the business environment and regulation) for 183 countries. The output results, which were obtained using 115 methodological scenarios for the assessment of this index and its ten sub-indices, indicated that the variability of the component indicators considered in each case influenced the sensitivity of the rankings to the methodological choices. Overall, the results of our multi-method assessment were consistent with the World Bank rankings except in cases where the indices involved cost indicators measured in per capita income which yielded more sensitive results. Low income level countries exhibited more sensitivity in their rankings and less agreement between the benchmark rankings and our multi-method based rankings than higher income country groups.

VHITS: Vertical handoff initiation and target selection in a heterogeneous wireless network

Global connectivity, for anyone, at anyplace, at anytime, to provide high-speed, high-quality, and reliable communication channels for mobile devices, is now becoming a reality. The credit mainly goes to the recent technological advances in wireless communications comprised of a wide range of technologies, services, and applications to fulfill the particular needs of end-users in different deployment scenarios (Wi-Fi, WiMAX, and 3G/4G cellular systems). In such a heterogeneous wireless environment, one of the key ingredients to provide efficient ubiquitous computing with guaranteed quality and continuity of service is the design of intelligent handoff algorithms. Traditional single-metric handoff decision algorithms, such as Received Signal Strength (RSS) based, are not efficient and intelligent enough to minimize the number of unnecessary handoffs, decision delays, and call-dropping and/or blocking probabilities. This research presented a novel approach for the design and implementation of a multi-criteria vertical handoff algorithm for heterogeneous wireless networks. Several parallel Fuzzy Logic Controllers were utilized in combination with different types of ranking algorithms and metric weighting schemes to implement two major modules: the first module estimated the necessity of handoff, and the other module was developed to select the best network as the target of handoff. Simulations based on different traffic classes, utilizing various types of wireless networks were carried out by implementing a wireless test-bed inspired by the concept of Rudimentary Network Emulator (RUNE). Simulation results indicated that the proposed scheme provided better performance in terms of minimizing the unnecessary handoffs, call dropping, and call blocking and handoff blocking probabilities. When subjected to Conversational traffic and compared against the RSS-based reference algorithm, the proposed scheme, utilizing the FTOPSIS ranking algorithm, was able to reduce the average outage probability of MSs moving with high speeds by 17%, new call blocking probability by 22%, the handoff blocking probability by 16%, and the average handoff rate by 40%. The significant reduction in the resulted handoff rate provides MS with efficient power consumption, and more available battery life. These percentages indicated a higher probability of guaranteed session continuity and quality of the currently utilized service, resulting in higher user satisfaction levels.

Integrated Ecological Modeling and Decision Analysis Within the Everglades Landscape

Planning for complex ecosystem restoration projects involves integrating ecological modeling with analysis of performance trade-offs among restoration alternatives. The authors used the Everglades Landscape Model and Multi-Criteria Decision Analysis to explore the effect of simulated ecosystem performance, risk preferences, and criteria weights on the ranking of three alternatives to restoring overland sheet flow in the Everglades. The ecological model outputs included both hydrologic and water quality criteria. Results were scored in the decision analysis framework, highlighting the trade-offs between hydrologic restoration and water quality constraints. Given equal weighting of performance measures, the alternative with more homogenous sheet flow was preferred over other alternatives, despite evidence of some localized eutrophication risk.

Intuition and its component parts in the heuristic processing of deception detection: Hunch, *evidence, and reliance on intuitive efficacy

Rated trust in intuitive efficacy (measured as trust, belief, use, accuracy and weighting of intuition) was investigated as a predictor of self-designated use of intuitive (hunch and hunch plus evidential belief) vs. deliberative (evidential belief and evidential belief plus hunch) deception detection judgments and actual accuracy. Twenty-nine student participants were filmed as they made true and deceptive statements about their everyday activities on a given evening (last Friday night), and college students (N=238) judged 20 (10=true, 10=deceptive) of these filmed statements as truthful or deceptive. Participants provided ratings of reliance on hunches vs. evidential belief, confidence in film judgments, intuitive efficacy, accuracy in deception detection, reliance on cues to deception, and experiences with intuition. Generalized estimated equation modeling using binary logistics demonstrated accuracy in identifying true vs. deceptive statements was predicted by film number, hunch-evidence ratings, weighting of intuition, and total cues cited. Weighting of intuition was predictive of accuracy across participants, with higher weighting predictive of higher accuracy in general. Participants who cited evidential belief plus hunch and moderate to high weighting incorrectly reversed their true vs. deceptive judgments. Accuracy for true statements was higher for hunches and hunch plus evidential belief, whereas accuracy for deceptive statements was higher for evidential belief Accuracy for participants who relied on evidential belief plus hunch was at chance. Subjective experiences underlying judgments differed by participant and type of film viewed (true vs. deceptive) and were predicted by hunch-evidence ratings, trust, use, intuitive accuracy, and total cues cited. Trust predicted increases in judging films to be true, whereas use and accuracy predicted increases in judging films as deceptive; none were predictive of accuracy. Increased number of cues cited predicted judgments of deception, whereas decreased number of cues cited predicted truth. The study concluded that participants have the capacity to self-define their judgments as subjectively vs. deliberately based, provide subjective assessments of the influence of intuitive vs. objective information on their judgments, and can apply this self-knowledge, through effective weighting of intuition vs. other types of information, in making accurate judgments of true and deceptive everyday statements.