A Visual Analytics Approach to Comparing Cohorts of Event Sequences

Sequences of timestamped events are currently being generated across nearly every domain of data analytics, from e-commerce web logging to electronic health records used by doctors and medical researchers. Every day, this data type is reviewed by humans who apply statistical tests, hoping to learn everything they can about how these processes work, why they break, and how they can be improved upon. To further uncover how these processes work the way they do, researchers often compare two groups, or cohorts, of event sequences to find the differences and similarities between outcomes and processes. With temporal event sequence data, this task is complex because of the variety of ways single events and sequences of events can differ between the two cohorts of records: the structure of the event sequences (e.g., event order, co-occurring events, or frequencies of events), the attributes about the events and records (e.g., gender of a patient), or metrics about the timestamps themselves (e.g., duration of an event). Running statistical tests to cover all these cases and determining which results are significant becomes cumbersome. Current visual analytics tools for comparing groups of event sequences emphasize a purely statistical or purely visual approach for comparison. Visual analytics tools leverage humans' ability to easily see patterns and anomalies that they were not expecting, but is limited by uncertainty in findings. Statistical tools emphasize finding significant differences in the data, but often requires researchers have a concrete question and doesn't facilitate more general exploration of the data. Combining visual analytics tools with statistical methods leverages the benefits of both approaches for quicker and easier insight discovery. Integrating statistics into a visualization tool presents many challenges on the frontend (e.g., displaying the results of many different metrics concisely) and in the backend (e.g., scalability challenges with running various metrics on multi-dimensional data at once). I begin by exploring the problem of comparing cohorts of event sequences and understanding the questions that analysts commonly ask in this task. From there, I demonstrate that combining automated statistics with an interactive user interface amplifies the benefits of both types of tools, thereby enabling analysts to conduct quicker and easier data exploration, hypothesis generation, and insight discovery. The direct contributions of this dissertation are: (1) a taxonomy of metrics for comparing cohorts of temporal event sequences, (2) a statistical framework for exploratory data analysis with a method I refer to as high-volume hypothesis testing (HVHT), (3) a family of visualizations and guidelines for interaction techniques that are useful for understanding and parsing the results, and (4) a user study, five long-term case studies, and five short-term case studies which demonstrate the utility and impact of these methods in various domains: four in the medical domain, one in web log analysis, two in education, and one each in social networks, sports analytics, and security. My dissertation contributes an understanding of how cohorts of temporal event sequences are commonly compared and the difficulties associated with applying and parsing the results of these metrics. It also contributes a set of visualizations, algorithms, and design guidelines for balancing automated statistics with user-driven analysis to guide users to significant, distinguishing features between cohorts. This work opens avenues for future research in comparing two or more groups of temporal event sequences, opening traditional machine learning and data mining techniques to user interaction, and extending the principles found in this dissertation to data types beyond temporal event sequences.

An Analysis of the Implementation and Perceived Effectiveness of the SchoolMAX Family Portal

ABSTRACT Title of Document: AN ANALYSIS OF THE IMPLEMENTATION AND PERCEIVED EFFECTIVENESS OF THE SCHOOLMAX FAMILY PORTAL Warren Wesley Watts, Doctor of Education, 2015 Directed By: Margaret J. McLaughlin, Ph.D. Department of Counseling, Higher Education and Special Education School districts have spent millions of dollars implementing student information systems that offer family portals with web-based access to parents and students. One of the main purposes of these systems is to improve school-to-home communication. Research has shown that when school-to-home communication is implemented effectively, parent involvement improves and student achievement increases (Epstein, 2001). The purpose of the study was to (a) understand why parents used or refrained from using the family portal and (b) determine what barriers to use might exist. To this end, this descriptive study identified the information parent users accessed in the SchoolMAX family portal, determined how frequently parents accessed the portal, and ascertained whether parents perceived an increase in communication with their children about academic matters after they began accessing the portal. Finally, the study sought to identify whether barriers existed that prevented parents from using the family portal. The inquiry employed three data sources to answer the aforementioned queries. These sources included (a) a survey sent electronically to 19,108 parents who registered online for the SchoolMAX family portal; (b) SchoolMAX portal usage data from the student information system for system usage between January 1, 2015 and June 30, 2015; and (c) a paper survey sent to 691 parents of students that had never used the SchoolMAX family portal in one elementary school, one middle school and one high school that were representative of other schools in the district. Survey results indicated that parents at all grade levels used the family portal. Usage data also confirmed that approximately 19% of the students had parents who monitored their progress through the family portal. Usage data also showed that parents were monitoring approximately 25% of students in secondary schools (6th – 12th grade) and 16% of students in elementary schools. Of the wide menu of resources available through the SchoolMAX family portal, parents used three areas most frequently: attendance, daily grades, and report cards. Approximately 70% of parents responded that their communication had improved with their children about academic matters since they started using the SchoolMAX family portal, and 90% of parents responded that the SchoolMAX family portal was an effective or somewhat effective tool. Parents also expressed interest in the addition of additional information to the SchoolMAX family portal. Specifically, the top three additions parents wanted to see included homework assignments, high stakes test scores, and graduation requirements. Parents also reported that 92% of them spoke to their children at least 2 to 3 times per week about academics. Due to the low response rate of the parent non-user survey, potential barriers to using the SchoolMAX family portal could not be addressed in this study. However, this issue may be a useful research topic in a future study. Keywords: school to home communication, student information systems, family portal, parent portal

User Behavioral Modeling of Web-based Systems for Continuous User Authentication

Authentication plays an important role in how we interact with computers, mobile devices, the web, etc. The idea of authentication is to uniquely identify a user before granting access to system privileges. For example, in recent years more corporate information and applications have been accessible via the Internet and Intranet. Many employees are working from remote locations and need access to secure corporate files. During this time, it is possible for malicious or unauthorized users to gain access to the system. For this reason, it is logical to have some mechanism in place to detect whether the logged-in user is the same user in control of the user's session. Therefore, highly secure authentication methods must be used. We posit that each of us is unique in our use of computer systems. It is this uniqueness that is leveraged to "continuously authenticate users" while they use web software. To monitor user behavior, n-gram models are used to capture user interactions with web-based software. This statistical language model essentially captures sequences and sub-sequences of user actions, their orderings, and temporal relationships that make them unique by providing a model of how each user typically behaves. Users are then continuously monitored during software operations. Large deviations from "normal behavior" can possibly indicate malicious or unintended behavior. This approach is implemented in a system called Intruder Detector (ID) that models user actions as embodied in web logs generated in response to a user's actions. User identification through web logs is cost-effective and non-intrusive. We perform experiments on a large fielded system with web logs of approximately 4000 users. For these experiments, we use two classification techniques; binary and multi-class classification. We evaluate model-specific differences of user behavior based on coarse-grain (i.e., role) and fine-grain (i.e., individual) analysis. A specific set of metrics are used to provide valuable insight into how each model performs. Intruder Detector achieves accurate results when identifying legitimate users and user types. This tool is also able to detect outliers in role-based user behavior with optimal performance. In addition to web applications, this continuous monitoring technique can be used with other user-based systems such as mobile devices and the analysis of network traffic.

An Analysis of Principal Attrition in a Large Urban School District

Principal attrition is a national problem particularly in large urban school districts. Research confirms that schools that serve high proportions of children living in poverty have the most difficulty attracting and retaining competent school leaders. Principals who are at the helm of high poverty schools have a higher turnover rate than the national average of three to four years and higher rates of teacher attrition. This leadership turnover has a fiscal impact on districts and negatively affects student achievement. Research identifies a myriad of reasons why administrators leave the role of principal: some leave the position for retirement; some exit based on difficulty of the role and lack of support; and some simply leave for other opportunities within and outside of the profession altogether. As expectations for both teacher and learner performance drive the national education agenda, understanding how to keep effective principals in their jobs is critical. This study examined the factors that principals in a large urban district identified as potentially affecting their decisions to stay in the position. The study utilized a multi-dimensional, web-based questionnaire to examine principals’ perceptions regarding contributing factors that impact tenure. Results indicated that: • having a quality teaching staff and establishing a positive work-life balance were important stay factors for principals; • having an effective supervisor and collegial support from other principals, were helpful supports; and • having adequate resources, time for long-term planning, and teacher support and resources were critical working conditions. Taken together, these indicators were the most frequently cited factors that would keep principals in their positions. The results were used to create a framework that may serve as a potential guide for addressing principal retention.

THE FEASIBILITY OF USING WEB-BASED TECHNOLOGY FOR THE MANAGEMENT OF DREDGING PROJECTS

The research investigates the feasibility of using web-based project management systems for dredging. To achieve this objective the research assessed both the positive and negative aspects of using web-based technology for the management of dredging projects. Information gained from literature review and prior investigations of dredging projects revealed that project performance, social, political, technical, and business aspects of the organization were important factors in deciding to use web-based systems for the management of dredging projects. These factors were used to develop the research assumptions. An exploratory case study methodology was used to gather the empirical evidence and perform the analysis. An operational prototype of the system was developed to help evaluate developmental and functional requirements, as well as the influence on performance, and on the organization. The evidence gathered from three case study projects, and from a survey of 31 experts, were used to validate the assumptions. Baselines, representing the assumptions, were created as a reference to assess the responses and qualitative measures. The deviation of the responses was used to evaluate for the analysis. Finally, the conclusions were assessed by validating the assumptions with the evidence, derived from the analysis. The research findings are as follows: 1. The system would help improve project performance. 2. Resistance to implementation may be experienced if the system is implemented. Therefore, resistance to implementation needs to be investigated further and more R&D work is needed in order to advance to the final design and implementation. 3. System may be divided into standalone modules in order to simplify the system and facilitate incremental changes. 4. The QA/QC conceptual approach used by this research needs to be redefined during future R&D to satisfy both owners and contractors. Yin (2009) Case Study Research Design and Methods was used to develop the research approach, design, data collection, and analysis. Markus (1983) Resistance Theory was used during the assumptions definition to predict potential problems to the implementation of web-based project management systems for the dredging industry. Keen (1981) incremental changes and facilitative approach tactics were used as basis to classify solutions, and how to overcome resistance to implementation of the web-based project management system. Davis (1989) Technology Acceptance Model (TAM) was used to assess the solutions needed to overcome the resistances to the implementation of web-base management systems for dredging projects.