The human factors of automatic speech recognition in control room systems

This thesis addresses the viability of automatic speech recognition for control room systems; with careful system design, automatic speech recognition (ASR) devices can be useful means for human computer interaction in specific types of task. These tasks can be defined as complex verbal activities, such as command and control, and can be paired with spatial tasks, such as monitoring, without detriment. It is suggested that ASR use be confined to routine plant operation, as opposed the critical incidents, due to possible problems of stress on the operators' speech.  It is proposed that using ASR will require operators to adapt a commonly used skill to cater for a novel use of speech. Before using the ASR device, new operators will require some form of training. It is shown that a demonstration by an experienced user of the device can lead to superior performance than instructions. Thus, a relatively cheap and very efficient form of operator training can be supplied by demonstration by experienced ASR operators. From a series of studies into speech based interaction with computers, it is concluded that the interaction be designed to capitalise upon the tendency of operators to use short, succinct, task specific styles of speech. From studies comparing different types of feedback, it is concluded that operators be given screen based feedback, rather than auditory feedback, for control room operation. Feedback will take two forms: the use of the ASR device will require recognition feedback, which will be best supplied using text; the performance of a process control task will require task feedback integrated into the mimic display. This latter feedback can be either textual or symbolic, but it is suggested that symbolic feedback will be more beneficial. Related to both interaction style and feedback is the issue of handling recognition errors. These should be corrected by simple command repetition practices, rather than use error handling dialogues. This method of error correction is held to be non intrusive to primary command and control operations. This thesis also addresses some of the problems of user error in ASR use, and provides a number of recommendations for its reduction.

Differences between theoretical predictions and operational performance in a stock control application

Over the past decade, several experienced Operational Researchers have advanced the view that the theoretical aspects of model building have raced ahead of the ability of people to use them. Consequently, the impact of Operational Research on commercial organisations and the public sector is limited, and many systems fail to achieve their anticipated benefits in full. The primary objective of this study is to examine a complex interactive Stock Control system, and identify the reasons for the differences between the theoretical expectations and the operational performance. The methodology used is to hypothesise all the possible factors which could cause a divergence between theory and practice, and to evaluate numerically the effect each of these factors has on two main control indices - Service Level and Average Stock Value. Both analytical and empirical methods are used, and simulation is employed extensively. The factors are divided into two main categories for analysis - theoretical imperfections in the model, and the usage of the system by Buyers. No evidence could be found in the literature of any previous attempts to place the differences between theory and practice in a system in quantitative perspective nor, more specifically, to study the effects of Buyer/computer interaction in a Stock Control system. The study reveals that, in general, the human factors influencing performance are of a much higher order of magnitude than the theoretical factors, thus providing objective evidence to support the original premise. The most important finding is that, by judicious intervention into an automatic stock control algorithm, it is possible for Buyers to produce results which not only attain but surpass the algorithmic predictions. However, the complexity and behavioural recalcitrance of these systems are such that an innately numerate, enquiring type of Buyer needs to be inducted to realise the performance potential of the overall man/computer system.

Edge coding in human vision: a psychophysical and computational investigation

This thesis presents a study of how edges are detected and encoded by the human visual system. The study begins with theoretical work on the development of a model of edge processing, and includes psychophysical experiments on humans, and computer simulations of these experiments, using the model. The first chapter reviews the literature on edge processing in biological and machine vision, and introduces the mathematical foundations of this area of research. The second chapter gives a formal presentation of a model of edge perception that detects edges and characterizes their blur, contrast and orientation, using Gaussian derivative templates. This model has previously been shown to accurately predict human performance in blur matching tasks with several different types of edge profile. The model provides veridical estimates of the blur and contrast of edges that have a Gaussian integral profile. Since blur and contrast are independent parameters of Gaussian edges, the model predicts that varying one parameter should not affect perception of the other. Psychophysical experiments showed that this prediction is incorrect: reducing the contrast makes an edge look sharper; increasing the blur reduces the perceived contrast. Both of these effects can be explained by introducing a smoothed threshold to one of the processing stages of the model. It is shown that, with this modification,the model can predict the perceived contrast and blur of a number of edge profiles that differ markedly from the ideal Gaussian edge profiles on which the templates are based. With only a few exceptions, the results from all the experiments on blur and contrast perception can be explained reasonably well using one set of parameters for each subject. In the few cases where the model fails, possible extensions to the model are discussed.

Social and organizational impacts of emerging mobile devices:evaluating use

As mobile devices become increasingly diverse and continue to shrink in size and weight, their portability is enhanced but, unfortunately, their usability tends to suffer. Ultimately, the usability of mobile technologies determines their future success in terms of end-user acceptance and, thereafter, adoption and social impact. Widespread acceptance will not, however, be achieved if users’ interaction with mobile technology amounts to a negative experience. Mobile user interfaces need to be designed to meet the functional and sensory needs of users. Social and Organizational Impacts of Emerging Mobile Devices: Evaluating Use focuses on human-computer interaction related to the innovation and research in the design, evaluation, and use of innovative handheld, mobile, and wearable technologies in order to broaden the overall body of knowledge regarding such issues. It aims to provide an international forum for researchers, educators, and practitioners to advance knowledge and practice in all facets of design and evaluation of human interaction with mobile technologies.

Handbook of research on user interface design and evaluation for mobile technology

In recent years, mobile technology has been one of the major growth areas in computing. Designing the user interface for mobile applications, however, is a very complex undertaking which is made even more challenging by the rapid technological developments in mobile hardware. Mobile human-computer interaction, unlike desktop-based interaction, must be cognizant of a variety of complex contextual factors affecting both users and technology. The Handbook of Research on User Interface Design and Evaluation provides students, researchers, educators, and practitioners with a compendium of research on the key issues surrounding the design and evaluation of mobile user interfaces, such as the physical environment and social context in which a mobile device is being used and the impact of multitasking behavior typically exhibited by mobile-device users. Compiling the expertise of over 150 leading experts from 26 countries, this exemplary reference tool will make an indispensable addition to every library collection.

Pattern integration in the normal and abnormal human visual system

The processing conducted by the visual system requires the combination of signals that are detected at different locations in the visual field. The processes by which these signals are combined are explored here using psychophysical experiments and computer modelling. Most of the work presented in this thesis is concerned with the summation of contrast over space at detection threshold. Previous investigations of this sort have been confounded by the inhomogeneity in contrast sensitivity across the visual field. Experiments performed in this thesis find that the decline in log contrast sensitivity with eccentricity is bilinear, with an initial steep fall-off followed by a shallower decline. This decline is scale-invariant for spatial frequencies of 0.7 to 4 c/deg. A detailed map of the inhomogeneity is developed, and applied to area summation experiments both by incorporating it into models of the visual system and by using it to compensate stimuli in order to factor out the effects of the inhomogeneity. The results of these area summation experiments show that the summation of contrast over area is spatially extensive (occurring over 33 stimulus carrier cycles), and that summation behaviour is the same in the fovea, parafovea, and periphery. Summation occurs according to a fourth-root summation rule, consistent with a “noisy energy” model. This work is extended to investigate the visual deficit in amblyopia, finding that area summation is normal in amblyopic observers. Finally, the methods used to study the summation of threshold contrast over area are adapted to investigate the integration of coherent orientation signals in a texture. The results of this study are described by a two-stage model, with a mandatory local combination stage followed by flexible global pooling of these local outputs. In each study, the results suggest a more extensive combination of signals in vision than has been previously understood.

“Showing off” your mobile device:adult literacy learning in the classroom and beyond

For a very large number of adults, tasks such as reading. understanding, and using everyday items are a challenge. Although many community-based organizations offer resources and support for adults with limited literacy skills. current programs have difficulty reaching and retaining those that would benefit most. In this paper we present the findings of an exploratory study aimed at investigating how a technological solution that addresses these challenges is received and adopted by adult learners. For this, we have developed a mobile application to support literacy programs and to assist low-literacy adults in today's information-centric society. ALEX© (Adult Literacy support application for Experiential learning) is a mobile language assistant that is designed to be used both in the classroom and in daily life in order to help low-literacy adults become increasingly literate and independent. Through a long-term study with adult learners we show that such a solution complements literacy programs by increasing users' motivation and interest in learning, and raising their confidence levels both in their education pursuits and in facing the challenges of their daily lives.

Designing a mobile diet diary application with and for older adults with AMD:a case ctudy

Age-Related Macular Degeneration (AMD) is the UK’s leading cause of severe visual impairment amongst the elderly. It accounts for 16,000 blind/partial sight registrations per year and is the leading cause of blindness among people aged 55 years and older in western countries (Bressler, 2004). Our research aims to design and develop a self-monitoring, ability-reactive technology (SMART) for users with AMD to support their dietary-based AMD risk mitigation and progression retardation over time. In this paper, we reflect on our experience of adapting and applying a participatory design (PD) approach to support the effective design of our application with and for older adults with AMD. We introduce the outcome of a series of PD sessions with older adults with AMD - that is, a paper prototype of our proposed application which focuses on accessibility for our target users - and discuss implications for the eventual prototype development

Interface Engineering and Design: Adaptibility Problems

* The research work reviewed in this paper has been carried out in the context of the Russian Foundation for Basic Research funded project “Adaptable Intelligent Interfaces Research and Development for Distance Learning Systems”(grant N 02-01-81019). The authors wish to acknowledge the co-operation with the Byelorussian partners of this project.

Emerging perspectives on the design, use, and evaluation of mobile and handheld devices

Human-computer interaction is a growing field of study in which researchers and professionals aim to understand and evaluate the impact of new technologies on human behavior. With the integration of smart phones, tablets, and other portable devices into everyday life, there is a greater need to understand the influence of such technology on the human experience. Emerging Perspectives on the Design, Use, and Evaluation of Mobile and Handheld Devices is an authoritative reference source consisting of the latest scholarly research and theories from international experts and professionals on the topic of human-computer interaction with mobile devices. Featuring a comprehensive collection of chapters on critical topics in this dynamic field, this publication is an essential reference source for researchers, educators, students, and practitioners interested in the use of mobile and handheld devices and their impact on individuals and society as a whole. This publication features timely, research-based chapters pertaining to topics in the design and evaluation of smart devices including, but not limited to, app stores, category-based interfaces, gamified mobility applications, mobile interaction, mobile learning, pervasive multimodal applications, smartphone interaction, and social media use.

A longitudinal evaluation of the acceptability and impact of a diet diary app for older adults with age-related macular degeneration

Ongoing advances in technology are increasing the scope for enhancing and supporting older adults’ daily living. The digital divide between older and younger adults raises concerns, however, about the suitability of technological solutions for older adults, especially for those with impairments. Taking older adults with Age-Related Macular Degeneration (AMD) as a case study, we used user-centred and participatory design approaches to develop an assistive mobile app for self-monitoring their intake of food [12,13]. In this paper we report on findings of a longitudinal field evaluation of our app that was conducted to investigate how it was received and adopted by older adults with AMD and its impact on their lives. Demonstrating the benefit of applying inclusive design methods for technology for older adults, our findings reveal how the use of the app raises participants’ awareness and facilitates self-monitoring of diet, encourages positive (diet) behaviour change, and encourages learning.

A digital signal processing approach for affective sensing of a computer user through pupil diameter monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

An assessment of natural and human disturbance effects on Mexican ecosystems: current trends and research gaps

Mexico harbors more than 10% of the planet’s endemic species. However, the integrity and biodiversity of many ecosystems is experiencing rapid transformation under the influence of a wide array of human and natural disturbances. In order to disentangle the effects of human and natural disturbance regimes at different spatial and temporal scales, we selected six terrestrial (temperate montane forests, montane cloud forests, tropical rain forests, tropical semi-deciduous forests, tropical dry forests, and deserts) and four aquatic (coral reefs, mangrove forests, kelp forests and saline lakes) ecosystems. We used semiquantitative statistical methods to assess (1) the most important agents of disturbance affecting the ecosystems, (2) the vulnerability of each ecosystem to anthropogenic and natural disturbance, and (3) the differences in ecosystem disturbance regimes and their resilience. Our analysis indicates a significant variation in ecological responses, recovery capacity, and resilience among ecosystems. The constant and widespread presence of human impacts on both terrestrial and aquatic ecosystems is reflected either in reduced area coverage for most systems, or reduced productivity and biodiversity, particularly in the case of fragile ecosystems (e.g., rain forests, coral reefs). In all cases, the interaction between historical human impacts and episodic high intensity natural disturbance (e.g., hurricanes, fires) has triggered a reduction in species diversity and induced significant changes in habitat distribution or species dominance. The lack of monitoring programs assessing before/after effects of major disturbances in Mexico is one of the major limitations to quantifying the commonalities and differences of disturbance effects on ecosystem properties.

A Digital Signal Processing Approach for Affective Sensing of a Computer User through Pupil Diameter Monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.