Signal processing of physiological signals for automated assessment of stress in computer users

The need to provide computers with the ability to distinguish the affective state of their users is a major requirement for the practical implementation of affective computing concepts. This dissertation proposes the application of signal processing methods on physiological signals to extract from them features that can be processed by learning pattern recognition systems to provide cues about a person's affective state. In particular, combining physiological information sensed from a user's left hand in a non-invasive way with the pupil diameter information from an eye-tracking system may provide a computer with an awareness of its user's affective responses in the course of human-computer interactions. In this study an integrated hardware-software setup was developed to achieve automatic assessment of the affective status of a computer user. A computer-based "Paced Stroop Test" was designed as a stimulus to elicit emotional stress in the subject during the experiment. Four signals: the Galvanic Skin Response (GSR), the Blood Volume Pulse (BVP), the Skin Temperature (ST) and the Pupil Diameter (PD), were monitored and analyzed to differentiate affective states in the user. Several signal processing techniques were applied on the collected signals to extract their most relevant features. These features were analyzed with learning classification systems, to accomplish the affective state identification. Three learning algorithms: Naïve Bayes, Decision Tree and Support Vector Machine were applied to this identification process and their levels of classification accuracy were compared. The results achieved indicate that the physiological signals monitored do, in fact, have a strong correlation with the changes in the emotional states of the experimental subjects. These results also revealed that the inclusion of pupil diameter information significantly improved the performance of the emotion recognition system. ^

Signal processing and information fusing algorithms for the synthesis of an alternative electromyogram/eye gaze tracking computer cursor control system

This research pursued the conceptualization and real-time verification of a system that allows a computer user to control the cursor of a computer interface without using his/her hands. The target user groups for this system are individuals who are unable to use their hands due to spinal dysfunction or other afflictions, and individuals who must use their hands for higher priority tasks while still requiring interaction with a computer. ^ The system receives two forms of input from the user: Electromyogram (EMG) signals from muscles in the face and point-of-gaze coordinates produced by an Eye Gaze Tracking (EGT) system. In order to produce reliable cursor control from the two forms of user input, the development of this EMG/EGT system addressed three key requirements: an algorithm was created to accurately translate EMG signals due to facial movements into cursor actions, a separate algorithm was created that recognized an eye gaze fixation and provided an estimate of the associated eye gaze position, and an information fusion protocol was devised to efficiently integrate the outputs of these algorithms. ^ Experiments were conducted to compare the performance of EMG/EGT cursor control to EGT-only control and mouse control. These experiments took the form of two different types of point-and-click trials. The data produced by these experiments were evaluated using statistical analysis, Fitts' Law analysis and target re-entry (TRE) analysis. ^ The experimental results revealed that though EMG/EGT control was slower than EGT-only and mouse control, it provided effective hands-free control of the cursor without a spatial accuracy limitation, and it also facilitated a reliable click operation. This combination of qualities is not possessed by either EGT-only or mouse control, making EMG/EGT cursor control a unique and practical alternative for a user's cursor control needs. ^

A digital signal processing approach for affective sensing of a computer user through pupil diameter monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

A Digital Signal Processing Approach for Affective Sensing of a Computer User through Pupil Diameter Monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

Immunophylogenetic aspects of a gorgonian coral

One goal of comparative immunology is to derive inferences about evolutionary pathways in the development of immune-defense systems. Almost 700 million years ago, a major divergence occurred in the phylogeny of animals, spitting all descendants into either the protostome or deuterostome (includes vertebrates) lineages. Genes have evolved independently along these lineages for that amount of time. Cnidarians originated before that divergence event, and can hold clues as to which immune response genes are homologous to both lineages. This work uses the gorgonian coral, Swiftia exserta, for two major reasons: (1) because of their phylogenetic position, corals are an important animal model in studies concerning the phylogeny of immune-response genes, and (2) nothing is known about the genes controlling immunocompetence in corals. The work described here has important implications in both innate and adaptive immunity. ^ The vertebrate complement system is a major component of innate immunity. C3 is a critical component of the three pathways of complement. Because of its opsonic properties, a C3-like protein is expected to have evolved early. However, currently available data suggests that complement-like components are unique to the deuterostome lineage. This work describes the cloning and characterization of a C3-like gene from S. exserta. The deduced polypeptide sequence reveals conservation of multiple, functionally critical, sites while sharing physiochemical and structural properties with the complement components C3/C4/C5. ^ Antigen processing, via intracellular enzymatic proteasomes, is a major requirement of vertebrate adaptive immunity. These organelles have a catalytic core, through which pass intracellular proteins for degradation into peptides presentable to the immune system. LMP 7 is one component of the paralogous “immuno-proteasome”. LMP 7 is a paralog of the ubiquitous LMP X, but is restricted to vertebrates. While LMP 7 is absent in the coral, this work describes a coral LMP X gene. Phylogenetic analyses, along with hydropathy profiling of a critical portion of the invertebrate and vertebrate paralogous genes, suggests that some invertebrates have two diverging LMP X genes. In some cases, one LMP X protein shares characteristics with vertebrate LMP 7. This work presents new evidence for how the LMP X and 7 genes evolved. ^

Quantitative and Qualitative Aspects of Dissolved Organic Carbon Leached from Senescent Plants in an Oligotrophic Wetland

We conducted a series of experiments whereby dissolved organic matter (DOM) was leached from various wetland and estuarine plants, namely sawgrass (Cladium jamaicense), spikerush (Eleocharis cellulosa), red mangrove (Rhizophora mangle), cattail (Typha domingensis), periphyton (dry and wet mat), and a seagrass (turtle grass; Thalassia testudinum). All are abundant in the Florida Coastal Everglades (FCE) except for cattail, but this species has a potential to proliferate in this environment. Senescent plant samples were immersed into ultrapure water with and without addition of 0.1% NaN3 (w/ and w/o NaN3, respectively) for 36 days. We replaced the water every 3 days. The amount of dissolved organic carbon (DOC), sugars, and phenols in the leachates were analyzed. The contribution of plant leachates to the ultrafiltered high molecular weight fraction of DOM (>1 kDa; UDOM) in natural waters in the FCE was also investigated. UDOM in plant leachates was obtained by tangential flow ultrafiltration and its carbon and phenolic compound compositions were analyzed using solid state 13C cross-polarization magic angle spinning nuclear magnetic resonance (13C CPMAS NMR) spectroscopy and thermochemolysis in the presence of tetramethylammonium hydroxide (TMAH thermochemolysis), respectively. The maximum yield of DOC leached from plants over the 36-day incubations ranged from 13.0 to 55.2 g C kg−1 dry weight. This amount was lower in w/o NaN3 treatments (more DOC was consumed by microbes than produced) except for periphyton. During the first 2 weeks of the 5 week incubation period, 60–85% of the total amount of DOC was leached, and exponential decay models fit the leaching rates except for periphyton w/o NaN3. Leached DOC (w/ NaN3) contained different concentrations of sugars and phenols depending on the plant types (1.09–7.22 and 0.38–12.4 g C kg−1 dry weight, respectively), and those biomolecules comprised 8–34% and 4–28% of the total DOC, respectively. This result shows that polyphenols that readily leach from senescent plants can be an important source of chromophoric DOM (CDOM) in wetland environments. The O-alkyl C was found to be the major C form (55±9%) of UDOM in plant leachates as determined by 13C CPMAS NMR. The relative abundance of alkyl C and carbonyl C was consistently lower in plant-leached UDOM than that in natural water UDOM in the FCE, which suggests that these constituents increase in relative abundance during diagenetic processing. TMAH thermochemolysis analysis revealed that the phenolic composition was different among the UDOM leached from different plants, and was expected to serve as a source indicator of UDOM in natural water. Polyphenols are, however, very reactive and photosensitive in aquatic environments, and thus may loose their plant-specific molecular characteristics shortly. Our study suggests that variations in vegetative cover across a wetland landscape will affect the quantity and quality of DOM leached into the water, and such differences in DOM characteristics may affect other biogeochemical processes.

A novel signal processing method for intraoperative neurophysiological monitoring in spinal surgeries

Intraoperative neurophysiologic monitoring is an integral part of spinal surgeries and involves the recording of somatosensory evoked potentials (SSEP). However, clinical application of IONM still requires anywhere between 200 to 2000 trials to obtain an SSEP signal, which is excessive and introduces a significant delay during surgery to detect a possible neurological damage. The aim of this study is to develop a means to obtain the SSEP using a much less, twelve number of recordings. The preliminary step involved was to distinguish the SSEP with the ongoing brain activity. We first establish that the brain activity is indeed quasi-stationary whereas an SSEP is expected to be identical every time a trial is recorded. An algorithm was developed using Chebychev time windowing for preconditioning of SSEP trials to retain the morphological characteristics of somatosensory evoked potentials (SSEP). This preconditioning was followed by the application of a principal component analysis (PCA)-based algorithm utilizing quasi-stationarity of EEG on 12 preconditioned trials. A unique Walsh transform operation was then used to identify the position of the SSEP event. An alarm is raised when there is a 10% time in latency deviation and/or 50% peak-to-peak amplitude deviation, as per the clinical requirements. The algorithm shows consistency in the results in monitoring SSEP in up to 6-hour surgical procedures even under this significantly reduced number of trials. In this study, the analysis was performed on the data recorded in 29 patients undergoing surgery during which the posterior tibial nerve was stimulated and SSEP response was recorded from scalp. This method is shown empirically to be more clinically viable than present day approaches. In all 29 cases, the algorithm takes 4sec to extract an SSEP signal, as compared to conventional methods, which take several minutes. The monitoring process using the algorithm was successful and proved conclusive under the clinical constraints throughout the different surgical procedures with an accuracy of 91.5%. Higher accuracy and faster execution time, observed in the present study, in determining the SSEP signals provide a much improved and effective neurophysiological monitoring process.

Digital signal processing implementation for near real-time wavelet transformation system for binary images

Communication has become an essential function in our civilization. With the increasing demand for communication channels, it is now necessary to find ways to optimize the use of their bandwidth. One way to achieve this is by transforming the information before it is transmitted. This transformation can be performed by several techniques. One of the newest of these techniques is the use of wavelets. Wavelet transformation refers to the act of breaking down a signal into components called details and trends by using small waveforms that have a zero average in the time domain. After this transformation the data can be compressed by discarding the details, transmitting the trends. In the receiving end, the trends are used to reconstruct the image. In this work, the wavelet used for the transformation of an image will be selected from a library of available bases. The accuracy of the reconstruction, after the details are discarded, is dependent on the wavelets chosen from the wavelet basis library. The system developed in this thesis takes a 2-D image and decomposes it using a wavelet bank. A digital signal processor is used to achieve near real-time performance in this transformation task. A contribution of this thesis project is the development of DSP-based test bed for the future development of new real-time wavelet transformation algorithms.