Signal processing of physiological signals for automated assessment of stress in computer users

The need to provide computers with the ability to distinguish the affective state of their users is a major requirement for the practical implementation of affective computing concepts. This dissertation proposes the application of signal processing methods on physiological signals to extract from them features that can be processed by learning pattern recognition systems to provide cues about a person's affective state. In particular, combining physiological information sensed from a user's left hand in a non-invasive way with the pupil diameter information from an eye-tracking system may provide a computer with an awareness of its user's affective responses in the course of human-computer interactions. In this study an integrated hardware-software setup was developed to achieve automatic assessment of the affective status of a computer user. A computer-based "Paced Stroop Test" was designed as a stimulus to elicit emotional stress in the subject during the experiment. Four signals: the Galvanic Skin Response (GSR), the Blood Volume Pulse (BVP), the Skin Temperature (ST) and the Pupil Diameter (PD), were monitored and analyzed to differentiate affective states in the user. Several signal processing techniques were applied on the collected signals to extract their most relevant features. These features were analyzed with learning classification systems, to accomplish the affective state identification. Three learning algorithms: Naïve Bayes, Decision Tree and Support Vector Machine were applied to this identification process and their levels of classification accuracy were compared. The results achieved indicate that the physiological signals monitored do, in fact, have a strong correlation with the changes in the emotional states of the experimental subjects. These results also revealed that the inclusion of pupil diameter information significantly improved the performance of the emotion recognition system. ^

A digital signal processing approach for affective sensing of a computer user through pupil diameter monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

Off-line and on-line affective recognition of a computer user through a biosignal processing approach

Physiological signals, which are controlled by the autonomic nervous system (ANS), could be used to detect the affective state of computer users and therefore find applications in medicine and engineering. The Pupil Diameter (PD) seems to provide a strong indication of the affective state, as found by previous research, but it has not been investigated fully yet. ^ In this study, new approaches based on monitoring and processing the PD signal for off-line and on-line affective assessment ("relaxation" vs. "stress") are proposed. Wavelet denoising and Kalman filtering methods are first used to remove abrupt changes in the raw Pupil Diameter (PD) signal. Then three features (PDmean, PDmax and PDWalsh) are extracted from the preprocessed PD signal for the affective state classification. In order to select more relevant and reliable physiological data for further analysis, two types of data selection methods are applied, which are based on the paired t-test and subject self-evaluation, respectively. In addition, five different kinds of the classifiers are implemented on the selected data, which achieve average accuracies up to 86.43% and 87.20%, respectively. Finally, the receiver operating characteristic (ROC) curve is utilized to investigate the discriminating potential of each individual feature by evaluation of the area under the ROC curve, which reaches values above 0.90. ^ For the on-line affective assessment, a hard threshold is implemented first in order to remove the eye blinks from the PD signal and then a moving average window is utilized to obtain the representative value PDr for every one-second time interval of PD. There are three main steps for the on-line affective assessment algorithm, which are preparation, feature-based decision voting and affective determination. The final results show that the accuracies are 72.30% and 73.55% for the data subsets, which were respectively chosen using two types of data selection methods (paired t-test and subject self-evaluation). ^ In order to further analyze the efficiency of affective recognition through the PD signal, the Galvanic Skin Response (GSR) was also monitored and processed. The highest affective assessment classification rate obtained from GSR processing is only 63.57% (based on the off-line processing algorithm). The overall results confirm that the PD signal should be considered as one of the most powerful physiological signals to involve in future automated real-time affective recognition systems, especially for detecting the "relaxation" vs. "stress" states.^

A Digital Signal Processing Approach for Affective Sensing of a Computer User through Pupil Diameter Monitoring

Recent research has indicated that the pupil diameter (PD) in humans varies with their affective states. However, this signal has not been fully investigated for affective sensing purposes in human-computer interaction systems. This may be due to the dominant separate effect of the pupillary light reflex (PLR), which shrinks the pupil when light intensity increases. In this dissertation, an adaptive interference canceller (AIC) system using the H∞ time-varying (HITV) adaptive algorithm was developed to minimize the impact of the PLR on the measured pupil diameter signal. The modified pupil diameter (MPD) signal, obtained from the AIC was expected to reflect primarily the pupillary affective responses (PAR) of the subject. Additional manipulations of the AIC output resulted in a processed MPD (PMPD) signal, from which a classification feature, PMPDmean, was extracted. This feature was used to train and test a support vector machine (SVM), for the identification of stress states in the subject from whom the pupil diameter signal was recorded, achieving an accuracy rate of 77.78%. The advantages of affective recognition through the PD signal were verified by comparatively investigating the classification of stress and relaxation states through features derived from the simultaneously recorded galvanic skin response (GSR) and blood volume pulse (BVP) signals, with and without the PD feature. The discriminating potential of each individual feature extracted from GSR, BVP and PD was studied by analysis of its receiver operating characteristic (ROC) curve. The ROC curve found for the PMPDmean feature encompassed the largest area (0.8546) of all the single-feature ROCs investigated. The encouraging results seen in affective sensing based on pupil diameter monitoring were obtained in spite of intermittent illumination increases purposely introduced during the experiments. Therefore, these results confirmed the benefits of using the AIC implementation with the HITV adaptive algorithm to isolate the PAR and the potential of using PD monitoring to sense the evolving affective states of a computer user.

Off-line and On-line Affective Recognition of a Computer User through A Biosignal Processing Approach

Physiological signals, which are controlled by the autonomic nervous system (ANS), could be used to detect the affective state of computer users and therefore find applications in medicine and engineering. The Pupil Diameter (PD) seems to provide a strong indication of the affective state, as found by previous research, but it has not been investigated fully yet. In this study, new approaches based on monitoring and processing the PD signal for off-line and on-line affective assessment (“relaxation” vs. “stress”) are proposed. Wavelet denoising and Kalman filtering methods are first used to remove abrupt changes in the raw Pupil Diameter (PD) signal. Then three features (PDmean, PDmax and PDWalsh) are extracted from the preprocessed PD signal for the affective state classification. In order to select more relevant and reliable physiological data for further analysis, two types of data selection methods are applied, which are based on the paired t-test and subject self-evaluation, respectively. In addition, five different kinds of the classifiers are implemented on the selected data, which achieve average accuracies up to 86.43% and 87.20%, respectively. Finally, the receiver operating characteristic (ROC) curve is utilized to investigate the discriminating potential of each individual feature by evaluation of the area under the ROC curve, which reaches values above 0.90. For the on-line affective assessment, a hard threshold is implemented first in order to remove the eye blinks from the PD signal and then a moving average window is utilized to obtain the representative value PDr for every one-second time interval of PD. There are three main steps for the on-line affective assessment algorithm, which are preparation, feature-based decision voting and affective determination. The final results show that the accuracies are 72.30% and 73.55% for the data subsets, which were respectively chosen using two types of data selection methods (paired t-test and subject self-evaluation). In order to further analyze the efficiency of affective recognition through the PD signal, the Galvanic Skin Response (GSR) was also monitored and processed. The highest affective assessment classification rate obtained from GSR processing is only 63.57% (based on the off-line processing algorithm). The overall results confirm that the PD signal should be considered as one of the most powerful physiological signals to involve in future automated real-time affective recognition systems, especially for detecting the “relaxation” vs. “stress” states.

Correlates of Vitamin D Status in Healthy Older Adults Living in Miami-Dade and its Association with Non-Skeletal Outcomes: A Cross-Sectional Study

Examining factors that affect vitamin D status in the fast-growing elderly population of Miami-Dade, Florida, is needed. Vitamin D deficiency in older adults has been linked to correlates of disability, including falls and fractures, and cardiovascular disease. The purpose of this study was to determine the proportion of vitamin D insufficient individuals and their relationship with vitamin D insufficiency in older adults (n=97) living in Miami-Dade. We evaluated the association between vitamin D status and 1) dual task physical performance to understand the link between vitamin D and cognition in the context of mobility; and 2) cardiometabolic risk, measured by galvanic skin response, pulse oximetry, and blood pressure to create a composite score based on autonomic nervous system and endothelial function. Participants completed baseline assessments that included serum levels of vitamin D, anthropometrics, body composition, dual task physical performance and cardiometabolic risk. Surveys to evaluate vitamin D intake, sun exposure, physical activity, and depressive symptoms were completed. Spearman’s correlations, independent t-tests, paired t-tests, repeated measures ANOVAs, and multiple logistic and linear regressions were used to examine the relationship of vitamin D insufficiency (25(OH)D /ml) and sufficiency (25(OH)D ≥30 ng/ml) with determinants of vitamin D status, dual task physical performance variables and cardiometabolic risk scores. Although the proportion of vitamin D insufficient individuals was lower when compared to the prevalance of the general United States elderly population, it was still common in healthy community-dwelling older adults living in Miami-Dade County, especially among Hispanics. Factors that affected skin synthesis (ethnicity, and sun exposure), and bioavailability/metabolism (obesity) were significant predictors of vitamin D status. Vitamin D insufficiency was not significantly correlated with worse dual task physical performance; however, cognitive performance was worse in the vitamin D insufficient group. Our results suggest a relationship of vitamin D insufficiency with executive dysfunction, and support an association with cardiometabolic risk using an innovative electro-sensor complex, possibly by modulating autonomic nervous system activity and vascular function, thus affecting cardiac performance.

An ethnopharmacological approach to multidrug -resistant Staphylococcus aureus: Evaluation of Italian plants used in the traditional healing of skin disease

One-third of botanical remedies from southern Italy are used to treat skin and soft tissue infections (SST's). Methicillin-resistant Staphylococcus aureus (MRSA), a common cause of SSTIs, is responsible for increased morbidity and mortality from infections. Therapeutic options are limited by antibiotic resistance. Many plants possess potent antimicrobial compounds for these disorders. Validation of traditional medical practices is important for the people who rely on medicinal plants. Moreover, identification of novel antibiotics and anti-pathogenic agents for MRSA is important to global healthcare.^ I took an ethnopharmacological approach to understand how Italian medicinal plants used for the treatment of SSTIs affect MRSA growth and virulence. My hypothesis was that plants used in folk remedies for SSTI would exhibit lower cytotoxicity and greater inhibition of bacterial growth, biofilm formation and toxin production in MRSA than plants used for remedies unrelated to the skin or for plants with no ethnomedical application. The field portion of my research was conducted in the Vulture-Alto Bradano area of southern Italy. I collected 104 plant species and created 168 crude extracts. In the lab, I screened samples for activity against MRSA in a battery of bioassays. Growth inhibition was analyzed using broth microtiter assays for determination of the minimum inhibitory concentration. Interference with quorum-sensing (QS) processes, which mediate pathogenicity, was quantified through RP-HPLC of δ-toxin production. Interference with biofilm formation and adherence was assessed using staining methods. The mammalian cytotoxicity of natural products was analyzed using MTT cell proliferation assay techniques.^ Although bacteriostatic activity was limited, extracts from six plants used in Italian folk medicine (Arundo donax, Ballota nigra, Juglans regia, Leopoldia comosa, Marrubium vulgare, and Rubus ulmifolius ) significantly inhibited biofilm formation and adherence. Moreover, plants used to treat SSTI demonstrated significantly greater anti-biofilm activity when compared to plants with no ethnomedical application. QSI activity was evident in 90% of the extracts tested and extracts from four plants ( Ballota nigra, Castanea saliva, Rosmarinus officinalis, and Sambucus ebulus) exhibited a significant dose-dependent response. Some of the plant remedies for SSTI identified in this study can be validated due to anti-MRSA activity.^

Effects of Endothelin 3 on the Tumor-Angiogenic Response in a Novel Melanoma Mouse Model

Skin cancer is the most common form of cancer in the United States. Melanoma is a particular type of skin cancer, which arises from the malignant transformation of melanocytes and generally exhibits a high propensity to metastasize. Melanoma progression is dependent on angiogenesis to deliver the oxygen and nutrients required to maintain the altered metabolism of rapidly proliferating tumorigenic cells. Recent studies have implicated the growth factor Endothelin 3 (Edn3) in melanoma progression and metastasis. The aim of this study was to examine the role that Edn3 plays in the angiogenesis of melanocytic lesions. For this purpose, Dct-Grm1 transgenic mice, which spontaneously acquire melanocytic lesions through the aberrant expression of the metabotropic glutamate receptor 1 (mGluR1), were crossed with K5-Edn3 transgenic mice that overexpress Edn3. Tumors in the Dct-Grm1/K5-Edn3 experimental population were examined and compared to the control Dct-Grm1 population using immuno-fluorescent staining targeted against the vascular endothelial cell marker CD31. Proteomic arrays were also used and identified changes in the expression of specific angiogenic factors. CD31 antibody staining results revealed an increased vascular density in Dct-Grm1/K5-Edn3 tumors compared with tumors from the Dct-Grm1 controls. Analysis of the relative expression of angiogenic proteins showed an upregulation of various vascular factors in tumors from the Dct-Grm1/K5-Edn3 population, including VEGF-B, MMP-8, MMP-9, and Angiogenin. These results suggest that endothelin signaling promotes angiogenesis in melanocytic lesions. Targeting the factors upregulated by Edn3 signaling may prove effective in hindering melanoma progression.