Constructing and Using Multimodal Narratives to Research in Science Education: Contributions Based on Practical Classroom

This study deals with the problem of how to collect genuine and useful data about science classroom practices, and preserving the complex and holistic nature of teaching and learning. Additionally, we were looking for an instrument that would allow comparability and verifiability for teaching and research purposes. Given the multimodality of teaching and learning processes, we developed the multimodal narrative (MN), which describes what happens during a task and incorporates data such as examples of students’ work.

Check your biosignals here: a new dataset for off-person ECG biometrics

The Check Your Biosignals Here initiative (CYBHi) was developed as a way of creating a dataset and consistently repeatable acquisition framework, to further extend research in electrocardiographic (ECG) biometrics. In particular, our work targets the novel trend towards off-the-person data acquisition, which opens a broad new set of challenges and opportunities both for research and industry. While datasets with ECG signals collected using medical grade equipment at the chest can be easily found, for off-the-person ECG data the solution is generally for each team to collect their own corpus at considerable expense of resources. In this paper we describe the context, experimental considerations, methods, and preliminary findings of two public datasets created by our team, one for short-term and another for long-term assessment, with ECG data collected at the hand palms and fingers. (C) 2013 Elsevier Ireland Ltd. All rights reserved.

ECG Biometrics: A Template Selection Approach

Electrocardiography (ECG) biometrics is emerging as a viable biometric trait. Recent developments at the sensor level have shown the feasibility of performing signal acquisition at the fingers and hand palms, using one-lead sensor technology and dry electrodes. These new locations lead to ECG signals with lower signal to noise ratio and more prone to noise artifacts; the heart rate variability is another of the major challenges of this biometric trait. In this paper we propose a novel approach to ECG biometrics, with the purpose of reducing the computational complexity and increasing the robustness of the recognition process enabling the fusion of information across sessions. Our approach is based on clustering, grouping individual heartbeats based on their morphology. We study several methods to perform automatic template selection and account for variations observed in a person's biometric data. This approach allows the identification of different template groupings, taking into account the heart rate variability, and the removal of outliers due to noise artifacts. Experimental evaluation on real world data demonstrates the advantages of our approach.

Enhancing Multimodal Silent Speech Interfaces With Feature Selection

In research on Silent Speech Interfaces (SSI), different sources of information (modalities) have been combined, aiming at obtaining better performance than the individual modalities. However, when combining these modalities, the dimensionality of the feature space rapidly increases, yielding the well-known "curse of dimensionality". As a consequence, in order to extract useful information from this data, one has to resort to feature selection (FS) techniques to lower the dimensionality of the learning space. In this paper, we assess the impact of FS techniques for silent speech data, in a dataset with 4 non-invasive and promising modalities, namely: video, depth, ultrasonic Doppler sensing, and surface electromyography. We consider two supervised (mutual information and Fisher's ratio) and two unsupervised (meanmedian and arithmetic mean geometric mean) FS filters. The evaluation was made by assessing the classification accuracy (word recognition error) of three well-known classifiers (knearest neighbors, support vector machines, and dynamic time warping). The key results of this study show that both unsupervised and supervised FS techniques improve on the classification accuracy on both individual and combined modalities. For instance, on the video component, we attain relative performance gains of 36.2% in error rates. FS is also useful as pre-processing for feature fusion. Copyright © 2014 ISCA.

ECG-based biometrics: A real time classification approach

Behavioral biometrics is one of the areas with growing interest within the biosignal research community. A recent trend in the field is ECG-based biometrics, where electrocardiographic (ECG) signals are used as input to the biometric system. Previous work has shown this to be a promising trait, with the potential to serve as a good complement to other existing, and already more established modalities, due to its intrinsic characteristics. In this paper, we propose a system for ECG biometrics centered on signals acquired at the subject's hand. Our work is based on a previously developed custom, non-intrusive sensing apparatus for data acquisition at the hands, and involved the pre-processing of the ECG signals, and evaluation of two classification approaches targeted at real-time or near real-time applications. Preliminary results show that this system leads to competitive results both for authentication and identification, and further validate the potential of ECG signals as a complementary modality in the toolbox of the biometric system designer.

Experimental apparatus for finger ECG biometrics

Current Electrocardiographic (ECG) signal acquisition methods are generally highly intrusive, as they involve the use of pre-gelled electrodes and cabled sensors placed directly on the person, at the chest or limbs level. Moreover, systems that make use of alternative conductive materials to overcome this issue, only provide heart rate information and not the detailed signal itself. We present a comparison and evaluation of two types of dry electrodes as interface with the skin, targeting wearable and low intrusiveness applications, which enable ECG measurement without the need for any apparatus permanently fitted to the individual. In particular, our approach is targeted at ECG biometrics using signals collected at the hand or finger level. A custom differential circuit with virtual ground was also developed for enhanced usability. Our work builds upon the current stateof-the-art in sensoring devices and processing tools, and enables novel data acquisition settings through the use of dry electrodes. Experimental evaluation was performed for Ag/AgCl and Electrolycra materials, and results show that both materials exhibit adequate performance for the intended application.

ECG biometrics using a dissimilarity space representation

Electrocardiogram (ECG) biometrics are a relatively recent trend in biometric recognition, with at least 13 years of development in peer-reviewed literature. Most of the proposed biometric techniques perform classifi-cation on features extracted from either heartbeats or from ECG based transformed signals. The best representation is yet to be decided. This paper studies an alternative representation, a dissimilarity space, based on the pairwise dissimilarity between templates and subjects' signals. Additionally, this representation can make use of ECG signals sourced from multiple leads. Configurations of three leads will be tested and contrasted with single-lead experiments. Using the same k-NN classifier the results proved superior to those obtained through a similar algorithm which does not employ a dissimilarity representation. The best Authentication EER went as low as 1:53% for a database employing 503 subjects. However, the employment of extra leads did not prove itself advantageous.

Embedded system for individual recognition based on ECG biometrics

Biometric recognition is emerging has an alternative solution for applications where the privacy of the information is crucial. This paper presents an embedded biometric recognition system based on the Electrocardiographic signals (ECG) for individual identification and authentication. The proposed system implements a real-time state-of-the-art recognition algorithm, which extracts information from the frequency domain. The system is based on a ARM Cortex 4. Preliminary results show that embedded platforms are a promising path for the implementation of ECG-based applications in real-world scenario.

Sistema multimodal para captura e anotação de vídeo

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Personalization platform for multimodal ubiquitous computing applications

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Fatores de prognóstico para os resultados de sucesso da fisioterapia multimodal em utentes com dor cervical crónica

RESUMO: Introdução: O conhecimento acerca da influência das características dos indivíduos com dor cervical crónica (DCC) no prognóstico dos resultados alcançados com a Fisioterapia é ainda inconsistente, sendo escassos os estudos desenvolvidos neste âmbito. Objetivo: Este relatório pretende determinar se um modelo baseado em fatores de prognóstico é capaz de prever os resultados de sucesso da Fisioterapia, a curto prazo, em utentes com DCC, ao nível da incapacidade funcional, intensidade da dor e perceção global de melhoria. Metodologia: Trata-se de estudo de coorte prospetivo com 112 participantes. Os utentes foram avaliados na primeira semana de tratamento e sete semanas após o início da intervenção. Os instrumentos utilizados foram o Neck Disability Index–Versão Portuguesa (NDI-PT) e a Escala Numérica da Dor (END) nos dois momentos de avaliação, um Questionário de Caracterização Sociodemográfica e Clínica da Amostra na baseline e a Patient Global Impression Change Scale–Versão Portuguesa (PGIC-PT) no follow-up. As características sociodemográficas e clínicas foram incluídas como potenciais fatores de prognóstico e estes foram definidos com base nas diferenças mínimas clinicamente importantes (DMCI) dos instrumentos NDIPT (DMCI≥6) e END (DMCI≥2) e no critério de pontuação ≥5 na PGIC-PT. A análise dos dados foi realizada através do método de regressão logística (backward conditional procedure) para identificar as associações entre os indicadores e as variáveis de resultado (p<0.05). Resultados: Dos 112 participantes incluídos no estudo, 108 completaram o follow-up (média de idade: 51.76±10.19). No modelo multivariado relativo à incapacidade funcional, os resultados de sucesso encontram-se associados a elevados níveis de incapacidade na baseline (OR=1.123; 95% IC 1.056–1.194) e a duração da dor inferior a 12 meses (OR=2.704; 95% IC 1.138–6.424). Este modelo explica 30.0% da variância da melhoria da funcionalidade e classifica corretamente 74.1% dos utentes (sensibilidade: 75.9%; especificidade: 72.0%). O modelo relativo à intensidade da dor identificou apenas a associação do outcome com níveis elevados de intensidade da dor na baseline (OR=1.321; 95% IC 1.047–1.668), explicando 7.5% da variância da redução da mesma e classificando corretamente 68.2% dos utentes (sensibilidade: 94.4%; especificidade: 16.7%). O modelo final referente à perceção global de melhoria apresentou uma associação com a intensidade da dor na baseline (OR=0.621; 95% IC 0.465–0.829), com a presença de cefaleias e/ou tonturas (OR=2.538; 95% IC 0.987–6.526) e com a duração da dor superior a 12 meses (OR=0.279; 95% IC 0.109–0.719). Este modelo explica 27.5% da variância dos resultados de sucesso para este outcome e classifica corretamente 73.1% dos utentes (sensibilidade: 81.8%; especificidade: 59.5%). Conclusões: Utentes com DCC com elevada incapacidade na baseline e queixas de dor há menos de 12 meses apresentam maior probabilidade de obter melhorias ao nível da incapacidade funcional. Elevados níveis de intensidade da dor na baseline predizem resultados de sucesso na redução da dor após sete semanas de tratamento. Utentes com DCC com baixos níveis de dor na baseline, com cefaleias e/ou tonturas e com queixas de dor há mais de 12 meses apresentam maior probabilidade de obter uma melhor perceção de melhoria.--------------- ABSTRACT:Introduction: The influence of the characteristics of individuals with chronic neck pain (CNP) on the prognosis of physiotherapy outcomes is still inconsistent, there being few studies developed in this context. Aim: This study seeks to determine whether a model based on prognostic factors can predict the short-term physiotherapy successful outcomes in CNP patients, regarding functional disability, pain intensity and perceived recovery. Methodology: This is a prospective cohort study with 112 participants. Patients were assessed during the first week of treatment and seven weeks after the start of the intervention. The instruments used were the Neck Disability Index–Portuguese Version (NDI-PT) and the Numerical Rating Scale (NRS) at both moments of assessment, a Sample Sociodemographic and Clinical Characterization Questionnaire at baseline and Patient Global Impression Change Scale–Portuguese Version (PGIC-PT) at the follow-up. The sociodemographic and clinical characteristics were included as potential predictors of successful outcomes, and these were defined on the basis of minimal clinically important differences (MCID) of NDI-PT (MCID≥6) and END (MCID≥2) and the criteria score ≥5 on the PGIC-PT. Data analysis was performed using logistic regression (backward conditional procedure) to identify associations between predictors and outcomes (p<0.05). Results: Of the 112 participants included in the study, 108 completed the follow-up (mean age: 51.76±10.19). In the multivariate model of functional disability, the successful outcomes are associated with high levels of disability at baseline (OR = 1.123; 95% CI 1.056-1.194), and pain duration shorter than 12 months (OR=2.704; 95% CI 1.138–6.424). This model explains 30.0% of the variance of improved functional capacity and correctly classifies 74.1% of the patients (sensitivity: 75.9%, specificity: 72.0%). The model for pain intensity solely identified an outcome association with high pain intensity at baseline (OR=1.321; 95% CI 1.047-1.668), explaining 7.5% of the variance of pain reduction and correctly classifying 68.2% of the patients (sensitivity: 94.4%, specificity: 16.7%). The final model of perceived recovery showed an association with pain intensity at baseline (OR=0.621; 95% CI 0465-0829), with the presence of headache and/or dizziness (OR=2.538; 95% CI 0.987-6.526) and the duration of pain over 12 months (OR=0.279; 95% CI 0.109-0.719). This model explains 27.5% of the variance of successful outcomes and correctly classifies 73.1% of the patients (sensitivity: 81.8%, specificity: 59.5%). Conclusions: Patients with CNP with high disability at baseline and complaints of pain for less than 12 months are more likely to obtain improvements in functional disability. High levels of pain intensity at baseline predict successful outcomes in pain reduction after seven weeks of treatment. Patients with CNP with low levels of pain at baseline, with headache and/or dizziness and with pain complaints for more than 12 months are more likely to get a better perceived recovery.

Interação multimodal de dispositivos robóticos em ambiente simulado

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação

A network approach to brain aging through multimodal neuroimaging

Tese de Doutoramento em Ciências da Saúde.

Multimodal automatic user disposition recognition in human-machine interaction

Magdeburg, Univ., Fak. für Elektrotechnik und Informationstechnik, Diss., 2013

Primary localized rectal/pararectal gastrointestinal stromal tumors: results of surgical and multimodal therapy from the French Sarcoma group.

BACKGROUND: Rectal and pararectal gastrointestinal stromal tumors (GISTs) are rare. The optimal management strategy for primary localized GISTs remains poorly defined. METHODS: We conducted a retrospective analysis of 41 patients with localized rectal or pararectal GISTs treated between 1991 and 2011 in 13 French Sarcoma Group centers. RESULTS: Of 12 patients who received preoperative imatinib therapy for a median duration of 7 (2-12) months, 8 experienced a partial response, 3 had stable disease, and 1 had a complete response. Thirty and 11 patients underwent function-sparing conservative surgery and abdominoperineal resection, respectively. Tumor resections were mostly R0 and R1 in 35 patients. Tumor rupture occurred in 12 patients. Eleven patients received postoperative imatinib with a median follow-up of 59 (2.4-186) months. The median time to disease relapse was 36 (9.8-62) months. The 5-year overall survival rate was 86.5%. Twenty patients developed local recurrence after surgery alone, two developed recurrence after resection combined with preoperative and/or postoperative imatinib, and eight developed metastases. In univariate analysis, the mitotic index (≤5) and tumor size (≤5 cm) were associated with a significantly decreased risk of local relapse. Perioperative imatinib was associated with a significantly reduced risk of overall relapse and local relapse. CONCLUSIONS: Perioperative imatinib therapy was associated with improved disease-free survival. Preoperative imatinib was effective. Tumor shrinkage has a clear benefit for local excision in terms of feasibility and function preservation. Given the complexity of rectal GISTs, referral of patients with this rare disease to expert centers to undergo a multidisciplinary approach is recommended.