Interface Multimodal com Predição de Movimentos para Uso em Reabilitação de Membros Inferiores

Esta dissertação apresenta o desenvolvimento de uma plataforma multimodal de aquisição e processamento de sinais. O projeto proposto insere-se no contexto do desenvolvimento de interfaces multimodais para aplicação em dispositivos robóticos cujo propósito é a reabilitação motora adaptando o controle destes dispositivos de acordo com a intenção do usuário. A interface desenvolvida adquire, sincroniza e processa sinais eletroencefalográficos (EEG), eletromiográficos (EMG) e sinais provenientes de sensores inerciais (IMUs). A aquisição dos dados é feita em experimentos realizados com sujeitos saudáveis que executam tarefas motoras de membros inferiores. O objetivo é analisar a intenção de movimento, a ativação muscular e o início efetivo dos movimentos realizados, respectivamente, através dos sinais de EEG, EMG e IMUs. Para este fim, uma análise offline foi realizada. Nessa análise, são utilizadas técnicas de processamento dos sinais biológicos e técnicas para processar sinais provenientes de sensores inerciais. A partir destes, os ângulos da articulação do joelho também são aferidos ao longo dos movimentos. Um protocolo experimental de testes foi proposto para as tarefas realizadas. Os resultados demonstraram que o sistema proposto foi capaz de adquirir, sincronizar, processar e classificar os sinais combinadamente. Análises acerca da acurácia dos classificadores utilizados mostraram que a interface foi capaz de identificar intenção de movimento em 76, 0 ± 18, 2% dos movimentos. A maior média de tempo de antecipação ao movimento foi obtida através da análise do sinal de EEG e foi de 716, 0±546, 1 milisegundos. A partir da análise apenas do sinal de EMG, este valor foi de 88, 34 ± 67, 28 milisegundos. Os resultados das etapas de processamento dos sinais biológicos, a medição dos ângulos da articulação, bem como os valores de acurácia e tempo de antecipação ao movimento se mostraram em conformidade com a literatura atual relacionada.

Robust temporal alignment of multimodal cardiac sequences

Given the dynamic nature of cardiac function, correct temporal alignment of pre-operative models and intraoperative images is crucial for augmented reality in cardiac image-guided interventions. As such, the current study focuses on the development of an image-based strategy for temporal alignment of multimodal cardiac imaging sequences, such as cine Magnetic Resonance Imaging (MRI) or 3D Ultrasound (US). First, we derive a robust, modality-independent signal from the image sequences, estimated by computing the normalized crosscorrelation between each frame in the temporal sequence and the end-diastolic frame. This signal is a resembler for the left-ventricle (LV) volume curve over time, whose variation indicates di erent temporal landmarks of the cardiac cycle. We then perform the temporal alignment of these surrogate signals derived from MRI and US sequences of the same patient through Dynamic Time Warping (DTW), allowing to synchronize both sequences. The proposed framework was evaluated in 98 patients, which have undergone both 3D+t MRI and US scans. The end-systolic frame could be accurately estimated as the minimum of the image-derived surrogate signal, presenting a relative error of 1:6 1:9% and 4:0 4:2% for the MRI and US sequences, respectively, thus supporting its association with key temporal instants of the cardiac cycle. The use of DTW reduces the desynchronization of the cardiac events in MRI and US sequences, allowing to temporally align multimodal cardiac imaging sequences. Overall, a generic, fast and accurate method for temporal synchronization of MRI and US sequences of the same patient was introduced. This approach could be straightforwardly used for the correct temporal alignment of pre-operative MRI information and intra-operative US images.

Theoretical Analysis of Multimodal Four-Wave Mixing in Optical Microwires

Optical fiber microwires (OFMs) are nonlinear optical waveguides that support several spatial modes. The multimodal generalized nonlinear Schrodinger equation (MM-GNLSE) is deduced taking into account the linear and nonlinear modal coupling. A detailed theoretical description of four-wave mixing (FWM) considering the modal coupling is developed. Both, the intramode and the intermode phase-matching conditions is calculated for an optical microwire in a strong guiding regime. Finally, the FWM dynamics is studied and the amplitude evolution of the pump beams, the signal and the idler are analyzed.

Multimodal species abundance distributions: a deconstruction approach reveals the processes behind the pattern

Copyright © 2014 The Authors. Oikos © 2014 Nordic Society Oikos.

Classification and staging of chronic liver disease from multimodal data

Chronic liver disease (CLD) is most of the time an asymptomatic, progressive, and ultimately potentially fatal disease. In this study, an automatic hierarchical procedure to stage CLD using ultrasound images, laboratory tests, and clinical records are described. The first stage of the proposed method, called clinical based classifier (CBC), discriminates healthy from pathologic conditions. When nonhealthy conditions are detected, the method refines the results in three exclusive pathologies in a hierarchical basis: 1) chronic hepatitis; 2) compensated cirrhosis; and 3) decompensated cirrhosis. The features used as well as the classifiers (Bayes, Parzen, support vector machine, and k-nearest neighbor) are optimally selected for each stage. A large multimodal feature database was specifically built for this study containing 30 chronic hepatitis cases, 34 compensated cirrhosis cases, and 36 decompensated cirrhosis cases, all validated after histopathologic analysis by liver biopsy. The CBC classification scheme outperformed the nonhierachical one against all scheme, achieving an overall accuracy of 98.67% for the normal detector, 87.45% for the chronic hepatitis detector, and 95.71% for the cirrhosis detector.

How new online communication tools and the availability of content influence the audiences and the creative process

We live in a changing world. At an impressive speed, every day new technological resources appear. We increasingly use the Internet to obtain and share information, and new online communication tools are emerging. Each of them encompasses new potential and creates new audiences. In recent years, we witnessed the emergence of Facebook, Twitter, YouTube and other media platforms. They have provided us with an even greater interactivity between sender and receiver, as well as generated a new sense of community. At the same time we also see the availability of content like it never happened before. We are increasingly sharing texts, videos, photos, etc. This poster intends to explore the potential of using these new online communication tools in the cultural sphere to create new audiences, to develop of a new kind of community, to provide information as well as different ways of building organizations’ memory. The transience of performing arts is accompanied by the need to counter that transience by means of documentation. This desire to ‘save’ events reaches its expression with the information archive of the different production moments as well as the opportunity to record the event and present it through, for instance, digital platforms. In this poster we intend to answer the following questions: which online communication tools are being used to engage audiences in the cultural sphere (specifically between theater companies in Lisbon)? Is there a new relationship with the public? Are online communication tools creating a new kind of community? What changes are these tools introducing in the creative process? In what way the availability of content and its archive contribute to the organization memory? Among several references, we will approach the two-way communication model that James E. Grunig & Todd T. Hunt (1984) already presented and the concept of mass self-communication of Manuel Castells (2010). Castells also tells us that we have moved from traditional media to a system of communication networks. For Scott Kirsner (2010), we have entered an era of digital creativity, where artists have the tools to do what they imagined and the public no longer wants to just consume cultural goods, but instead to have a voice and participate. The creativity process is now depending on the public choice as they wander through the screen. It is the receiver who owns an object which can be exchanged. Virtual reality has encouraged the receiver to abandon its position of passive observer and to become a participant agent, which implies a challenge to organizations: inventing new forms of interfaces. Therefore, we intend to find new and effective online tools that can be used by cultural organizations; the best way to manage them; to show how organizations can create a community with the public and how the availability of online content and its archive can contribute to the organizations’ memory.

Constructing and Using Multimodal Narratives to Research in Science Education: Contributions Based on Practical Classroom

This study deals with the problem of how to collect genuine and useful data about science classroom practices, and preserving the complex and holistic nature of teaching and learning. Additionally, we were looking for an instrument that would allow comparability and verifiability for teaching and research purposes. Given the multimodality of teaching and learning processes, we developed the multimodal narrative (MN), which describes what happens during a task and incorporates data such as examples of students’ work.

Enhancing Multimodal Silent Speech Interfaces With Feature Selection

In research on Silent Speech Interfaces (SSI), different sources of information (modalities) have been combined, aiming at obtaining better performance than the individual modalities. However, when combining these modalities, the dimensionality of the feature space rapidly increases, yielding the well-known "curse of dimensionality". As a consequence, in order to extract useful information from this data, one has to resort to feature selection (FS) techniques to lower the dimensionality of the learning space. In this paper, we assess the impact of FS techniques for silent speech data, in a dataset with 4 non-invasive and promising modalities, namely: video, depth, ultrasonic Doppler sensing, and surface electromyography. We consider two supervised (mutual information and Fisher's ratio) and two unsupervised (meanmedian and arithmetic mean geometric mean) FS filters. The evaluation was made by assessing the classification accuracy (word recognition error) of three well-known classifiers (knearest neighbors, support vector machines, and dynamic time warping). The key results of this study show that both unsupervised and supervised FS techniques improve on the classification accuracy on both individual and combined modalities. For instance, on the video component, we attain relative performance gains of 36.2% in error rates. FS is also useful as pre-processing for feature fusion. Copyright © 2014 ISCA.

Parallel texts alignment

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial para obtenção do grau de Mestre em Engenharia Informática

Literary texts and intercultural learning: exploring new directions

Hoje em dia, a interculturalidade faz parte das análises, estudos e pensamentos de muitos estudiosos e teóricos das mais diversas disciplinas.

Sistema multimodal para captura e anotação de vídeo

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Personalization platform for multimodal ubiquitous computing applications

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Texts prepared for delivery at Paul Krugman's honorary doctorate by three Lisbon universities on 27 February, 2012

Three texts were prepared for delivery at the first honorary doctorate awarded simultaneously by three Lisbon universities on 27 February, 2012: an introduction, a lecture and a comment. The event included the award of member of the Lisbon Academy of Science (ACL) by Manuel Jacinto Nunes, dean of the economics and finance section of ACL who proposed his name and Olivier Blanchard’s on the 30th anniversary of James Tobin receiving an honorary doctorate from Nova University. On 24 February , Paul Krugman visited ACL and participated in a session of the project dubbed “Letter to the lusofonia Queen”. Since this project is promoted by Nova SBE’s Center for Globalization and Governance and has been featured in some of the graduate courses, a short note on the meeting is included in annex. On 15 June, the three universities authorized an edition in Portuguese and donated the copyrights to a student award on “Krugman economics”, in a way still to be determined by the editor. The lecture and the comment will be translated as soon as a suitable publisher is found. Since a lot of the teaching at Nova SBE is in English, it seemed appropriate to reproduce the original texts in the order in which they were presented. A lively question and answer period was also recorded by Nova TV and should be made available in the book, together with highlights of the media coverage. Introduced as a “militant economist”, he speaks about a crisis “his mind loves but does not let the heart forget the poor and the unemployed”. The Nobel prize winner described as a“progressist pessimist of the world economy” concludes with a severe indictment of the profession. “In normal times, when things are going pretty well, the world can function reasonably well without professional economic advice. It’s in times of crisis, when practical experience suddenly proves useless and events are beyond anyone’s normal experience, that we need professors with their models to light the path forward. And when the moment came, we failed”. The comment, by the official responsible for Paul Krugman’s mission to Portugal in 1976, contains an equally dire prediction: “I would very much like to see in the near future the weakening of the influence not only of freshwater economists but also of their conservative European followers. But I fear that this will not happen until we find ourselves in a more calamitous situation than at present”. Fortunately Silva Lopes closes in the hope “that the ideas of Paul Krugman will soon have more influence in policy makers than at present seems to be the case”.

Risky behavior regarding drug use and HIV infection: an Internet questionnaire coupled with short education texts for Portuguese speakers

Surveys of risky behavior relating to HIV/AIDS are generally made for groups at risk of infection, for which HIV/AIDS prevalence is usually expected to be higher than in the general population. Therefore, an educational homepage in Portuguese was created on the Internet to inform/ask internauts regarding knowledge and behavior. The internauts were classified as adolescents (13 to 25 years) and adults (>25 years). The number of STDs was reported as 1. 8 ± 2. 6 infections (range: 1 to 20 infections); 43% used condoms during sexual intercourse. Alcohol consumption was reported by 63% and illicit drug use by 32% (marijuana 24% and inhalants 15%). Among the adolescents, 31% did not classified alcohol as a drug. The adults more frequently reported homosexuality, anal intercourse and STDs, although the adolescents also presented high rates of risky behavior. These results show the need to reach out to internauts through better control strategies. Different types of strategies must be encouraged, in order to reach people that use this means of communication and entertainment.

Automatic extraction of concepts from texts and applications

The extraction of relevant terms from texts is an extensively researched task in Text- Mining. Relevant terms have been applied in areas such as Information Retrieval or document clustering and classification. However, relevance has a rather fuzzy nature since the classification of some terms as relevant or not relevant is not consensual. For instance, while words such as "president" and "republic" are generally considered relevant by human evaluators, and words like "the" and "or" are not, terms such as "read" and "finish" gather no consensus about their semantic and informativeness. Concepts, on the other hand, have a less fuzzy nature. Therefore, instead of deciding on the relevance of a term during the extraction phase, as most extractors do, I propose to first extract, from texts, what I have called generic concepts (all concepts) and postpone the decision about relevance for downstream applications, accordingly to their needs. For instance, a keyword extractor may assume that the most relevant keywords are the most frequent concepts on the documents. Moreover, most statistical extractors are incapable of extracting single-word and multi-word expressions using the same methodology. These factors led to the development of the ConceptExtractor, a statistical and language-independent methodology which is explained in Part I of this thesis. In Part II, I will show that the automatic extraction of concepts has great applicability. For instance, for the extraction of keywords from documents, using the Tf-Idf metric only on concepts yields better results than using Tf-Idf without concepts, specially for multi-words. In addition, since concepts can be semantically related to other concepts, this allows us to build implicit document descriptors. These applications led to published work. Finally, I will present some work that, although not published yet, is briefly discussed in this document.