A multimodal genre recognition prototype

In this paper, a multimodal and interactive prototype to perform music genre classification is presented. The system is oriented to multi-part files in symbolic format but it can be adapted using a transcription system to transform audio content in music scores. This prototype uses different sources of information to give a possible answer to the user. It has been developed to allow a human expert to interact with the system to improve its results. In its current implementation, it offers a limited range of interaction and multimodality. Further development aimed at full interactivity and multimodal interactions is discussed.

A Prototype for a Multimodal Biometric Security system based on Face and Audio Signatures

Any automatically measurable, robust and distinctive physical characteristic or personal trait that can be used to identify an individual or verify the claimed identity of an individual, referred to as biometrics, has gained significant interest in the wake of heightened concerns about security and rapid advancements in networking, communication and mobility. Multimodal biometrics is expected to be ultra-secure and reliable, due to the presence of multiple and independent—verification clues. In this study, a multimodal biometric system utilising audio and facial signatures has been implemented and error analysis has been carried out. A total of one thousand face images and 250 sound tracks of 50 users are used for training the proposed system. To account for the attempts of the unregistered signatures data of 25 new users are tested. The short term spectral features were extracted from the sound data and Vector Quantization was done using K-means algorithm. Face images are identified based on Eigen face approach using Principal Component Analysis. The success rate of multimodal system using speech and face is higher when compared to individual unimodal recognition systems

Sistema multimodal para la planificación de viajes en transportes públicos de código abierto y basado en estándares 'de facto' = A multimodal journey planner based on open source software and 'de facto' standards

Este artículo muestra cómo con bajo coste y riesgo se puede desarrollar un sistema de planificación de viaje multimodal, basado en un enfoque de código abierto y estándares ‘de facto’. Se ha desarrollado completamente una solución de código abierto para un sistema de información de transporte público puerta a puerta basado en estándares ‘de facto’. El cálculo de rutas se realiza mediante Graphserver, mientras que la cartografía se basa en OpenStreetMap. También se ha demostrado cómo exportar una base de datos real de horarios de transporte público como la del operador ETM (Empresa de Transporte Metropolitano de València) a la especificación de Google Transit, para permitir el cálculo de rutas, tanto desde nuestro prototipo como desde Google Transit

Uniquitous internet middleware: architecture design and prototype evaluation

Technology advances in recent years have dramatically changed the way users exploit contents and services available on the Internet, by enforcing pervasive and mobile computing scenarios and enabling access to networked resources almost from everywhere, at anytime, and independently of the device in use. In addition, people increasingly require to customize their experience, by exploiting specific device capabilities and limitations, inherent features of the communication channel in use, and interaction paradigms that significantly differ from the traditional request/response one. So-called Ubiquitous Internet scenario calls for solutions that address many different challenges, such as device mobility, session management, content adaptation, context-awareness and the provisioning of multimodal interfaces. Moreover, new service opportunities demand simple and effective ways to integrate existing resources into new and value added applications, that can also undergo run-time modifications, according to ever-changing execution conditions. Despite service-oriented architectural models are gaining momentum to tame the increasing complexity of composing and orchestrating distributed and heterogeneous functionalities, existing solutions generally lack a unified approach and only provide support for specific Ubiquitous Internet aspects. Moreover, they usually target rather static scenarios and scarcely support the dynamic nature of pervasive access to Internet resources, that can make existing compositions soon become obsolete or inadequate, hence in need of reconfiguration. This thesis proposes a novel middleware approach to comprehensively deal with Ubiquitous Internet facets and assist in establishing innovative application scenarios. We claim that a truly viable ubiquity support infrastructure must neatly decouple distributed resources to integrate and push any kind of content-related logic outside its core layers, by keeping only management and coordination responsibilities. Furthermore, we promote an innovative, open, and dynamic resource composition model that allows to easily describe and enforce complex scenario requirements, and to suitably react to changes in the execution conditions.

Multimodal field data entry:performance and usability issues

Mobile technologies have yet to be widely adopted by the Architectural, Engineering, and Construction (AEC) industry despite being one of the major growth areas in computing in recent years. This lack of uptake in the AEC industry is likely due, in large part, to the combination of small screen size and inappropriate interaction demands of current mobile technologies. This paper discusses the scope for multimodal interaction design with a specific focus on speech-based interaction to enhance the suitability of mobile technology use within the AEC industry by broadening the field data input capabilities of such technologies. To investigate the appropriateness of using multimodal technology for field data collection in the AEC industry, we have developed a prototype Multimodal Field Data Entry (MFDE) application. This application, which allows concrete testing technicians to record quality control data in the field, has been designed to support two different modalities of data input speech-based data entry and stylus-based data entry. To compare the effectiveness or usability of, and user preference for, the different input options, we have designed a comprehensive lab-based evaluation of the application. To appropriately reflect the anticipated context of use within the study design, careful consideration had to be given to the key elements of a construction site that would potentially influence a test technician's ability to use the input techniques. These considerations and the resultant evaluation design are discussed in detail in this paper.

Audio games: investigation of the potential through prototype development

This article discusses the potential of audio games based on the evaluation of three projects: a story-driven audio role-playing game (RPG), an interactive audiobook with RPG elements, and a set of casual sound-based games. The potential is understood, both in popularity and playability terms. The first factor is connected to the degree of players’ interest, while the second one to the degree of their engagement in sound-based game worlds. Although presented projects are embedded within the landscape of past and contemporary audio games and gaming platforms, the authors reach into the near future, concluding with possible development directions for this non-visual interactive entertainment.

Multimodal Mri-based Study In Patients With Spg4 Mutations.

Mutations in the SPG4 gene (SPG4-HSP) are the most frequent cause of hereditary spastic paraplegia, but the extent of the neurodegeneration related to the disease is not yet known. Therefore, our objective is to identify regions of the central nervous system damaged in patients with SPG4-HSP using a multi-modal neuroimaging approach. In addition, we aimed to identify possible clinical correlates of such damage. Eleven patients (mean age 46.0 ± 15.0 years, 8 men) with molecular confirmation of hereditary spastic paraplegia, and 23 matched healthy controls (mean age 51.4 ± 14.1years, 17 men) underwent MRI scans in a 3T scanner. We used 3D T1 images to perform volumetric measurements of the brain and spinal cord. We then performed tract-based spatial statistics and tractography analyses of diffusion tensor images to assess microstructural integrity of white matter tracts. Disease severity was quantified with the Spastic Paraplegia Rating Scale. Correlations were then carried out between MRI metrics and clinical data. Volumetric analyses did not identify macroscopic abnormalities in the brain of hereditary spastic paraplegia patients. In contrast, we found extensive fractional anisotropy reduction in the corticospinal tracts, cingulate gyri and splenium of the corpus callosum. Spinal cord morphometry identified atrophy without flattening in the group of patients with hereditary spastic paraplegia. Fractional anisotropy of the corpus callosum and pyramidal tracts did correlate with disease severity. Hereditary spastic paraplegia is characterized by relative sparing of the cortical mantle and remarkable damage to the distal portions of the corticospinal tracts, extending into the spinal cord.

Ultraviolet analysis of donated corneas: a portable prototype

As technology improves human vision, some procedures currently performed may be causing a decrease of the natural UV protection of the cornea. A portable dual beam system prototype was assembled for physicians for clinical studies of these effects on the corneas endowing two types of 300-400 nm evaluations: 1, regularly donated corneas and 2, simulating refractive keratectomy by corneal lamellae removal. The system performs 500 measurements/s, providing +/- 0.25% precision for the transmittance. The measurements performed on the prototype are 95% in agreement with Cary 17 and HR4000CG-UV-NIR Ocean Optics spectrophotometers. Preliminary studies on cadaveric corneas demonstrate that, as the stromal layer is reduced (similar to 150 mu m depth), there is significant loss-an average of 7.1%.-of the cornea's natural UV protection. The prototype is being tested in an eye bank for routine evaluation of donor corneas. (C) 2010 Optical Society of America

Multimodal passive vibration control of sandwich beams with shunted shear piezoelectric materials

This work presents a performance analysis of multimodal passive vibration control of a sandwich beam using shear piezoelectric materials, embedded in a sandwich beam core, connected to independent resistive shunt circuits. Shear piezoelectric actuators were recently shown to be more interesting for higher frequencies and stiffer structures. In particular, for shunted damping, it was shown that equivalent material loss factors of up to 31% can be achieved by optimizing the shunt circuit. In the present work, special attention is given to the design of multimodal vibration control through independent shunted shear piezoelectric sensors. In particular, a parametric analysis is performed to evaluate optimal configurations for a set of modes to be damped. Then, a methodology to evaluate the modal damping resulting from each shunted piezoelectric sensor is presented using the modal strain energy method. Results show that modal damping factors of 1%-2% can be obtained for three selected vibration modes.

Pressure-Frequency Sensing Subxiphoid Access System for Use in Percutaneous Cardiac Electrophysiology: Prototype Design and Pilot Study Results

We have designed, built, and tested an early prototype of a novel subxiphoid access system intended to facilitate epicardial electrophysiology, but with possible applications elsewhere in the body. The present version of the system consists of a commercially available insertion needle, a miniature pressure sensor and interconnect tubing, read-out electronics to monitor the pressures measured during the access procedure, and a host computer with user-interface software. The nominal resolution of the system is <0.1 mmHg, and it has deviations from linearity of <1%. During a pilot series of human clinical studies with this system, as well as in an auxiliary study done with an independent method, we observed that the pericardial space contained pressure-frequency components related to both the heart rate and respiratory rate, while the thorax contained components related only to the respiratory rate, a previously unobserved finding that could facilitate access to the pericardial space. We present and discuss the design principles, details of construction, and performance characteristics of this system.

Application of a wrist dosimeter prototype for radiation monitoring ((153)Sm) during a therapeutic procedure simulation

Gamma and beta radiation emitting radiopharmaceuticals are handled in nuclear medicine services, and in many cases there is only individual monitoring of gamma radiation. In this paper, the results obtained using a wrist dosimeter prototype (CaSO(4):Dy + Teflon pellets) show that the doses for workers occupationally exposed to beta radiation from (153)Sm are not negligible. It is important that this dose is evaluated, and it has to be taken into consideration in the individual monitoring system.

Exercise Echocardiography in Cryopreserved Aortic Homografts: Comparison of a Prototype Stentless, a Stented Bioprosthesis, and Native Aortic Valves

Background: Aortic valve replacement with a cryopreserved aortic homograft (CH) is an attractive alternative to bioprosthesis implantation. The aim of the study was to compare the hemodynamic performance of CH implanted with aortic root inclusion compared to prototype stentless (SS) bioprosthesis, standard stented (SD) bioprosthesis, and a native aortic valve. Methods: Hemodynamics and Doppler echocardiographic measurements such as left ventricular ejection fraction, aortic valve orifice area index (AVOAI), mean and maximal transvalvular gradients, were obtained at rest and immediately after exercise in 28 patients after aortic valve replacement with CH (n = 10), SS (n = 9), or SD (n = 9), and in a control group (CG) of 15 normal volunteers. Results: Rest and peak exercise heart rate and workload achieved were not different among the groups. Baseline AVOAI was larger for CH and CG compared to SS and SD groups (P < 0.05). Maximal and mean transvalvular pressure gradients at rest were lower for CH compared to SS and SD groups (P < 0.05), but higher than CG (P < 0,05). Conclusion: Implanted aortic CH had better hemodynamic performance than SS and SD bioprosthesis and similar to native normal aortic valves, both at rest and immediately after exercise. (ECHOCARDIOGRAPHY, Volume 26, November 2009).

Interface Multimodal com Predição de Movimentos para Uso em Reabilitação de Membros Inferiores

Esta dissertação apresenta o desenvolvimento de uma plataforma multimodal de aquisição e processamento de sinais. O projeto proposto insere-se no contexto do desenvolvimento de interfaces multimodais para aplicação em dispositivos robóticos cujo propósito é a reabilitação motora adaptando o controle destes dispositivos de acordo com a intenção do usuário. A interface desenvolvida adquire, sincroniza e processa sinais eletroencefalográficos (EEG), eletromiográficos (EMG) e sinais provenientes de sensores inerciais (IMUs). A aquisição dos dados é feita em experimentos realizados com sujeitos saudáveis que executam tarefas motoras de membros inferiores. O objetivo é analisar a intenção de movimento, a ativação muscular e o início efetivo dos movimentos realizados, respectivamente, através dos sinais de EEG, EMG e IMUs. Para este fim, uma análise offline foi realizada. Nessa análise, são utilizadas técnicas de processamento dos sinais biológicos e técnicas para processar sinais provenientes de sensores inerciais. A partir destes, os ângulos da articulação do joelho também são aferidos ao longo dos movimentos. Um protocolo experimental de testes foi proposto para as tarefas realizadas. Os resultados demonstraram que o sistema proposto foi capaz de adquirir, sincronizar, processar e classificar os sinais combinadamente. Análises acerca da acurácia dos classificadores utilizados mostraram que a interface foi capaz de identificar intenção de movimento em 76, 0 ± 18, 2% dos movimentos. A maior média de tempo de antecipação ao movimento foi obtida através da análise do sinal de EEG e foi de 716, 0±546, 1 milisegundos. A partir da análise apenas do sinal de EMG, este valor foi de 88, 34 ± 67, 28 milisegundos. Os resultados das etapas de processamento dos sinais biológicos, a medição dos ângulos da articulação, bem como os valores de acurácia e tempo de antecipação ao movimento se mostraram em conformidade com a literatura atual relacionada.

Robust temporal alignment of multimodal cardiac sequences

Given the dynamic nature of cardiac function, correct temporal alignment of pre-operative models and intraoperative images is crucial for augmented reality in cardiac image-guided interventions. As such, the current study focuses on the development of an image-based strategy for temporal alignment of multimodal cardiac imaging sequences, such as cine Magnetic Resonance Imaging (MRI) or 3D Ultrasound (US). First, we derive a robust, modality-independent signal from the image sequences, estimated by computing the normalized crosscorrelation between each frame in the temporal sequence and the end-diastolic frame. This signal is a resembler for the left-ventricle (LV) volume curve over time, whose variation indicates di erent temporal landmarks of the cardiac cycle. We then perform the temporal alignment of these surrogate signals derived from MRI and US sequences of the same patient through Dynamic Time Warping (DTW), allowing to synchronize both sequences. The proposed framework was evaluated in 98 patients, which have undergone both 3D+t MRI and US scans. The end-systolic frame could be accurately estimated as the minimum of the image-derived surrogate signal, presenting a relative error of 1:6 1:9% and 4:0 4:2% for the MRI and US sequences, respectively, thus supporting its association with key temporal instants of the cardiac cycle. The use of DTW reduces the desynchronization of the cardiac events in MRI and US sequences, allowing to temporally align multimodal cardiac imaging sequences. Overall, a generic, fast and accurate method for temporal synchronization of MRI and US sequences of the same patient was introduced. This approach could be straightforwardly used for the correct temporal alignment of pre-operative MRI information and intra-operative US images.

MASDScheGATS: a prototype system for dynamic scheduling

A manufacturing system has a natural dynamic nature observed through several kinds of random occurrences and perturbations on working conditions and requirements over time. For this kind of environment it is important the ability to efficient and effectively adapt, on a continuous basis, existing schedules according to the referred disturbances, keeping performance levels. The application of Meta-Heuristics and Multi-Agent Systems to the resolution of this class of real world scheduling problems seems really promising. This paper presents a prototype for MASDScheGATS (Multi-Agent System for Distributed Manufacturing Scheduling with Genetic Algorithms and Tabu Search).