Robust Audio-Visual Speech Recognition under Noisy Audio-Video Conditions

This paper presents the maximum weighted stream posterior (MWSP) model as a robust and efficient stream integration method for audio-visual speech recognition in environments, where the audio or video streams may be subjected to unknown and time-varying corruption. A significant advantage of MWSP is that it does not require any specific measurements of the signal in either stream to calculate appropriate stream weights during recognition, and as such it is modality-independent. This also means that MWSP complements and can be used alongside many of the other approaches that have been proposed in the literature for this problem. For evaluation we used the large XM2VTS database for speaker-independent audio-visual speech recognition. The extensive tests include both clean and corrupted utterances with corruption added in either/both the video and audio streams using a variety of types (e.g., MPEG-4 video compression) and levels of noise. The experiments show that this approach gives excellent performance in comparison to another well-known dynamic stream weighting approach and also compared to any fixed-weighted integration approach in both clean conditions or when noise is added to either stream. Furthermore, our experiments show that the MWSP approach dynamically selects suitable integration weights on a frame-by-frame basis according to the level of noise in the streams and also according to the naturally fluctuating relative reliability of the modalities even in clean conditions. The MWSP approach is shown to maintain robust recognition performance in all tested conditions, while requiring no prior knowledge about the type or level of noise.

Processing of short auditory stimuli:The rapid audio sequential presentation paradigm (RASP)

Human listeners seem to be remarkably able to recognise acoustic sound sources based on timbre cues. Here we describe a psychophysical paradigm to estimate the time it takes to recognise a set of complex sounds differing only in timbre cues: both in terms of the minimum duration of the sounds and the inferred neural processing time. Listeners had to respond to the human voice while ignoring a set of distractors. All sounds were recorded from natural sources over the same pitch range and equalised to the same duration and power. In a first experiment, stimuli were gated in time with a raised-cosine window of variable duration and random onset time. A voice/non-voice (yes/no) task was used. Performance, as measured by d', remained above chance for the shortest sounds tested (2 ms); d's above 1 were observed for durations longer than or equal to 8 ms. Then, we constructed sequences of short sounds presented in rapid succession. Listeners were asked to report the presence of a single voice token that could occur at a random position within the sequence. This method is analogous to the "rapid sequential visual presentation" paradigm (RSVP), which has been used to evaluate neural processing time for images. For 500-ms sequences made of 32-ms and 16-ms sounds, d' remained above chance for presentation rates of up to 30 sounds per second. There was no effect of the pitch relation between successive sounds: identical for all sounds in the sequence or random for each sound. This implies that the task was not determined by streaming or forward masking, as both phenomena would predict better performance for the random pitch condition. Overall, the recognition of familiar sound categories such as the voice seems to be surprisingly fast, both in terms of the acoustic duration required and of the underlying neural time constants.

Wearable inkjet-printed antenna performance for medical applications at 868/915 MHz

The use of biosensors attached to the body for health monitoring is now readily accepted, and the merits of such systems and their potential impact on healthcare receive much attention. Wearable medical systems used in clinical applications to monitor vital signs must be comfortable to wear, yet have robust performance to ensure reliable communications links. Additionally, and vital to the success of these innovations, is that these solutions are disposable to avoid risk of patient infection and this means that they must be ultra-low cost. Antennas optimized for printing using conductive inks offer new exciting advances in making a truly disposable solution.

Mobile Wearable Communications

This special issue provides the latest research and development on wireless mobile wearable communications. According to a report by Juniper Research, the market value of connected wearable devices is expected to reach $1.5 billion by 2014, and the shipment of wearable devices may reach 70 million by 2017. Good examples of wearable devices are the prominent Google Glass and Microsoft HoloLens. As wearable technology is rapidly penetrating our daily life, mobile wearable communication is becoming a new communication paradigm. Mobile wearable device communications create new challenges compared to ordinary sensor networks and short-range communication. In mobile wearable communications, devices communicate with each other in a peer-to-peer fashion or client-server fashion and also communicate with aggregation points (e.g., smartphones, tablets, and gateway nodes). Wearable devices are expected to integrate multiple radio technologies for various applications' needs with small power consumption and low transmission delays. These devices can hence collect, interpret, transmit, and exchange data among supporting components, other wearable devices, and the Internet. Such data are not limited to people's personal biomedical information but also include human-centric social and contextual data. The success of mobile wearable technology depends on communication and networking architectures that support efficient and secure end-to-end information flows. A key design consideration of future wearable devices is the ability to ubiquitously connect to smartphones or the Internet with very low energy consumption. Radio propagation and, accordingly, channel models are also different from those in other existing wireless technologies. A huge number of connected wearable devices require novel big data processing algorithms, efficient storage solutions, cloud-assisted infrastructures, and spectrum-efficient communications technologies.

Channel characterisation for indoor wearable active RFID at 868 MHz

Active radio-frequency identification systems that are used for the localisation and tracking of people will be subject to the same body centric processes that impact other forms of wearable communications. To achieve the goal of creating body worn tags with multiyear life spans, it will be necessary to gain an understanding of the channel conditions which are likely to impact the reader-tag interrogation process. In this paper we present the preliminary results of an indoor channel measurement campaign conducted at 868 MHz aimed at understanding and modelling signal characteristics for a wrist-worn tag. Using a model selection process based on the Akaike Information Criterion, the lognormal distribution was selected most often to describe the received signal amplitude. Parameter estimates are provided so that the channels investigated in this study may be readily simulated.

Wearable antennas for medical monitoring systems

A wearable silver nano particle inkjet printed antenna suitable for wireless biomedical sensing is presented. The performance is evaluated on a synthetic variable layered phantom test-bed, representative of human tissue for operation in the 868/915 MHz, and 2400 MHz industrial, scientific and medical frequency bands. Antenna radiation efficiency measurements on the phantom were compared with antennas prototyped with copper. Total radiation efficiencies up to ???6.5 dB are reported, with less than 0.5 dB difference in performance between copper and silver nano particle variants, showing promising application for low-cost disposable wireless sensing.

Using Audio-visual Presentation to Teach Global Mindedness in Social Work Education

Direct experience of social work in another country is making an increasingly important contribution to internationalising the social work academic curriculum together with the cultural competency of students. However at present this opportunity is still restricted to a limited number of students. The aim of this paper is to describe and reflect on the production of an audio-visual presentation as representing the experience of three students who participated in an exchange with a social work programme in Pune, India. It describes and assesses the rationale, production and use of video to capture student learning from the Belfast/Pune exchange. We also describe the use of the video in a classroom setting with a year group of 53 students from a younger cohort. This exercise aimed to stimulate students’ curiosity about international dimensions of social work and add to their awareness of poverty, social justice, cultural competence and community social work as global issues. Written classroom feedback informs our discussion of the technical as well as the pedagogical benefits and challenges of this approach. We conclude that some benefit of audio-visual presentation in helping students connect with diverse cultural contexts, but that a complementary discussion challenging stereotyped viewpoints and unconscious professional imperialism is also crucial.

Heterogeneous error-resilient scheme for spectral analysis in ultra-low power wearable electrocardiogram devices

Wearable devices performing advanced bio-signal analysis algorithms are aimed to foster a revolution in healthcare provision of chronic cardiac diseases. In this context, energy efficiency is of paramount importance, as long-term monitoring must be ensured while relying on a tiny power source. Operating at a scaled supply voltage, just above the threshold voltage, effectively helps in saving substantial energy, but it makes circuits, and especially memories, more prone to errors, threatening the correct execution of algorithms. The use of error detection and correction codes may help to protect the entire memory content, however it incurs in large area and energy overheads which may not be compatible with the tight energy budgets of wearable systems. To cope with this challenge, in this paper we propose to limit the overhead of traditional schemes by selectively detecting and correcting errors only in data highly impacting the end-to-end quality of service of ultra-low power wearable electrocardiogram (ECG) devices. This partition adopts the protection of either significant words or significant bits of each data element, according to the application characteristics (statistical properties of the data in the application buffers), and its impact in determining the output. The proposed heterogeneous error protection scheme in real ECG signals allows substantial energy savings (11% in wearable devices) compared to state-of-the-art approaches, like ECC, in which the whole memory is protected against errors. At the same time, it also results in negligible output quality degradation in the evaluated power spectrum analysis application of ECG signals.

The Use of Audio Podcasts to Enhance the Delivery of a Computer Networks Course

Experience obtained in the support of mobile learning using podcast audio is reported. The paper outlines design, storage and distribution via a web site. An initial evaluation of the uptake of the approach in a final year computing module was undertaken. Audio objects were tailored to meet different pedagogical needs resulting in a repository of persistent glossary terms and disposable audio lectures distributed by podcasting. An aim of our approach is to document the interest from the students, and evaluate the potential of mobile learning for supplementing revision

SWORD: an intelligent vibratory wearable device to improve rehabilitation in stroke patients

Anualmente ocorrem cerca de 16 milhões AVCs em todo o mundo. Cerca de metade dos sobreviventes irá apresentar défice motor que necessitará de reabilitação na janela dos 3 aos 6 meses depois do AVC. Nos países desenvolvidos, é estimado que os custos com AVCs representem cerca de 0.27% do Produto Interno Bruto de cada País. Esta situação implica um enorme peso social e financeiro. Paradoxalmente a esta situação, é aceite na comunidade médica a necessidade de serviços de reabilitação motora mais intensivos e centrados no doente. Na revisão do estado da arte, demonstra-se o arquétipo que relaciona metodologias terapêuticas mais intensivas com uma mais proficiente reabilitação motora do doente. Revelam-se também as falhas nas soluções tecnológicas existentes que apresentam uma elevada complexidade e custo associado de aquisição e manutenção. Desta forma, a pergunta que suporta o trabalho de doutoramento seguido inquire a possibilidade de criar um novo dispositivo de simples utilização e de baixo custo, capaz de apoiar uma recuperação motora mais eficiente de um doente após AVC, aliando intensidade com determinação da correcção dos movimentos realizados relativamente aos prescritos. Propondo o uso do estímulo vibratório como uma ferramenta proprioceptiva de intervenção terapêutica a usar no novo dispositivo, demonstra-se a tolerabilidade a este tipo de estímulos através do teste duma primeira versão do sistema apenas com a componente de estimulação num primeiro grupo de 5 doentes. Esta fase validará o subsequente desenvolvimento do sistema SWORD. Projectando o sistema SWORD como uma ferramenta complementar que integra as componentes de avaliação motora e intervenção proprioceptiva por estimulação, é descrito o desenvolvimento da componente de quantificação de movimento que o integra. São apresentadas as diversas soluções estudadas e o algoritmo que representa a implementação final baseada na fusão sensorial das medidas provenientes de três sensores: acelerómetro, giroscópio e magnetómetro. O teste ao sistema SWORD, quando comparado com o método de reabilitação tradicional, mostrou um ganho considerável de intensidade e qualidade na execução motora para 4 dos 5 doentes testados num segundo grupo experimental. É mostrada a versatilidade do sistema SWORD através do desenvolvimento do módulo de Tele-Reabilitação que complementa a componente de quantificação de movimento com uma interface gráfica de feedback e uma ferramenta de análise remota da evolução motora do doente. Finalmente, a partir da componente de quantificação de movimento, foi ainda desenvolvida uma versão para avaliação motora automatizada, implementada a partir da escala WMFT, que visa retirar o factor subjectivo da avaliação humana presente nas escalas de avaliação motora usadas em Neurologia. Esta versão do sistema foi testada num terceiro grupo experimental de cinco doentes.

Identificação de utilizadores seniores em televisão interativa (iTV) : uma matriz de decisão tecnológica

Desde que surgiu há mais de 50 anos, a televisão sofreu muitas transformações, tanto ao nível tecnológico (por exemplo com a passagem da emissão a preto/branco para cor, o som analógico para digital, a difusão digital) como a nível da sua influência nas sociedades. Entre outros fatores de ordem tecnológica, a consolidação da Internet com o seu elevado nível de personalização, da experiência de utilização, e a sua enorme quantidade de conteúdos disponíveis, catapultou a televisão no sentido de esta se tornar mais interativa. Assim, o telespectador passou a poder usufruir de uma experiência televisiva que pode, por um lado, ser mais participativa, sendo-lhe possível, por exemplo, alvitrar sobre a qualidade de um programa enquanto assiste à sua exibição, e, por outro, ser mais personalizada, possibilitando-lhe, por exemplo, receber conteúdos automaticamente adequados ao seu perfil e contexto. No entanto, esta experiência mais participativa e personalizável carece de uma identificação, idealmente automática e não intrusiva, de quem pode beneficiar da mesma – o telespectador. Contudo, e apesar de significativos avanços na área da televisão interativa, tanto ao nível da infraestrutura de suporte como ao nível dos serviços disponibilizados, a identificação dos utilizadores é, ainda, uma área de estudo com muitos aspetos por compreender. Os seniores, em particular, são grandes consumidores de televisão e representam uma fatia muito considerável das pessoas que podem beneficiar das potencialidades disponibilizadas pela interatividade presente em muitos serviços atuais. Um número crescente destes serviços são desenhados com o objetivo de promoverem um envelhecimento ativo e um concreto apoio à vida, pelo que os seniores podem beneficiar, em vários aspetos do seu quotidiano, se os utilizarem. Nesta faixa etária, a identificação de utilizadores tem, como elemento potenciador da experiência de utilização, um papel especialmente importante ao nível de um aproveitamento personalizado e dirigido destes serviços. No entanto, atendendo às diferentes combinações de características físicas, sensoriais, cognitivas e, mesmo, de literacia digital que tipificam os seniores, perspetivou-se existir uma dependência do perfil do utilizador na seleção do método de identificação mais adequado, os quais podem ser baseados, por exemplo, num leitor de impressões digitais, instalado no telecomando; na leitura de uma wearable tag ou de um cartão RFiD; no reconhecimento da face e, eventualmente, na voz do utilizador. Assim, a inerente investigação desenrolou-se em várias fases, no sentido de permitir alicerçar a construção de uma matriz de decisão tecnológica que, em função do perfil de utilizador, selecione o sistema de identificação mais adequado. O procedimento metodológico inerente à construção desta matriz de decisão, passou por um longo processo envolvendo utilizadores reais, que se iniciou com a realização de entrevistas exploratórias com o objetivo de permitir conhecer melhor os seniores e a forma como estes encaram a tecnologia e, mais concretamente, a televisão interativa. Foi depois implementado um protótipo de alta-fidelidade, completamente funcional, para a realização de testes com o objetivo de perceber qual a preferência relativamente a um subconjunto de tecnologias de identificação. Estes testes, uma vez que não permitiram testar todas as tecnologias em estudo, revelaram-se inconclusivos, porém permitiram reforçar a necessidade de identificar e caracterizar os referidos aspetos do perfil do utilizador que podem interferir na sua preferência relativamente ao sistema de identificação. As características identificadas constituíram-se como os parâmetros de entrada da matriz, sendo que para preencher as respetivas células realizaramse testes de aceitação, com um conjunto de seniores, tendo por base um protótipo, wizard of oz, especificamente implementado para permitir experienciar todas as tecnologias em estudo. Estes testes foram precedidos pela avaliação das capacidades funcionais dos participantes, nos diversos parâmetros definidos. Este texto relata, assim, todo o processo de investigação que foi conduzido, terminando com uma descrição de exemplos de utilização da matriz de decisão implementada e com a identificação de potenciais caminhos de desenvolvimento deste trabalho.

Automatic measurement of key ski jumping phases and temporal events with a wearable system.

We propose a new method, based on inertial sensors, to automatically measure at high frequency the durations of the main phases of ski jumping (i.e. take-off release, take-off, and early flight). The kinematics of the ski jumping movement were recorded by four inertial sensors, attached to the thigh and shank of junior athletes, for 40 jumps performed during indoor conditions and 36 jumps in field conditions. An algorithm was designed to detect temporal events from the recorded signals and to estimate the duration of each phase. These durations were evaluated against a reference camera-based motion capture system and by trainers conducting video observations. The precision for the take-off release and take-off durations (indoor < 39 ms, outdoor = 27 ms) can be considered technically valid for performance assessment. The errors for early flight duration (indoor = 22 ms, outdoor = 119 ms) were comparable to the trainers' variability and should be interpreted with caution. No significant changes in the error were noted between indoor and outdoor conditions, and individual jumping technique did not influence the error of take-off release and take-off. Therefore, the proposed system can provide valuable information for performance evaluation of ski jumpers during training sessions.

Non-negative matrix decomposition approaches to frequency domain analysis of music audio signals

On étudie l’application des algorithmes de décomposition matricielles tel que la Factorisation Matricielle Non-négative (FMN), aux représentations fréquentielles de signaux audio musicaux. Ces algorithmes, dirigés par une fonction d’erreur de reconstruction, apprennent un ensemble de fonctions de base et un ensemble de coef- ficients correspondants qui approximent le signal d’entrée. On compare l’utilisation de trois fonctions d’erreur de reconstruction quand la FMN est appliquée à des gammes monophoniques et harmonisées: moindre carré, divergence Kullback-Leibler, et une mesure de divergence dépendente de la phase, introduite récemment. Des nouvelles méthodes pour interpréter les décompositions résultantes sont présentées et sont comparées aux méthodes utilisées précédemment qui nécessitent des connaissances du domaine acoustique. Finalement, on analyse la capacité de généralisation des fonctions de bases apprises par rapport à trois paramètres musicaux: l’amplitude, la durée et le type d’instrument. Pour ce faire, on introduit deux algorithmes d’étiquetage des fonctions de bases qui performent mieux que l’approche précédente dans la majorité de nos tests, la tâche d’instrument avec audio monophonique étant la seule exception importante.