QoE-aware Multiple Path Video Transmission for Wireless Multimedia Sensor Networks

Wireless Multimedia Sensor Networks (WMSNs) promise a wide scope of emerging potential applications in both civilian and military areas, which require visual and audio information to enhance the level of collected information. The transmission of multimedia content requires a minimal video quality level from the user’s perspective. However, links in WMSN communi- cations are typically unreliable, as they often experience fluctuations in quality and weak connectivity, and thus, the routing protocol must evaluate the routes by using end-to-end link quality information to increase the packet delivery ratio. Moreover, the use multiple paths together with key video metrics can enhance the video quality level. In this paper, we propose a video-aware multiple path hierarchical routing protocol for efficient multimedia transmission over WMSN, called video-aware MMtransmission. This protocol finds node-disjoint multiple paths, and implements an end-to-end link quality estimation with minimal over- head to score the paths. Thus, our protocol assures multimedia transmission with Quality of Experience (QoE) and energy-efficiency support. The simula- tion results show the benefits of video-aware MMtransmission for disseminating video content by means of energy-efficiency and QoE analysis.

Informative State-Based Video Communication

We study state-based video communication where a client simultaneously informs the server about the presence status of various packets in its buffer. In sender-driven transmission, the client periodically sends to the server a single acknowledgement packet that provides information about all packets that have arrived at the client by the time the acknowledgment is sent. In receiver-driven streaming, the client periodically sends to the server a single request packet that comprises a transmission schedule for sending missing data to the client over a horizon of time. We develop a comprehensive optimization framework that enables computing packet transmission decisions that maximize the end-to-end video quality for the given bandwidth resources, in both prospective scenarios. The core step of the optimization comprises computing the probability that a single packet will be communicated in error as a function of the expected transmission redundancy (or cost) used to communicate the packet. Through comprehensive simulation experiments, we carefully examine the performance advances that our framework enables relative to state-of-the-art scheduling systems that employ regular acknowledgement or request packets. Consistent gains in video quality of up to 2B are demonstrated across a variety of content types. We show that there is a direct analogy between the error-cost efficiency of streaming a single packet and the overall rate-distortion performance of streaming the whole content. In the case of sender-driven transmission, we develop an effective modeling approach that accurately characterizes the end-to-end performance as a function of the packet loss rate on the backward channel and the source encoding characteristics.

Cross-Layer Optimizations for Multimedia Distribution over Wireless Multimedia Sensor Networks and Flying Ad-Hoc Networks with Quality of Experience Support

The proliferation of multimedia content and the demand for new audio or video services have fostered the development of a new era based on multimedia information, which allowed the evolution of Wireless Multimedia Sensor Networks (WMSNs) and also Flying Ad-Hoc Networks (FANETs). In this way, live multimedia services require real-time video transmissions with a low frame loss rate, tolerable end-to-end delay, and jitter to support video dissemination with Quality of Experience (QoE) support. Hence, a key principle in a QoE-aware approach is the transmission of high priority frames (protect them) with a minimum packet loss ratio, as well as network overhead. Moreover, multimedia content must be transmitted from a given source to the destination via intermediate nodes with high reliability in a large scale scenario. The routing service must cope with dynamic topologies caused by node failure or mobility, as well as wireless channel changes, in order to continue to operate despite dynamic topologies during multimedia transmission. Finally, understanding user satisfaction on watching a video sequence is becoming a key requirement for delivery of multimedia content with QoE support. With this goal in mind, solutions involving multimedia transmissions must take into account the video characteristics to improve video quality delivery. The main research contributions of this thesis are driven by the research question how to provide multimedia distribution with high energy-efficiency, reliability, robustness, scalability, and QoE support over wireless ad hoc networks. The thesis addresses several problem domains with contributions on different layers of the communication stack. At the application layer, we introduce a QoE-aware packet redundancy mechanism to reduce the impact of the unreliable and lossy nature of wireless environment to disseminate live multimedia content. At the network layer, we introduce two routing protocols, namely video-aware Multi-hop and multi-path hierarchical routing protocol for Efficient VIdeo transmission for static WMSN scenarios (MEVI), and cross-layer link quality and geographical-aware beaconless OR protocol for multimedia FANET scenarios (XLinGO). Both protocols enable multimedia dissemination with energy-efficiency, reliability and QoE support. This is achieved by combining multiple cross-layer metrics for routing decision in order to establish reliable routes.

High-Definition Video Distribution in 802.11g Home Wireless Networks

Multimedia distribution through wireless networks in the home environment presents a number of advantages which have fueled the interest of industry in recent years, such as simple connectivity and data delivery to a variety of devices. Together with High-Definition (HD) contents, multimedia wireless networks have been proposed for several applications, such as IPTV and Digital TV distribution for multiple devices in the home environment. For these scenarios, we propose a multicast distribution system for High-Definition video over 802.11 wireless networks based on rate-limited packet retransmission. We develop a limited rate ARQ system that retransmits packets according to the importance of their content (prioritization scheme) and according to their delay limitations (delay control). The performance of our proposed ARQ system is evaluated and compared with a similarly rate-limited ARQ algorithm. The results show a higher packet recovery rate and improvements in video quality for our proposed system.

Energy Management and Optimization for Video Decoders based on Functional-oriented Reconfiguration

In recent years, the increasing sophistication of embedded multimedia systems and wireless communication technologies has promoted a widespread utilization of video streaming applications. It has been reported in 2013 that youngsters, aged between 13 and 24, spend around 16.7 hours a week watching online video through social media, business websites, and video streaming sites. Video applications have already been blended into people daily life. Traditionally, video streaming research has focused on performance improvement, namely throughput increase and response time reduction. However, most mobile devices are battery-powered, a technology that grows at a much slower pace than either multimedia or hardware developments. Since battery developments cannot satisfy expanding power demand of mobile devices, research interests on video applications technology has attracted more attention to achieve energy-efficient designs. How to efficiently use the limited battery energy budget becomes a major research challenge. In addition, next generation video standards impel to diversification and personalization. Therefore, it is desirable to have mechanisms to implement energy optimizations with greater flexibility and scalability. In this context, the main goal of this dissertation is to find an energy management and optimization mechanism to reduce the energy consumption of video decoders based on the idea of functional-oriented reconfiguration. System battery life is prolonged as the result of a trade-off between energy consumption and video quality. Functional-oriented reconfiguration takes advantage of the similarities among standards to build video decoders reconnecting existing functional units. If a feedback channel from the decoder to the encoder is available, the former can signal the latter changes in either the encoding parameters or the encoding algorithms for energy-saving adaption. The proposed energy optimization and management mechanism is carried out at the decoder end. This mechanism consists of an energy-aware manager, implemented as an additional block of the reconfiguration engine, an energy estimator, integrated into the decoder, and, if available, a feedback channel connected to the encoder end. The energy-aware manager checks the battery level, selects the new decoder description and signals to build a new decoder to the reconfiguration engine. It is worth noting that the analysis of the energy consumption is fundamental for the success of the energy management and optimization mechanism. In this thesis, an energy estimation method driven by platform event monitoring is proposed. In addition, an event filter is suggested to automate the selection of the most appropriate events that affect the energy consumption. At last, a detailed study on the influence of the training data on the model accuracy is presented. The modeling methodology of the energy estimator has been evaluated on different underlying platforms, single-core and multi-core, with different characteristics of workload. All the results show a good accuracy and low on-line computation overhead. The required modifications on the reconfiguration engine to implement the energy-aware manager have been assessed under different scenarios. The results indicate a possibility to lengthen the battery lifetime of the system in two different use-cases.

Guidelines for filming digital camera video clips for the assessment of gait and movement disorders by teleneurology

Digital still cameras capable of filming short video clips are readily available, but the quality of these recordings for telemedicine has not been reported. We performed a blinded study using four commonly available digital cameras. A simulated patient with a hemiplegic gait pattern was filmed by the same videographer in an identical, brightly lit indoor setting. Six neurologists viewed the blinded video clips on their PC and comparisons were made between cameras, between video clips recorded with and without a tripod, and between video clips filmed on high- or low-quality settings. Use of a tripod had a smaller effect than expected, while images taken on a high-quality setting were strongly preferred to those taken on a low-quality setting. Although there was some variability in video quality between selected cameras, all were of sufficient quality to identify physical signs such as gait and tremor. Adequate-quality video clips of movement disorders can be produced with low-cost cameras and transmitted by email for teleneurology purposes.

H264 data partitioned video streaming

Motivated by the increasing demand and challenges of video streaming in this thesis, we investigate methods by which the quality of the video can be improved. We utilise overlay networks that have been created by implemented relay nodes to produce path diversity, and show through analytical and simulation models for which environments path diversity can improve the packet loss probability. We take the simulation and analytical models further by implementing a real overlay network on top of Planetlab, and show that when the network conditions remain constant the video quality received by the client can be improved. In addition, we show that in the environments where path diversity improves the video quality forward error correction can be used to further enhance the quality. We then investigate the effect of IEEE 802.11e Wireless LAN standard with quality of service enabled on the video quality received by a wireless client. We find that assigning all the video to a single class outperforms a cross class assignment scheme proposed by other researchers. The issue of virtual contention at the access point is also examined. We increase the intelligence of our relay nodes and enable them to cache video, in order to maximise the usefulness of these caches. For this purpose, we introduce a measure, called the PSNR profit, and present an optimal caching method for achieving the maximum PSNR profit at the relay nodes where partitioned video contents are stored and provide an enhanced quality for the client. We also show that the optimised cache the degradation in the video quality received by the client becomes more graceful than the non-optimised system when the network experiences packet loss or is congested.

Digital surveillance based on video CODEC System-on-a-Chip (SoC) platforms

Today, most conventional surveillance networks are based on analog system, which has a lot of constraints like manpower and high-bandwidth requirements. It becomes the barrier for today's surveillance network development. This dissertation describes a digital surveillance network architecture based on the H.264 coding/decoding (CODEC) System-on-a-Chip (SoC) platform. The proposed digital surveillance network architecture includes three major layers: software layer, hardware layer, and the network layer. The following outlines the contributions to the proposed digital surveillance network architecture. (1) We implement an object recognition system and an object categorization system on the software layer by applying several Digital Image Processing (DIP) algorithms. (2) For better compression ratio and higher video quality transfer, we implement two new modules on the hardware layer of the H.264 CODEC core, i.e., the background elimination module and the Directional Discrete Cosine Transform (DDCT) module. (3) Furthermore, we introduce a Digital Signal Processor (DSP) sub-system on the main bus of H.264 SoC platforms as the major hardware support system for our software architecture. Thus we combine the software and hardware platforms to be an intelligent surveillance node. Lab results show that the proposed surveillance node can dramatically save the network resources like bandwidth and storage capacity.

Robust H.264/AVC video transmission using data partitioning and unequal loss protection

In this work, we present an adaptive unequal loss protection (ULP) scheme for H264/AVC video transmission over lossy networks. This scheme combines erasure coding, H.264/AVC error resilience techniques and importance measures in video coding. The unequal importance of the video packets is identified in the group of pictures (GOP) and the H.264/AVC data partitioning levels. The presented method can adaptively assign unequal amount of forward error correction (FEC) parity across the video packets according to the network conditions, such as the available network bandwidth, packet loss rate and average packet burst loss length. A near optimal algorithm is developed to deal with the FEC assignment for optimization. The simulation results show that our scheme can effectively utilize network resources such as bandwidth, while improving the quality of the video transmission. In addition, the proposed ULP strategy ensures graceful degradation of the received video quality as the packet loss rate increases. © 2010 IEEE.

IPTV-palvelun pilotointi

Diplomityössä kehitettiin teleoperaattorille IPTV-palvelu (Internet Protocol Television) tietyillä rajatuilla ominaisuuksilla, kehitettiin IPTV-palvelulle uusi käyttöliittymä ja pilotoitiin uutta palvelua. Pilotoinnin tarkoituksena oli saada kokemusta uudentyyppisten tv-palvelujen käyttöönotosta. Pilotoinnin lopuksi koottiin pilottikäyttäjiltä saatu palaute ja analysoitiin palvelua sen perusteella. Työn taustatieto-osuudessa määritellään IPTV ja siihen tyypillisimmin kuuluvat palvelut. Lisäksi selvitetään, millaisia ovat IPTV:ssä käytetyt tekniikat ja verkkoarkkitehtuuri. Taustatiedoissa esitellään myös palvelun käyttäjän kokeman laadun parametrit ja kerrosmalli. Samalla tuodaan esiin menetelmät erityisesti kuvanlaadun ja käytettävyyden parantamiseksi. Opittavuus ja helppokäyttöisyys sekä viihteellisyys ja miellyttävyys on tunnistettu tärkeimmiksi käytettävyyden osa-alueiksi elinkaarensa alkupäässä olevissa IPTV-palveluissa. Diplomityön toteutusosuudessa esitetään työn lähtökohtana ollut vaatimusmäärittely ja sen vaikutus työn rajaukseen. Tämän jälkeen esitellään kehitetyn IPTV-palvelun ominaisuudet, joista tärkeimpinä tuodaan esiin tallennus- ja ajansiirtotoiminnot. Toteutusosuudessa selvitetään myös teleoperaattorin IPTV-verkon rakenne ja toiminta. Seuraavaksi käydään läpi alkuperäisestä käyttöliittymästä tehty analyysi ja analyysin perusteella toteutettu uusi käyttöliittymä, jossa merkittävimmät parannukset on tehty navigaatiorakenteeseen. Lopuksi selvitetään pilotoinnista kerätyt tulokset. Tuloksien perusteella voidaan nähdä, että ajansiirtotoiminnot riittävät takaamaan palvelun omaksumisen, mutta uudet toiminnot vaativat riittävän opastuksen. Tulokset osoittavat myös, että palvelun käytettävyys uudella käyttöliittymällä on tavoitetasolla, vaikkakin toimintojen viiveet heikentävät käyttökokemuksen miellyttävyyttä.

Infotainment across access devices: the perceptual impact of multimedia QoS

The introduction of multimedia on pervasive and mobile communication devices raises a number of perceptual quality issues. However, limited work has been done examining the 3-way interaction between use of equipment, user perceptual quality and quality of service. Our work measures user perceptual quality with the quality of perception (QoP) metrics which comprises levels of informational transfer (objective) and user satisfaction (subjective) when users are presented with multimedia video clips at three different frame rates, using four different display devices. Finally, our results will show that variation in frame-rate does not impact a user’s level of information assimilation (IA), however, does impact a users’ perception of multimedia video ‘quality’.

Arcabouço para aprovisionamento de QoS e QoE em redes sem fio heterogêneas WiMAX/Wi-Fi com garantia de equidade entre vazões

Recentemente a sociedade está vivendo em meio a um aglomerado de dispositivos móveis com múltiplas interfaces sem fio que permitem que clientes móveis domésticos e corporativos solucionem problemas rotineiros no menor espaço de tempo e em diversas localizações. Por outro lado, a disponibilidade de diversas tecnologias de acesso sem fio como: Wi-Fi (Wireless Fidelity), WiMAX (Worldwide Interoperability for Microwave Access) e LTE (Long Term Evolution), proporcionam um cenário heterogêneo com diversas oportunidades de conectividade para o usuário de dispositivos com múltiplas interfaces. Alguns dos desafios atuais no contexto da concepção da então denominada NGN (Next Generation Networks) são o desenvolvimento de arcabouços e mecanismos que viabilizem tanto o suporte adequado à Qualidade de Serviço (QoS – Quality of Service) e à Qualidade de Experiência (QoE – Quality of Experience) para aplicações multimídia, quanto a garantia de equidade entre vazões de diferentes aplicações em ambiente heterogêneo. Esta dissertação propõe um arcabouço para o aprovisionamento de QoS e QoE em redes sem fio heterogêneas formada por redes WiMAX e Wi-Fi. Especificamente, nossa solução provê mapeamento estático de QoS entre as classes de serviços WiMAX e categorias de acesso Wi-Fi. Além disso, nossa proposta também combina funcionalidades de equidade entre vazões com a solução de mapeamento, a fim de alcançar um bom compromisso tanto para o operador da rede, quanto para o usuário, através de uma nova metodologia de coleta e calculo de vazão agregada e algoritmo de decisão de handover vertical. A avaliação quantitativa da proposta foi realizada através de modelos de simulação no ns-2 (Network Simulator). Os resultados de desempenho demonstram a eficiência da arquitetura em termos de métricas de QoS (vazão, vazão média e atraso) e QoE (PSRN - Peak Signal to Noise Ratio, SSIM - Structural Similarity Index e VQM - Video Quality Metric).

Um mecanismo para realçar a conectividade de roteamento geográfico na transmissão multimídia em VANTs

As redes de Veículos Aéreos Não Tripulados (VANTs) têm potencializado o uso de aplicações multimídia devido ao seu elevado grau de mobilidade e versatilidade. Esse crescimento ocorre pela facilidade de acesso e utilização dos VANTs quadricópteros em diversas aplicações civis. As aplicações civis têm como característica principal o uso de dados multimídia, como vídeos, por oferecerem uma perspectiva visual mais detalhada do ambiente. A transmissão desse tipo de dado nas redes VANTs por meio de protocolos geográ cos melhora a taxa de entrega de dados. Entretanto, ela ainda não é su ciente para prover qualidade de experiência (QoE). Isso ocorre devido ao elevado grau de mobilidade dos quadricópteros que ocasiona quebras de enlace durante a transmissão multimídia. Por consequência, ela prejudica a conectividade e induz a altas perdas de pacotes e atrasos. Então, este trabalho propõe um mecanismo, chamado RCRV, com base em técnicas de predição de mobilidade em termos de posicionamento e da estimativa do tempo de enlace para realçar a tomada de decisão de roteamento em protocolos geográ cos. Além disso, ele utiliza critérios que consideram os comportamentos de mobilidade dos VANTs. Assim, o RCRV possibilita o prolongamento da conectividade nas redes VANTs. O RCRV é um mecanismo que complementa as estratégias de roteamento dos protocolos geográ cos. Desse modo, adicionou-se o RCRV ao protocolo GPSR a m de avaliar os ganhos e benefícios do mecanismo em um protocolo de roteamento geográ co. Simulações foram realizadas com diferentes velocidades e números de VANTs. Os resultados mostraram que o RCRV aumenta a conectividade da transmissão, melhorando a entrega do conteúdo multimídia e a qualidade do vídeo observado pelo usuário.

Constructing a user experience user experience-based mobile learning environment

Mobile learning, in the past defined as learning with mobile devices, now refers to any type of learning-on-the-go or learning that takes advantage of mobile technologies. This new definition shifted its focus from the mobility of technology to the mobility of the learner (O'Malley and Stanton 2002; Sharples, Arnedillo-Sanchez et al. 2009). Placing emphasis on the mobile learner’s perspective requires studying “how the mobility of learners augmented by personal and public technology can contribute to the process of gaining new knowledge, skills, and experience” (Sharples, Arnedillo-Sanchez et al. 2009). The demands of an increasingly knowledge based society and the advances in mobile phone technology are combining to spur the growth of mobile learning. Around the world, mobile learning is predicted to be the future of online learning, and is slowly entering the mainstream education. However, for mobile learning to attain its full potential, it is essential to develop more advanced technologies that are tailored to the needs of this new learning environment. A research field that allows putting the development of such technologies onto a solid basis is user experience design, which addresses how to improve usability and therefore user acceptance of a system. Although there is no consensus definition of user experience, simply stated it focuses on how a person feels about using a product, system or service. It is generally agreed that user experience adds subjective attributes and social aspects to a space that has previously concerned itself mainly with ease-of-use. In addition, it can include users’ perceptions of usability and system efficiency. Recent advances in mobile and ubiquitous computing technologies further underline the importance of human-computer interaction and user experience (feelings, motivations, and values) with a system. Today, there are plenty of reports on the limitations of mobile technologies for learning (e.g., small screen size, slow connection), but there is a lack of research on user experience with mobile technologies. This dissertation will fill in this gap by a new approach in building a user experience-based mobile learning environment. The optimized user experience we suggest integrates three priorities, namely a) content, by improving the quality of delivered learning materials, b) the teaching and learning process, by enabling live and synchronous learning, and c) the learners themselves, by enabling a timely detection of their emotional state during mobile learning. In detail, the contributions of this thesis are as follows: • A video codec optimized for screencast videos which achieves an unprecedented compression rate while maintaining a very high video quality, and a novel UI layout for video lectures, which together enable truly mobile access to live lectures. • A new approach in HTTP-based multimedia delivery that exploits the characteristics of live lectures in a mobile context and enables a significantly improved user experience for mobile live lectures. • A non-invasive affective learning model based on multi-modal emotion detection with very high recognition rates, which enables real-time emotion detection and subsequent adaption of the learning environment on mobile devices. The technology resulting from the research presented in this thesis is in daily use at the School of Continuing Education of Shanghai Jiaotong University (SOCE), a blended-learning institution with 35.000 students.

An IEEE 802.11 energy efficient mechanism for continuous media applications

The widespread deployment of wireless mobile communications enables an almost permanent usage of portable devices, which imposes high demands on the battery of these devices. Indeed, battery lifetime is becoming one the most critical factors on the end-users satisfaction when using wireless communications. In this work, the optimized power save algorithm for continuous media applications (OPAMA) is proposed, aiming at enhancing the energy efficiency on end-users devices. By combining the application specific requirements with data aggregation techniques, {OPAMA} improves the standard {IEEE} 802.11 legacy Power Save Mode (PSM) performance. The algorithm uses the feedback on the end-user expected quality to establish a proper tradeoff between energy consumption and application performance. {OPAMA} was assessed in the OMNeT++ simulator, using real traces of variable bitrate video streaming applications, and in a real testbed employing a novel methodology intended to perform an accurate evaluation concerning video Quality of Experience (QoE) perceived by the end-users. The results revealed the {OPAMA} capability to enhance energy efficiency without degrading the end-user observed QoE, achieving savings up to 44 when compared with the {IEEE} 802.11 legacy PSM.