Content Classification of Multimedia Documents using Partitions of Low-Level Features

Audio-visual documents obtained from German TV news are classified according to the IPTC topic categorization scheme. To this end usual text classification techniques are adapted to speech, video, and non-speech audio. For each of the three modalities word analogues are generated: sequences of syllables for speech, “video words” based on low level color features (color moments, color correlogram and color wavelet), and “audio words” based on low-level spectral features (spectral envelope and spectral flatness) for non-speech audio. Such audio and video words provide a means to represent the different modalities in a uniform way. The frequencies of the word analogues represent audio-visual documents: the standard bag-of-words approach. Support vector machines are used for supervised classification in a 1 vs. n setting. Classification based on speech outperforms all other single modalities. Combining speech with non-speech audio improves classification. Classification is further improved by supplementing speech and non-speech audio with video words. Optimal F-scores range between 62% and 94% corresponding to 50% - 84% above chance. The optimal combination of modalities depends on the category to be recognized. The construction of audio and video words from low-level features provide a good basis for the integration of speech, non-speech audio and video.

Cross-Layer Optimizations for Multimedia Distribution over Wireless Multimedia Sensor Networks and Flying Ad-Hoc Networks with Quality of Experience Support

The proliferation of multimedia content and the demand for new audio or video services have fostered the development of a new era based on multimedia information, which allowed the evolution of Wireless Multimedia Sensor Networks (WMSNs) and also Flying Ad-Hoc Networks (FANETs). In this way, live multimedia services require real-time video transmissions with a low frame loss rate, tolerable end-to-end delay, and jitter to support video dissemination with Quality of Experience (QoE) support. Hence, a key principle in a QoE-aware approach is the transmission of high priority frames (protect them) with a minimum packet loss ratio, as well as network overhead. Moreover, multimedia content must be transmitted from a given source to the destination via intermediate nodes with high reliability in a large scale scenario. The routing service must cope with dynamic topologies caused by node failure or mobility, as well as wireless channel changes, in order to continue to operate despite dynamic topologies during multimedia transmission. Finally, understanding user satisfaction on watching a video sequence is becoming a key requirement for delivery of multimedia content with QoE support. With this goal in mind, solutions involving multimedia transmissions must take into account the video characteristics to improve video quality delivery. The main research contributions of this thesis are driven by the research question how to provide multimedia distribution with high energy-efficiency, reliability, robustness, scalability, and QoE support over wireless ad hoc networks. The thesis addresses several problem domains with contributions on different layers of the communication stack. At the application layer, we introduce a QoE-aware packet redundancy mechanism to reduce the impact of the unreliable and lossy nature of wireless environment to disseminate live multimedia content. At the network layer, we introduce two routing protocols, namely video-aware Multi-hop and multi-path hierarchical routing protocol for Efficient VIdeo transmission for static WMSN scenarios (MEVI), and cross-layer link quality and geographical-aware beaconless OR protocol for multimedia FANET scenarios (XLinGO). Both protocols enable multimedia dissemination with energy-efficiency, reliability and QoE support. This is achieved by combining multiple cross-layer metrics for routing decision in order to establish reliable routes.

WORKLOAD AND PERFORMANCE FACTORS ASSOCIATED WITH MULTIMEDIA JOB AIDS FOR COMMUNITY HEALTH WORKERS

This dissertation focuses on factors of multimedia job aids that modify workload, protocol adherence and clinical errors in community health workers. Literature shows that community health workers performance is not acceptable even with support of paper job aids. There are cognitive theories that try to explain reasons why the performance of community health workers is poor regardless of the access to paper based-job aid. Based on cognitive science and multimedia design theories an intervention was designed to compare alternative representations for the information contained on paper job aids and the capability of this new designed job aids to enhance community health workers performance. The dissertation is divided in 5 main parts: 1. identification and description of the problem, 2. a methodological approach to create and evaluate an intervention, 3. Presentation of results of the intervention evaluation, 4. Discussion of findings and 5. Conclusions

Multipedia: Enriching DBpedia with Multimedia Information

Enriching knowledge bases with multimedia information makes it possible to complement textual descriptions with visual and audio information. Such complementary information can help users to understand the meaning of assertions, and in general improve the user experience with the knowledge base. In this paper we address the problem of how to enrich ontology instances with candidate images retrieved from existing Web search engines. DBpedia has evolved into a major hub in the Linked Data cloud, interconnecting millions of entities organized under a consistent ontology. Our approach taps into the Wikipedia corpus to gather context information for DBpedia instances and takes advantage of image tagging information when this is available to calculate semantic relatedness between instances and candidate images. We performed experiments with focus on the particularly challenging problem of highly ambiguous names. Both methods presented in this work outperformed the baseline. Our best method leveraged context words from Wikipedia, tags from Flickr and type information from DBpedia to achieve an average precision of 80%.

Magic cards: a new augmented reality approach

Augmented reality (AR) commonly uses markers for detection and tracking. Such multimedia applications associate each marker with a virtual 3D model stored in the memory of the camera-equipped device running the application. Application users are limited in their interactions, which require knowing how to design and program 3D objects. This generally prevents them from developing their own entertainment AR applications. The Magic Cards application solves this problem by offering an easy way to create and manage an unlimited number of virtual objects that are encoded on special markers.

A MAUT approach for reusing ontologies

Knowledge resource reuse has become a popular approach within the ontology engineering field, mainly because it can speed up the ontology development process, saving time and money and promoting the application of good practices. The NeOn Methodology provides guidelines for reuse. These guidelines include the selection of the most appropriate knowledge resources for reuse in ontology development. This is a complex decision-making problem where different conflicting objectives, like the reuse cost, understandability, integration workload and reliability, have to be taken into account simultaneously. GMAA is a PC-based decision support system based on an additive multi-attribute utility model that is intended to allay the operational difficulties involved in the Decision Analysis methodology. The paper illustrates how it can be applied to select multimedia ontologies for reuse to develop a new ontology in the multimedia domain. It also demonstrates that the sensitivity analyses provided by GMAA are useful tools for making a final recommendation.

Aprendizaje multimedia en inglés: Análisis de circuitos

Este proyecto tiene como objetivo facilitar el aprendizaje multimedia en inglés de los conceptos básicos relacionados con el análisis de circuitos para los alumnos de grado de la nuestra escuela, en especial para los de nuevo ingreso. El aprendizaje será realizado directamente en inglés, cumpliendo así el doble objetivo: por un lado el de abordar unos conceptos nuevos para el alumno de ingeniería y, además, el poder hacerlo en inglés, lo que le servirá para llegar a mejorar su nivel de competencia en la lengua extranjera, o al menos mantenerlo. El proyecto realizado tiene varios apartados. En primer lugar, un apartado didáctico en el que se exponen los conceptos básicos sobre el análisis de circuitos que se quieren explicar, siempre orientados al aprendizaje de inglés. Para abordar estos nuevos temas de un modo didáctico y que incite al autoaprendizaje, se recurrirá a la tecnología y por medio de vídeos e imágenes y actividades didácticas multimedia se podrá afrontar la asignatura con facilidad e interés para captar los conceptos en una lengua extranjera. Se han diseñado actividades para la práctica de: la audición, con dictado y asociación de sonidos con palabras; de la comprensión lectora y adquisición de vocabulario, con ejercicios de rellenar huecos y emparejar definiciones. Las actividades se han adaptado a la plataforma Moodle para obtener la retroalimentación de los alumnos, si se desea. El apartado de aprendizaje se complementará con un glosario específico y alfabéticamente ordenado donde también dispondremos de la transcripción fonética de cada una de las palabras incluidas en el mismo. Para llegar a conseguir el objetivo didáctico se ha diseñado un sitio web capaz de albergar todo el contenido anterior. Abstract The main objective of this Project has been to facilitate a multimedia resource for learning in English the basic concepts related to circuit analysis. The final product is directly addressed to the undergraduate students at a Telecommunications School, the EUITT from Universidad Politécnica de Madrid. Learning technical notions directly in English serves the students to reach the double purpose of not only acquiring a number of new concepts but also improving their proficiency in the foreign language or keeping up with their level of English through their studies. The project has several sections. There is a didactic section which explains the basic concepts using videos and images in order to approach the subject with ease and interest. This part is supplemented with a glossary in alphabetic order provided with the phonetics of the English words and some multimedia exercises for practicing different skills mainly listening, reading, and using technical vocabulary in English. All these activities have been adapted to the platform Moodle so that the students results can be assessed, if convenient. On the other hand, the practical implementation of the Project has consisted of designing a website capable of including all of the points mentioned above.

Using multimedia and peer assessment to promote collaborative e-learning

Collaborative e-learning is increasingly appealing as a pedagogical approach that can positively affect student learning. We propose a didactical model that integrates multimedia with collaborative tools and peer assessment to foster collaborative e-learning. In this paper, we explain it and present the results of its application to the “International Seminars on Materials Science” online course. The proposed didactical model consists of five educational activities. In the first three, students review the multimedia resources proposed by the teacher in collaboration with their classmates. Then, in the last two activities, they create their own multimedia resources and assess those created by their classmates. These activities foster communication and collaboration among students and their ability to use and create multimedia resources. Our purpose is to encourage the creativity, motivation, and dynamism of the learning process for both teachers and students.

Validation of a novel approach to subjective quality evaluation of conventional and 3D broadcasted video services

Transmission errors are the main cause of degradation of the quality of real broadcasted video services. Therefore, knowing their impact on the quality of experience of the end users is a crucial issue. For instance, it would help to improve the performance of the distribution systems, and to develop monitoring tools to automatically estimate the quality perceived by the end users. In this paper we validate a subjective evaluation approach specifically designed to obtain meaningful results of the effects of degradations caused by transmission errors. This methodology has been already used in our previous works with monoscopic and stereoscopic videos. The validation is done by comparing the subjective ratings obtained for typical transmission errors with the proposed methodology and with the standard method Absolute Category Rating. The results show that the proposed approach could provide more representative evaluations of the quality of experience perceived by end users of conventional and 3D broadcasted video services.

Bounded non-deterministic planning for multimedia adaptation

This paper proposes a novel combination of artificial intelligence planning and other techniques for improving decision-making in the context of multi-step multimedia content adaptation. In particular, it describes a method that allows decision-making (selecting the adaptation to perform) in situations where third-party pluggable multimedia conversion modules are involved and the multimedia adaptation planner does not know their exact adaptation capabilities. In this approach, the multimedia adaptation planner module is only responsible for a part of the required decisions; the pluggable modules make additional decisions based on different criteria. We demonstrate that partial decision-making is not only attainable, but also introduces advantages with respect to a system in which these conversion modules are not capable of providing additional decisions. This means that transferring decisions from the multi-step multimedia adaptation planner to the pluggable conversion modules increases the flexibility of the adaptation. Moreover, by allowing conversion modules to be only partially described, the range of problems that these modules can address increases, while significantly decreasing both the description length of the adaptation capabilities and the planning decision time. Finally, we specify the conditions under which knowing the partial adaptation capabilities of a set of conversion modules will be enough to compute a proper adaptation plan.

Experiences with free and open courses using on-line multimedia resources

This paper analyzes the learning experiences and opinions obtained from a group of undergraduate students in their interaction with several on-line multimedia resources included in a free on-line course about Computer Networks. These new educational resources employed are based on the Web2.0 approach such as blogs, videos and virtual labs which have been added in a web-site for distance self-learning.

A grammatical approach to the modeling of an autonomous robot

Virtual Worlds Generator is a grammatical model that is proposed to define virtual worlds. It integrates the diversity of sensors and interaction devices, multimodality and a virtual simulation system. Its grammar allows the definition and abstraction in symbols strings of the scenes of the virtual world, independently of the hardware that is used to represent the world or to interact with it. A case study is presented to explain how to use the proposed model to formalize a robot navigation system with multimodal perception and a hybrid control scheme of the robot. The result is an instance of the model grammar that implements the robotic system and is independent of the sensing devices used for perception and interaction. As a conclusion the Virtual Worlds Generator adds value in the simulation of virtual worlds since the definition can be done formally and independently of the peculiarities of the supporting devices.

Towards a reverse engineering approach for guiding user in applying data mining

Comunicación presentada en las XVI Jornadas de Ingeniería del Software y Bases de Datos, JISBD 2011, A Coruña, 5-7 septiembre 2011.

An Interdisciplinary Practical for Multimedia Engineering Students

Interdisciplinary projects in the industry typically require collaboration between professionals from various fields. However, this relationship is not generally addressed in the training offered by university programs, which often ignore this interdisciplinary approach. This paper offers an example of interdisciplinary interaction through joint laboratory activities in the curricula of two very different degree programs, i.e., Multimedia Engineering and Teacher Training in Primary Education. The programs' students formed an interdisciplinary team of multimedia engineers and trainee teachers to develop a Web product for children's cognitive development. The complexity of the task required students to engage in close and strong interdisciplinary cooperation and communication; in turn, they benefited from the synergy offered by collaborative work. The results of this paper, presented from the perspective of the multimedia engineering students, demonstrate a significant increase in their academic performance compared to the control group. This paper shows that university studies can incorporate an interdisciplinary perspective to engineering education without the need to introduce a specific course on the topic, thus avoiding further demands on the curriculum schedule.

File Format for Storage of Multimedia Information

AMS Subj. Classification: 68U05, 68P30