Content Classification of Multimedia Documents using Partitions of Low-Level Features

Audio-visual documents obtained from German TV news are classified according to the IPTC topic categorization scheme. To this end usual text classification techniques are adapted to speech, video, and non-speech audio. For each of the three modalities word analogues are generated: sequences of syllables for speech, “video words” based on low level color features (color moments, color correlogram and color wavelet), and “audio words” based on low-level spectral features (spectral envelope and spectral flatness) for non-speech audio. Such audio and video words provide a means to represent the different modalities in a uniform way. The frequencies of the word analogues represent audio-visual documents: the standard bag-of-words approach. Support vector machines are used for supervised classification in a 1 vs. n setting. Classification based on speech outperforms all other single modalities. Combining speech with non-speech audio improves classification. Classification is further improved by supplementing speech and non-speech audio with video words. Optimal F-scores range between 62% and 94% corresponding to 50% - 84% above chance. The optimal combination of modalities depends on the category to be recognized. The construction of audio and video words from low-level features provide a good basis for the integration of speech, non-speech audio and video.

Impact Study of Nonverbal Facial Cues on Spontaneous Chatting with Virtual Humans

Non-verbal communication (NVC) is considered to represent more than 90 percent of everyday communication. In virtual world, this important aspect of interaction between virtual humans (VH) is strongly neglected. This paper presents a user-test study to demonstrate the impact of automatically generated graphics-based NVC expression on the dialog quality: first, we wanted to compare impassive and emotion facial expression simulation for impact on the chatting. Second, we wanted to see whether people like chatting within a 3D graphical environment. Our model only proposes facial expressions and head movements induced from spontaneous chatting between VHs. Only subtle facial expressions are being used as nonverbal cues - i.e. related to the emotional model. Motion capture animations related to hand gestures, such as cleaning glasses, were randomly used to make the virtual human lively. After briefly introducing the technical architecture of the 3D-chatting system, we focus on two aspects of chatting through VHs. First, what is the influence of facial expressions that are induced from text dialog? For this purpose, we exploited an emotion engine extracting an emotional content from a text and depicting it into a virtual character developed previously [GAS11]. Second, as our goal was not addressing automatic generation of text, we compared the impact of nonverbal cues in conversation with a chatbot or with a human operator with a wizard of oz approach. Among main results, the within group study -involving 40 subjects- suggests that subtle facial expressions impact significantly not only on the quality of experience but also on dialog understanding.

Constructing a user experience user experience-based mobile learning environment

Mobile learning, in the past defined as learning with mobile devices, now refers to any type of learning-on-the-go or learning that takes advantage of mobile technologies. This new definition shifted its focus from the mobility of technology to the mobility of the learner (O'Malley and Stanton 2002; Sharples, Arnedillo-Sanchez et al. 2009). Placing emphasis on the mobile learner’s perspective requires studying “how the mobility of learners augmented by personal and public technology can contribute to the process of gaining new knowledge, skills, and experience” (Sharples, Arnedillo-Sanchez et al. 2009). The demands of an increasingly knowledge based society and the advances in mobile phone technology are combining to spur the growth of mobile learning. Around the world, mobile learning is predicted to be the future of online learning, and is slowly entering the mainstream education. However, for mobile learning to attain its full potential, it is essential to develop more advanced technologies that are tailored to the needs of this new learning environment. A research field that allows putting the development of such technologies onto a solid basis is user experience design, which addresses how to improve usability and therefore user acceptance of a system. Although there is no consensus definition of user experience, simply stated it focuses on how a person feels about using a product, system or service. It is generally agreed that user experience adds subjective attributes and social aspects to a space that has previously concerned itself mainly with ease-of-use. In addition, it can include users’ perceptions of usability and system efficiency. Recent advances in mobile and ubiquitous computing technologies further underline the importance of human-computer interaction and user experience (feelings, motivations, and values) with a system. Today, there are plenty of reports on the limitations of mobile technologies for learning (e.g., small screen size, slow connection), but there is a lack of research on user experience with mobile technologies. This dissertation will fill in this gap by a new approach in building a user experience-based mobile learning environment. The optimized user experience we suggest integrates three priorities, namely a) content, by improving the quality of delivered learning materials, b) the teaching and learning process, by enabling live and synchronous learning, and c) the learners themselves, by enabling a timely detection of their emotional state during mobile learning. In detail, the contributions of this thesis are as follows: • A video codec optimized for screencast videos which achieves an unprecedented compression rate while maintaining a very high video quality, and a novel UI layout for video lectures, which together enable truly mobile access to live lectures. • A new approach in HTTP-based multimedia delivery that exploits the characteristics of live lectures in a mobile context and enables a significantly improved user experience for mobile live lectures. • A non-invasive affective learning model based on multi-modal emotion detection with very high recognition rates, which enables real-time emotion detection and subsequent adaption of the learning environment on mobile devices. The technology resulting from the research presented in this thesis is in daily use at the School of Continuing Education of Shanghai Jiaotong University (SOCE), a blended-learning institution with 35.000 students.

Promoting research-user collaboration: an assessment of funding schemes: Fifth NCCR North-South Report on Effectiveness

Donors increasingly require that the research they fund be of benefit for society and the environment. To this end, researchers addressing complex and uncertain problems should work together with research users. This is not always easy: researchers are expected to collaborate with non-academic partners, but are not funded for the additional work. Collaborative research projects often cannot tap the full potential of user engagement. Therefore, specific institutional and organisational conditions are necessary that foresee or even foster research–user engagement; funding schemes are one possible solution. Right from the start, the NCCR North-South programme introduced Partnership Actions for Mitigating Syndromes of Global Change (PAMS). This evaluation assesses the types of collaboration supported by PAMS, as well as the value of PAMS as a funding scheme for collaboration. It compares PAMS with similar funding schemes of other universities, research programmes, or projects, and contains recommendations.

I like it because I (‘m) like You – measuring user attitudes towards information on Facebook

Limited in motivation and cognitive ability to process the increasing amount of information on their Newsfeed, users apply heuristic processing to form their attitudes. Rather than extensively analysing the content, they increasingly rely on heuristic cues – such as the amount of comments and likes as well as the level of relationship with the “poster” – to process the incoming information. In the paper we explore what impact these heuristic cues have on the affective and cognitive attitude of users towards the posts on their Newsfeed. We conduct a survey on based on a Facebook application that allows users to evaluate Newsfeed posts in real time. Applying two distinct panel-regression methods we report robust results that indicate that there is a certain relationship primacy effect when users are processing information: only if the level of relationship with the “poster” is low, the impact of comments and likes on the attitude is considered, whereby likes trigger positive, whereas comments – negative evaluations.

Keeping citizens, companies, and organizations informed by Web KnowARR

For the main part, electronic government (or e-government for short) aims to put digital public services at disposal for citizens, companies, and organizations. To that end, in particular, e-government comprises the application of Information and Communications Technology (ICT) to support government operations and provide better governmental services (Fraga, 2002) as possible with traditional means. Accordingly, e-government services go further as traditional governmental services and aim to fundamentally alter the processes in which public services are generated and delivered, after this manner transforming the entire spectrum of relationships of public bodies with its citizens, businesses and other government agencies (Leitner, 2003). To implement this transformation, one of the most important points is to inform the citizen, business, and/or other government agencies faithfully and in an accessible way. This allows all the partaking participants of governmental affairs for a transition from passive information access to active participation (Palvia and Sharma, 2007). In addition, by a corresponding handling of the participants' data, a personalization towards these participants may even be accomplished. For instance, by creating significant user profiles as a kind of participants' tailored knowledge structures, a better-quality governmental service may be provided (i.e., expressed by individualized governmental services). To create such knowledge structures, thus known information (e.g., a social security number) can be enriched by vague information that may be accurate to a certain degree only. Hence, fuzzy knowledge structures can be generated, which help improve governmental-participants relationship. The Web KnowARR framework (Portmann and Thiessen, 2013; Portmann and Pedrycz, 2014; Portmann and Kaltenrieder, 2014), which I introduce in my presentation, allows just all these participants to be automatically informed about changes of Web content regarding a- respective governmental action. The name Web KnowARR thereby stands for a self-acting entity (i.e. instantiated form the conceptual framework) that knows or apprehends the Web. In this talk, the frameworks respective three main components from artificial intelligence research (i.e. knowledge aggregation, representation, and reasoning), as well as its specific use in electronic government will be briefly introduced and discussed.

Detailed analysis of the organic carbon content in sediment cores M0002 and M0004

During IODP Expedition 302 (Arctic Coring Expedition-ACEX), the first scientific drilling campaign in the permantly ice-covered central Arctic Ocean, a 430 m thick sequence of upper Cretaceaous to Quaternary sediments has been drilled. The lower half of this sequence is composed of organic-carbon-rich (black shale-type) sediments with total organic carbon contents of about 1-14%. Significant amounts of the organic matter preserved in these sediments is of algae-type origin and accumulated under anoxic/euxinic conditions. Here, for the first time detailed data on the source-rock potential of these black shales are presented, indicating that most of the Eocene sediments have a (fair to) good source-rock potential, prone to generate a gas/oil mixture. The source-rock potential of the Campanian and upper Paleocene sediments, on the other hand, is rather low. The presence of oil or gas already generated in situ, however, can be ruled out due to the immaturity of the ACEX sediments.

Carbonate content in lower Cretaceous sediments of ODP Hole 123-765C

Sediments recovered during Ocean Drilling Program (ODP) Leg 123 from the Argo Abyssal Plain (AAP) consist largely of turbidites derived from the adjacent Australian continental margin. The oldest abundant turbidites are Valanginian-Aptian in age and have a mixed (smarl) composition; they contain subequal amounts of calcareous and siliceous biogenic components, as well as clay and lesser quartz. Most are thin-bedded, fine sand- to mud-sized, and best described by Stow and Piper's model (1984) for fine-grained biogenic turbidites. Thicker (to 3 m), coarser-grained (medium-to-coarse sand-sized) turbidites fit Bouma's model (1962) for sandy turbidites; these generally are base-cut-out (BCDE, BDE) sequences, with B-division parallel lamination as the dominant structure. Parallel laminae most commonly concentrate quartz and/or calcispheres vs. lithic clasts or clay, but distinctive millimeter- to centimeter-thick, radiolarian-rich laminae occur in both fine- and coarse-grained Valanginian-Hauterivian turbidites. AAP turbidites were derived from relatively deep parts of the continental margin (outer shelf, slope, or rise) that lay below the photic zone, but above the calcite compensation depth (CCD). Biogenic components are largely pelagic (calcispheres, foraminifers, radiolarians, nannofossils); lesser benthic foraminifers are characteristic of deep-water (abyssal to bathyal) environments. Abundant nonbiogenic components are mostly clay and clay clasts; smectite is the dominant clay species, and indicates a volcanogenic provenance, most likely the Triassic-Jurassic volcanic suite exposed along the northern Exmouth Plateau. Lower Cretaceous smarl turbidites were generated during eustatic lowstands and may have reached the abyssal plain via Swan Canyon, a submarine canyon thought to have formed during the Late Jurassic. In contrast to younger AAP turbidites, however, Lower Cretaceous turbidites are relatively fine-grained and do not contain notably older reworked fossils. Early in its history, the northwest Australian margin provided mainly contemporaneous slope sediment to the AAP; marginal basins adjacent to the continent trapped most terrigenous detritus, and pronounced canyon incisement did not occur until Late Cretaceous and, especially, Cenozoic time.

Major element oxides, trace element content and isotopic ratios of dolerites from the Schirmacher Oasis

New results on the petrochemistry and geochemistry of dolerites from the Schirmacher Oasis shed light on the development of the Karoo-Maud plume in Antarctica. The basalts and dolerites are petrologically identical to the rocks of western Dronning Maud Land (DML), which were previously studied and interpreted as a manifestation of the Karoo-Maud plume in Antarctica. The spatial distribution of the dikes suggests eastward spreading of the plume material, up to the Schirmacher Oasis for at least 10 Ma. The geochemical characteristics of magmas from the Schirmacher Oasis reflect the influence of crustal contamination, which accompanied both the ascent and spreading of the plume. The magmas of the initial stage of plume activity (western DML) appeared to be the most contaminated in crustal components. It was found that the geochemical characteristics of Mesozoic magmas from the Schirmacher Oasis are identical to those of enriched tholeiites from the Afanasy Nikitin Rise and the central Kerguelen Plateau (Hole 749), which indicates that their enrichment was related to the ancient material of the Gondwana continent. This was caused by the opening of the Indian Ocean under the influence of the Karoo-Maud plume. This process was peculiar in that it occurred in the presence of nonspreading blocks of varying thickness, for instance, Elan Bank in the central Kerguelen Plateau, and was accompanied by the formation of intraplate volcanic rises, which are documented in the seafloor relief of basins around Antarctica. The geochemical characteristics of igneous rocks from the resulting rises (Afanasy Nikitin, Kerguelen, Naturaliste, and Ninetyeast Ridge) indicate the influence of processes related to crustal assimilation. The magmatism that occurred 40 Ma after the main phase of the Karoo-Maud volcanism at the margins of the adjacent continents of Australia (Bunbury basalts) and India (Rajmahal trapps) could be generated by the Karoo-Maud plume flowing along the developing spreading zone. The plume moved subsequently and was localized at the Kerguelen Plateau, where it occurs at present as an active hotspot.

Recrystallization of amorphous nano-tracks and uniform layers generated by swift-ion-beam irradiation in lithium niobate.

The thermal annealing of amorphous tracks of nanometer-size diameter generated in lithium niobate (LiNbO3) by Bromine ions at 45 MeV, i.e., in the electronic stopping regime, has been investigated by RBS/C spectrometry in the temperature range from 250°C to 350°C. Relatively low fluences have been used (<1012 cm−2) to produce isolated tracks. However, the possible effect of track overlapping has been investigated by varying the fluence between 3×1011 cm−2 and 1012 cm−2. The annealing process follows a two-step kinetics. In a first stage (I) the track radius decreases linearly with the annealing time. It obeys an Arrhenius-type dependence on annealing temperature with activation energy around 1.5 eV. The second stage (II) operates after the track radius has decreased down to around 2.5 nm and shows a much lower radial velocity. The data for stage I appear consistent with a solid-phase epitaxial process that yields a constant recrystallization rate at the amorphous-crystalline boundary. HRTEM has been used to monitor the existence and the size of the annealed isolated tracks in the second stage. On the other hand, the thermal annealing of homogeneous (buried) amorphous layers has been investigated within the same temperature range, on samples irradiated with Fluorine at 20 MeV and fluences of ∼1014 cm−2. Optical techniques are very suitable for this case and have been used to monitor the recrystallization of the layers. The annealing process induces a displacement of the crystalline-amorphous boundary that is also linear with annealing time, and the recrystallization rates are consistent with those measured for tracks. The comparison of these data with those previously obtained for the heavily damaged (amorphous) layers produced by elastic nuclear collisions is summarily discussed.

Social and content hybrid image recommender system for mobile social networks

One of the advantages of social networks is the possibility to socialize and personalize the content created or shared by the users. In mobile social networks, where the devices have limited capabilities in terms of screen size and computing power, Multimedia Recommender Systems help to present the most relevant content to the users, depending on their tastes, relationships and profile. Previous recommender systems are not able to cope with the uncertainty of automated tagging and are knowledge domain dependant. In addition, the instantiation of a recommender in this domain should cope with problems arising from the collaborative filtering inherent nature (cold start, banana problem, large number of users to run, etc.). The solution presented in this paper addresses the abovementioned problems by proposing a hybrid image recommender system, which combines collaborative filtering (social techniques) with content-based techniques, leaving the user the liberty to give these processes a personal weight. It takes into account aesthetics and the formal characteristics of the images to overcome the problems of current techniques, improving the performance of existing systems to create a mobile social networks recommender with a high degree of adaptation to any kind of user.

Recommender system for sport videos based on user audiovisual consumption

This paper describes a recommender system for sport videos, transmitted over the Internet and/or broadcast, in the context of large-scale events, which has been tested for the Olympic Games. The recommender is based on audiovisual consumption and does not depend on the number of users, running only on the client side. This avoids the concurrence, computation and privacy problems of central server approaches in scenarios with a large number of users, such as the Olympic Games. The system has been designed to take advantage of the information available in the videos, which is used along with the implicit information of the user and the modeling of his/her audiovisual content consumption. The system is thus transparent to the user, who does not need to take any specific action. Another important characteristic is that the system can produce recommendations for both live and recorded events. Testing has showed advantages compared to previous systems, as will be shown in the results.

Developing front-end Web 2.0 technologies to access services, content and things in the future Internet

The future Internet is expected to be composed of a mesh of interoperable web services accessible from all over the web. This approach has not yet caught on since global user?service interaction is still an open issue. This paper states one vision with regard to next-generation front-end Web 2.0 technology that will enable integrated access to services, contents and things in the future Internet. In this paper, we illustrate how front-ends that wrap traditional services and resources can be tailored to the needs of end users, converting end users into prosumers (creators and consumers of service-based applications). To do this, we propose an architecture that end users without programming skills can use to create front-ends, consult catalogues of resources tailored to their needs, easily integrate and coordinate front-ends and create composite applications to orchestrate services in their back-end. The paper includes a case study illustrating that current user-centred web development tools are at a very early stage of evolution. We provide statistical data on how the proposed architecture improves these tools. This paper is based on research conducted by the Service Front End (SFE) Open Alliance initiative.

Using clustering techniques for intelligent camera-based user interfaces

The area of Human-Machine Interface is growing fast due to its high importance in all technological systems. The basic idea behind designing human-machine interfaces is to enrich the communication with the technology in a natural and easy way. Gesture interfaces are a good example of transparent interfaces. Such interfaces must identify properly the action the user wants to perform, so the proper gesture recognition is of the highest importance. However, most of the systems based on gesture recognition use complex methods requiring high-resource devices. In this work, we propose to model gestures capturing their temporal properties, which significantly reduce storage requirements, and use clustering techniques, namely self-organizing maps and unsupervised genetic algorithm, for their classification. We further propose to train a certain number of algorithms with different parameters and combine their decision using majority voting in order to decrease the false positive rate. The main advantage of the approach is its simplicity, which enables the implementation using devices with limited resources, and therefore low cost. The testing results demonstrate its high potential.