Semantic Service Architectures for Smart Environments

Many industries and academic institutions share the vision that an appropriate use of information originated from the environment may add value to services in multiple domains and may help humans in dealing with the growing information overload which often seems to jeopardize our life. It is also clear that information sharing and mutual understanding between software agents may impact complex processes where many actors (humans and machines) are involved, leading to relevant socioeconomic benefits. Starting from these two input, architectural and technological solutions to enable “environment-related cooperative digital services” are here explored. The proposed analysis starts from the consideration that our environment is physical space and here diversity is a major value. On the other side diversity is detrimental to common technological solutions, and it is an obstacle to mutual understanding. An appropriate environment abstraction and a shared information model are needed to provide the required levels of interoperability in our heterogeneous habitat. This thesis reviews several approaches to support environment related applications and intends to demonstrate that smart-space-based, ontology-driven, information-sharing platforms may become a flexible and powerful solution to support interoperable services in virtually any domain and even in cross-domain scenarios. It also shows that semantic technologies can be fruitfully applied not only to represent application domain knowledge. For example semantic modeling of Human-Computer Interaction may support interaction interoperability and transformation of interaction primitives into actions, and the thesis shows how smart-space-based platforms driven by an interaction ontology may enable natural ad flexible ways of accessing resources and services, e.g, with gestures. An ontology for computational flow execution has also been built to represent abstract computation, with the goal of exploring new ways of scheduling computation flows with smart-space-based semantic platforms.

Innovative Man Machine Interfaces In Aeronautics

The research activity focused on the study, design and evaluation of innovative human-machine interfaces based on virtual three-dimensional environments. It is based on the brain electrical activities recorded in real time through the electrical impulses emitted by the brain waves of the user. The achieved target is to identify and sort in real time the different brain states and adapt the interface and/or stimuli to the corresponding emotional state of the user. The setup of an experimental facility based on an innovative experimental methodology for “man in the loop" simulation was established. It allowed involving during pilot training in virtually simulated flights, both pilot and flight examiner, in order to compare the subjective evaluations of this latter to the objective measurements of the brain activity of the pilot. This was done recording all the relevant information versus a time-line. Different combinations of emotional intensities obtained, led to an evaluation of the current situational awareness of the user. These results have a great implication in the current training methodology of the pilots, and its use could be extended as a tool that can improve the evaluation of a pilot/crew performance in interacting with the aircraft when performing tasks and procedures, especially in critical situations. This research also resulted in the design of an interface that adapts the control of the machine to the situation awareness of the user. The new concept worked on, aimed at improving the efficiency between a user and the interface, and gaining capacity by reducing the user’s workload and hence improving the system overall safety. This innovative research combining emotions measured through electroencephalography resulted in a human-machine interface that would have three aeronautical related applications: • An evaluation tool during the pilot training; • An input for cockpit environment; • An adaptation tool of the cockpit automation.

Sviluppo di un sistema bci eeg-based per il controllo del movimento di un modello di arto superiore

Nel presente lavoro di tesi è stato sviluppato e testato un sistema BCI EEG-based che sfrutta la modulazione dei ritmi sensorimotori tramite immaginazione motoria della mano destra e della mano sinistra. Per migliorare la separabilità dei due stati mentali, in questo lavoro di tesi si è sfruttato l'algoritmo CSP (Common Spatial Pattern), in combinazione ad un classificatore lineare SVM. I due stati mentali richiesti sono stati impiegati per controllare il movimento (rotazione) di un modello di arto superiore a 1 grado di libertà, simulato sullo schermo. Il cuore del lavoro di tesi è consistito nello sviluppo del software del sistema BCI (basato su piattaforma LabVIEW 2011), descritto nella tesi. L'intero sistema è stato poi anche testato su 4 soggetti, per 6 sessioni di addestramento.

Le interfacce neurali e le loro possibili applicazioni nell'assistive technology

Ogni anno si registra un crescente aumento delle persone affette da patologie neurodegenerative come la sclerosi laterale amiotrofica, la sclerosi multipla, la malattia di Parkinson e persone soggette a gravi disabilità motorie dovute ad ictus, paralisi cerebrale o lesioni al midollo spinale. Spesso tali condizioni comportano menomazioni molto invalidanti e permanenti delle vie nervose, deputate al controllo dei muscoli coinvolti nell’esecuzione volontaria delle azioni. Negli ultimi anni, molti gruppi di ricerca si sono interessati allo sviluppo di sistemi in grado di soddisfare le volontà dell’utente. Tali sistemi sono generalmente definiti interfacce neurali e non sono pensati per funzionare autonomamente ma per interagire con il soggetto. Tali tecnologie, note anche come Brain Computer Interface (BCI), consentono una comunicazione diretta tra il cervello ed un’apparecchiatura esterna, basata generalmente sull’elettroencefalografia (EEG), in grado di far comunicare il sistema nervoso centrale con una periferica esterna. Tali strumenti non impiegano le usuali vie efferenti coinvolte nella produzione di azioni quali nervi e muscoli, ma collegano l'attività cerebrale ad un computer che ne registra ed interpreta le variazioni, permettendo quindi di ripristinare in modo alternativo i collegamenti danneggiati e recuperare, almeno in parte, le funzioni perse. I risultati di numerosi studi dimostrano che i sistemi BCI possono consentire alle persone con gravi disabilità motorie di condividere le loro intenzioni con il mondo circostante e provano perciò il ruolo importante che esse sono in grado di svolgere in alcune fasi della loro vita.

Innovazioni nel campo dell'ElettroEncefalografia

Il funzionamento del cervello umano, organo responsabile di ogni nostra azione e pensiero, è sempre stato di grande interesse per la ricerca scientifica. Dopo aver compreso lo sviluppo dei potenziali elettrici da parte di nuclei neuronali in risposta a stimoli, si è riusciti a graficare il loro andamento con l'avvento dell'ElettroEncefaloGrafia (EEG). Tale tecnologia è entrata a far parte degli esami di routine per la ricerca di neuropsicologia e di interesse clinico, poiché permette di diagnosticare e discriminare i vari tipi di epilessia, la presenza di traumi cranici e altre patologie del sistema nervoso centrale. Purtroppo presenta svariati difetti: il segnale è affetto da disturbi e richiede un'adeguata elaborazione tramite filtraggio e amplificazione, rimanendo comunque sensibile a disomogeneità dei tessuti biologici e rendendo difficoltoso il riconoscimento delle sorgenti del segnale che si sono attivate durante l'esame (il cosiddetto problema inverso). Negli ultimi decenni la ricerca ha portato allo sviluppo di nuove tecniche d'indagine, di particolare interesse sono la ElettroEncefaloGrafia ad Alta Risoluzione (HREEG) e la MagnetoEncefaloGrafia (MEG). L'HREEG impiega un maggior numero di elettrodi (fino a 256) e l'appoggio di accurati modelli matematici per approssimare la distribuzione di potenziale elettrico sulla cute del soggetto, garantendo una migliore risoluzione spaziale e maggior sicurezza nel riscontro delle sorgenti neuronali. Il progresso nel campo dei superconduttori ha reso possibile lo sviluppo della MEG, che è in grado di registrare i deboli campi magnetici prodotti dai segnali elettrici corticali, dando informazioni immuni dalle disomogeneità dei tessuti e andando ad affiancare l'EEG nella ricerca scientifica. Queste nuove tecnologie hanno aperto nuovi campi di sviluppo, più importante la possibilità di comandare protesi e dispositivi tramite sforzo mentale (Brain Computer Interface). Il futuro lascia ben sperare per ulteriori innovazioni.

Impact of acquired enamel pellicle modification on initial dental erosion

The acquired enamel pellicle that forms on the tooth surface serves as a natural protective barrier against dental erosion. Numerous proteins composing the pellicle serve different functions within this thin layer. Our study examined the effect of incorporated mucin and casein on the erosion-inhibiting potential of the acquired enamel pellicle. Cyclic acidic conditions were applied to mimic the erosive environment present at the human enamel interface during the consumption of soft drinks. One hundred enamel specimens were prepared for microhardness tests and distributed randomly into 5 groups (n = 20) that received the following treatment: deionized water, humidity chamber, mucin, casein, or a combination of mucin and casein. Each group was exposed to 3 cycles of a 2-hour incubation in human saliva, followed by a 2-hour treatment in the testing solution and a 1-min exposure to citric acid. The microhardness analysis demonstrated that the mixture of casein and mucin significantly improved the erosion-inhibiting properties of the human pellicle layer. The addition of individual proteins did not statistically impact the function of the pellicle. These data suggest that protein-protein interactions may play an important role in the effectiveness of the pellicle to prevent erosion.

Driver Safety in Far-side and Far-oblique Crashes: A Study of Patrol Vehicles in the United States of America

The dissertation titled "Driver Safety in Far-side and Far-oblique Crashes" presents a novel approach to assessing vehicle cockpit safety by integrating Human Factors and Applied Mechanics. The methodology of this approach is aimed at improving safety in compact mobile workspaces such as patrol vehicle cockpits. A statistical analysis performed using Michigan state's traffic crash data to assess various contributing factors that affect the risk of severe driver injuries showed that the risk was greater for unrestrained drivers (OR=3.38, p<0.0001) and for incidents involving front and far-side crashes without seatbelts (OR=8.0 and 23.0 respectively, p<0.005). Statistics also showed that near-side and far-side crashes pose similar threat to driver injury severity. A Human Factor survey was conducted to assess various Human-Machine/Human-Computer Interaction aspects in patrol vehicle cockpits. Results showed that tasks requiring manual operation, especially the usage of laptop, would require more attention and potentially cause more distraction. A vehicle survey conducted to evaluate ergonomics-related issues revealed that some of the equipment was in airbag deployment zones. In addition, experiments were conducted to assess the effects on driver distraction caused by changing the position of in-car accessories. A driving simulator study was conducted to mimic HMI/HCI in a patrol vehicle cockpit (20 subjects, average driving experience = 5.35 years, s.d. = 1.8). It was found that the mounting locations of manual tasks did not result in a significant change in response times. Visual displays resulted in response times less than 1.5sec. It can also be concluded that the manual task was equally distracting regardless of mounting positions (average response time was 15 secs). Average speeds and lane deviations did not show any significant results. Data from 13 full-scale sled tests conducted to simulate far-side impacts at 70 PDOF and 40 PDOF was used to analyze head injuries and HIC/AIS values. It was found that accelerations generated by the vehicle deceleration alone were high enough to cause AIS 3 - AIS 6 injuries. Pretensioners could mitigated injuries only in 40 PDOF (oblique) impacts but are useless in 70 PDOF impacts. Seat belts were ineffective in protecting the driver's head from injuries. Head would come in contact with the laptop during a far-oblique (40 PDOF) crash and far-side door for an angle-type crash (70 PDOF). Finite Element analysis head-laptop impact interaction showed that the contact velocity was the most crucial factor in causing a severe (and potentially fatal) head injury. Results indicate that no equipment may be mounted in driver trajectory envelopes. A very narrow band of space is left in patrol vehicles for installation of manual-task equipment to be both safe and ergonomic. In case of a contact, the material stiffness and damping properties play a very significant role in determining the injury outcome. Future work may be done on improving the interiors' material properties to better absorb and dissipate kinetic energy of the head. The design of seat belts and pretensioners may also be seen as an essential aspect to be further improved.

Playing with the Real World

In this paper we provide a framework that enables the rapid development of applications using non-standard input devices. Flash is chosen as programming language since it can be used for quickly assembling applications. We overcome the difficulties of Flash to access external devices by introducing a very generic concept: The state information generated by input devices is transferred to a PC where a program collects them, interprets them and makes them available on a web server. Application developers can now integrate a Flash component that accesses the data stored in XML format and directly use it in their application.

Augmenting a Laser Pointer with a Diffraction Grating for Monoscopic 6DOF Detection

This article illustrates the detection of 6 degrees of freedom (DOF) for Virtual Environment interactions using a modified simple laser pointer device and a camera. The laser pointer is combined with a diffraction rating to project a unique laser grid onto the projection planes used in projection-based immersive VR setups. The distortion of the projected grid is used to calculate the translational and rotational degrees of freedom required for human-computer interaction purposes.

Multi-Contact Grasp Interaction for Virtual Environments

The grasping of virtual objects has been an active research field for several years. Solutions providing realistic grasping rely on special hardware or require time-consuming parameterizations. Therefore, we introduce a flexible grasping algorithm enabling grasping without computational complex physics. Objects can be grasped and manipulated with multiple fingers. In addition, multiple objects can be manipulated simultaneously with our approach. Through the usage of contact sensors the technique is easily configurable and versatile enough to be used in different scenarios.

Evaluation of Binocular Eye Trackers and Algorithms for 3D Gaze Interaction in Virtual Reality Environments

Tracking user’s visual attention is a fundamental aspect in novel human-computer interaction paradigms found in Virtual Reality. For example, multimodal interfaces or dialogue-based communications with virtual and real agents greatly benefit from the analysis of the user’s visual attention as a vital source for deictic references or turn-taking signals. Current approaches to determine visual attention rely primarily on monocular eye trackers. Hence they are restricted to the interpretation of two-dimensional fixations relative to a defined area of projection. The study presented in this article compares precision, accuracy and application performance of two binocular eye tracking devices. Two algorithms are compared which derive depth information as required for visual attention-based 3D interfaces. This information is further applied to an improved VR selection task in which a binocular eye tracker and an adaptive neural network algorithm is used during the disambiguation of partly occluded objects.

International cooperation in ambient computing education

Funded by the US-EU Atlantis Program, the International Cooperation in Ambient Computing Education Project is establishing an international knowledge-building community for developing a broader computer science curriculum aimed at preparing students for real-world problems in a multidisciplinary, global world. The project is collaboration among Troy University (USA), University of Sunderland (UK), FernUniversität in Hagen (Germany), Universidade do Algarve (Portugal), University of Arkansas at Little Rock (USA) and San Diego State University (USA). The curriculum will include aspects of social science, cognitive science, human-computer interaction, organizational studies, global studies, and particular application areas as well as core computer science subjects. Programs offered at partner institutions will form trajectories through the curriculum. A degree will be defined in terms of combinations of trajectories which will satisfy degree requirements set by accreditation organizations. This is expected to lead to joint- or dual-degree programs among the partner institutions in the future. This paper describes the goals and activities of the project and discusses implementation issues.

Constructing a user experience user experience-based mobile learning environment

Mobile learning, in the past defined as learning with mobile devices, now refers to any type of learning-on-the-go or learning that takes advantage of mobile technologies. This new definition shifted its focus from the mobility of technology to the mobility of the learner (O'Malley and Stanton 2002; Sharples, Arnedillo-Sanchez et al. 2009). Placing emphasis on the mobile learner’s perspective requires studying “how the mobility of learners augmented by personal and public technology can contribute to the process of gaining new knowledge, skills, and experience” (Sharples, Arnedillo-Sanchez et al. 2009). The demands of an increasingly knowledge based society and the advances in mobile phone technology are combining to spur the growth of mobile learning. Around the world, mobile learning is predicted to be the future of online learning, and is slowly entering the mainstream education. However, for mobile learning to attain its full potential, it is essential to develop more advanced technologies that are tailored to the needs of this new learning environment. A research field that allows putting the development of such technologies onto a solid basis is user experience design, which addresses how to improve usability and therefore user acceptance of a system. Although there is no consensus definition of user experience, simply stated it focuses on how a person feels about using a product, system or service. It is generally agreed that user experience adds subjective attributes and social aspects to a space that has previously concerned itself mainly with ease-of-use. In addition, it can include users’ perceptions of usability and system efficiency. Recent advances in mobile and ubiquitous computing technologies further underline the importance of human-computer interaction and user experience (feelings, motivations, and values) with a system. Today, there are plenty of reports on the limitations of mobile technologies for learning (e.g., small screen size, slow connection), but there is a lack of research on user experience with mobile technologies. This dissertation will fill in this gap by a new approach in building a user experience-based mobile learning environment. The optimized user experience we suggest integrates three priorities, namely a) content, by improving the quality of delivered learning materials, b) the teaching and learning process, by enabling live and synchronous learning, and c) the learners themselves, by enabling a timely detection of their emotional state during mobile learning. In detail, the contributions of this thesis are as follows: • A video codec optimized for screencast videos which achieves an unprecedented compression rate while maintaining a very high video quality, and a novel UI layout for video lectures, which together enable truly mobile access to live lectures. • A new approach in HTTP-based multimedia delivery that exploits the characteristics of live lectures in a mobile context and enables a significantly improved user experience for mobile live lectures. • A non-invasive affective learning model based on multi-modal emotion detection with very high recognition rates, which enables real-time emotion detection and subsequent adaption of the learning environment on mobile devices. The technology resulting from the research presented in this thesis is in daily use at the School of Continuing Education of Shanghai Jiaotong University (SOCE), a blended-learning institution with 35.000 students.

Towards an Emergent Semantic Web

In his in uential article about the evolution of the Web, Berners-Lee [1] envisions a Semantic Web in which humans and computers alike are capable of understanding and processing information. This vision is yet to materialize. The main obstacle for the Semantic Web vision is that in today's Web meaning is rooted most often not in formal semantics, but in natural language and, in the sense of semiology, emerges not before interpretation and processing. Yet, an automated form of interpretation and processing can be tackled by precisiating raw natural language. To do that, Web agents extract fuzzy grassroots ontologies through induction from existing Web content. Inductive fuzzy grassroots ontologies thus constitute organically evolved knowledge bases that resemble automated gradual thesauri, which allow precisiating natural language [2]. The Web agents' underlying dynamic, self-organizing, and best-effort induction, enable a sub-syntactical bottom up learning of semiotic associations. Thus, knowledge is induced from the users' natural use of language in mutual Web interactions, and stored in a gradual, thesauri-like lexical-world knowledge database as a top-level ontology, eventually allowing a form of computing with words [3]. Since when computing with words the objects of computation are words, phrases and propositions drawn from natural languages, it proves to be a practical notion to yield emergent semantics for the Semantic Web. In the end, an improved understanding by computers on the one hand should upgrade human- computer interaction on the Web, and, on the other hand allow an initial version of human- intelligence amplification through the Web.