IDT-3D: Identification and tracking in controlled environments using a 3D unified user interface

Identification and tracking of objects in specific environments such as harbors or security areas is a matter of great importance nowadays. With this purpose, numerous systems based on different technologies have been developed, resulting in a great amount of gathered data displayed through a variety of interfaces. Such amount of information has to be evaluated by human operators in order to take the correct decisions, sometimes under highly critical situations demanding both speed and accuracy. In order to face this problem we describe IDT-3D, a platform for identification and tracking of vessels in a harbour environment able to represent fused information in real time using a Virtual Reality application. The effectiveness of using IDT-3D as an integrated surveillance system is currently under evaluation. Preliminary results point to a significant decrease in the times of reaction and decision making of operators facing up a critical situation. Although the current application focus of IDT-3D is quite specific, the results of this research could be extended to the identification and tracking of targets in other controlled environments of interest as coastlines, borders or even urban areas.

VLC-based light-weight portable user interface for in-house applications.

Advances in solid-state lighting have overcome common limitations on optical wireless such as power needs due to light dispersion. It's been recently proposed the modification of lamp's drivers to take advantages of its switching behaviour to include data links maintaining the illumination control they provide. In this paper, a remote access application using visible light communications is presented that provides wireless access to a remote computer using a touchscreen as user interface

Diseño e implementación de un controlador de ratón gestual para Microsoft Windows con Kinect - Touchless Gesture User Interface

El dispositivo Microsoft Kinect for Windows y similares, han introducido en el mundo del PC una nueva forma de interacción denominada “Touchless Gesture User Interface” o TGUI (Interfaz de Usuario por Gestos sin Contacto) [Gentile et al. 2011]. Se trata de una tecnología novedosa en proceso de evolución. La tecnología de Kinect detecta la presencia de un usuario y monitoriza la posición en el espacio de sus articulaciones principales. Esta información permite desarrollar aplicaciones que posibiliten interactuar al usuario con una computadora mediante gestos y sin la necesidad de estar en contacto con periférico alguno. Desde la invención del periférico ratón en los años 60, resulta curioso que con la frenética evolución que ha experimentado el mundo de la informática en todos estos años, este dispositivo no haya sufrido cambios significativos o no haya sido incluso sustituido por otro periférico. En este proyecto se ha abordado el reto de desarrollar un controlador de ratón gestual para Windows utilizando Microsoft Kinect, de tal forma que se sustituya el uso del típico ratón y sea el propio usuario el que actúe como controlador mediante gestos y movimientos de sus manos. El resultado es llamativo y aporta numerosas mejoras y novedades frente a aplicaciones similares, aunque deja en evidencia algunas de las limitaciones de la tecnología implementada por Kinect a día de hoy. Es de esperar que cuando evolucione su tecnología, su uso se convierta en cotidiano.

MEDVIR: 3D visual interface applied to gene profile analisys.

The origins for this work arise in response to the increasing need for biologists and doctors to obtain tools for visual analysis of data. When dealing with multidimensional data, such as medical data, the traditional data mining techniques can be a tedious and complex task, even to some medical experts. Therefore, it is necessary to develop useful visualization techniques that can complement the expert’s criterion, and at the same time visually stimulate and make easier the process of obtaining knowledge from a dataset. Thus, the process of interpretation and understanding of the data can be greatly enriched. Multidimensionality is inherent to any medical data, requiring a time-consuming effort to get a clinical useful outcome. Unfortunately, both clinicians and biologists are not trained in managing more than four dimensions. Specifically, we were aimed to design a 3D visual interface for gene profile analysis easy in order to be used both by medical and biologist experts. In this way, a new analysis method is proposed: MedVir. This is a simple and intuitive analysis mechanism based on the visualization of any multidimensional medical data in a three dimensional space that allows interaction with experts in order to collaborate and enrich this representation. In other words, MedVir makes a powerful reduction in data dimensionality in order to represent the original information into a three dimensional environment. The experts can interact with the data and draw conclusions in a visual and quickly way.

MedVir: 3D visual interface applied to gene profile analysis

The use of data mining techniques for the gene profile discovery of diseases, such as cancer, is becoming usual in many researches. These techniques do not usually analyze the relationships between genes in depth, depending on the different variety of manifestations of the disease (related to patients). This kind of analysis takes a considerable amount of time and is not always the focus of the research. However, it is crucial in order to generate personalized treatments to fight the disease. Thus, this research focuses on finding a mechanism for gene profile analysis to be used by the medical and biologist experts. Results: In this research, the MedVir framework is proposed. It is an intuitive mechanism based on the visualization of medical data such as gene profiles, patients, clinical data, etc. MedVir, which is based on an Evolutionary Optimization technique, is a Dimensionality Reduction (DR) approach that presents the data in a three dimensional space. Furthermore, thanks to Virtual Reality technology, MedVir allows the expert to interact with the data in order to tailor it to the experience and knowledge of the expert.

Design evaluation of a prototype user interface to support a guideline-based decision support system in gestational diabetes

Gestational Diabetes (GD) has increased over the last 20 years, affecting up to 15% of pregnant women worldwide. The complications associated can be reduced with the appropriate glycemic control during the pregnancy.

3D simulation of complex shading affecting PV systems taking benefit from the power of graphics cards developed for the video game industry

Shading reduces the power output of a photovoltaic (PV) system. The design engineering of PV systems requires modeling and evaluating shading losses. Some PV systems are affected by complex shading scenes whose resulting PV energy losses are very difficult to evaluate with current modeling tools. Several specialized PV design and simulation software include the possibility to evaluate shading losses. They generally possess a Graphical User Interface (GUI) through which the user can draw a 3D shading scene, and then evaluate its corresponding PV energy losses. The complexity of the objects that these tools can handle is relatively limited. We have created a software solution, 3DPV, which allows evaluating the energy losses induced by complex 3D scenes on PV generators. The 3D objects can be imported from specialized 3D modeling software or from a 3D object library. The shadows cast by this 3D scene on the PV generator are then directly evaluated from the Graphics Processing Unit (GPU). Thanks to the recent development of GPUs for the video game industry, the shadows can be evaluated with a very high spatial resolution that reaches well beyond the PV cell level, in very short calculation times. A PV simulation model then translates the geometrical shading into PV energy output losses. 3DPV has been implemented using WebGL, which allows it to run directly from a Web browser, without requiring any local installation from the user. This also allows taken full benefits from the information already available from Internet, such as the 3D object libraries. This contribution describes, step by step, the method that allows 3DPV to evaluate the PV energy losses caused by complex shading. We then illustrate the results of this methodology to several application cases that are encountered in the world of PV systems design. Keywords: 3D, modeling, simulation, GPU, shading, losses, shadow mapping, solar, photovoltaic, PV, WebGL

Mejoras y extensiones en el Front-End de un modelo de predicción de acciones de un estudiante en un laboratorio virtual 3D

Este documento presenta las mejoras y las extensiones introducidas en la herramienta de visualización del modelo predictivo del comportamiento del estudiante o Student Behavior Predictor Viewer (SBPV), implementada en un trabajo anterior. El modelo predictivo del comportamiento del estudiante es parte de un sistema inteligente de tutoría, y se construye a partir de los registros de actividad de los estudiantes en un laboratorio virtual 3D, como el Laboratorio Virtual de Biotecnología Agroforestal, implementado en un trabajo anterior, y cuyos registros de actividad de los estudiantes se han utilizado para validar este trabajo fin de grado. El SBPV es una herramienta para visualizar una representación gráfica 2D del grafo extendido asociado con cualquiera de los clusters del modelo predictivo del estudiante. Además de la visualización del grafo extendido, el SBPV controla la navegación a través del grafo por medio del navegador web. Más concretamente, el SBPV permite al usuario moverse a través del grafo, ampliar o reducir el zoom del gráfico o buscar un determinado estado. Además, el SBPV también permite al usuario modificar el diseño predeterminado del grafo en la pantalla al cambiar la posición de los estados con el ratón. Como parte de este trabajo fin de grado, se han corregido errores existentes en la versión anterior y se han introducido una serie de mejoras en el rendimiento y la usabilidad. En este sentido, se han implementado nuevas funcionalidades, tales como la visualización del modelo de comportamiento de cada estudiante individualmente o la posibilidad de elegir el método de clustering para crear el modelo predictivo del estudiante; así como ha sido necesario rediseñar la interfaz de usuario cambiando el tipo de estructuras gráficas con que se muestran los elementos del modelo y mejorando la visualización del grafo al interaccionar el usuario con él. Todas estas mejoras se explican detenidamente en el presente documento.---ABSTRACT---This document presents the improvements and extensions made to the visualization tool Student Behavior Predictor Viewer (SBPV), implemented in a previous job. The student behavior predictive model is part of an intelligent tutoring system, and is built from the records of students activity in a 3D virtual laboratory, like the “Virtual Laboratory of Agroforestry Biotechnology” implemented in a previous work, and whose records of students activity have been used to validate this final degree work. The SBPV is a tool for visualizing a 2D graphical representation of the extended graph associated with any of the clusters of the student predictive model. Apart from visualizing the extended graph, the SBPV supports the navigation across the graph by means of desktop devices. More precisely, the SBPV allows user to move through the graph, to zoom in/out the graphic or to locate a given state. In addition, the SBPV also allows user to modify the default layout of the graph on the screen by changing the position of the states by means of the mouse. As part of this work, some bugs of the previous version have been fixed and some enhancements have been implemented to improve the performance and the usability. In this sense, we have implemented new features, such as the display of the model behavior of only one student or the possibility of selecting the clustering method to create the student predictive model; as well as it was necessary to redesign the user interface changing the type of graphic structures that show model elements and improving the rendering of the graph when the user interacts with it. All these improvements are explained in detail in the next sections.

End-User Development Success Factors and their Application to Composite Web Development Environments

The Future Internet is expected to be composed of a mesh of interoperable Web services accessed from all over the Web. This approach has not yet caught on since global user-service interaction is still an open issue. Successful composite applications rely on heavyweight service orchestration technologies that raise the bar far above end-user skills. The weakness lies in the abstraction of the underlying service front-end architecture rather than the infrastructure technologies themselves. In our opinion, the best approach is to offer end-to-end composition from user interface to service invocation, as well as an understandable abstraction of both building blocks and a visual composition technique. In this paper we formalize our vision with regard to the next-generation front-end Web technology that will enable integrated access to services, contents and things in the Future Internet. We present a novel reference architecture designed to empower non-technical end users to create and share their own self-service composite applications. A tool implementing this architecture has been developed as part of the European FP7 FAST Project and EzWeb Project, allowing us to validate the rationale behind our approach.

Using the Journalistic Metaphor to Design User Interfaces That Explain Sensor Data

Facilitating general access to data from sensor networks (including traffic, hydrology and other domains) increases their utility. In this paper we argue that the journalistic metaphor can be effectively used to automatically generate multimedia presentations that help non-expert users analyze and understand sensor data. The journalistic layout and style are familiar to most users. Furthermore, the journalistic approach of ordering information from most general to most specific helps users obtain a high-level understanding while providing them the freedom to choose the depth of analysis to which they want to go. We describe the general characteristics and architectural requirements for an interactive intelligent user interface for exploring sensor data that uses the journalistic metaphor. We also describe our experience in developing this interface in real-world domains (e.g., hydrology).

Diseño de una interfaz de representación gráfica de la base de datos de HRTF

El actual proyecto consiste en la creación de una interfaz gráfica de usuario (GUI) en entorno de MATLAB que realice una representación gráfica de la base de datos de HRTF (Head-Related Transfer Function). La función de transferencia de la cabeza es una herramienta muy útil en el estudio de la capacidad del ser humano para percibir su entorno sonoro, además de la habilidad de éste en la localización de fuentes sonoras en el espacio que le rodea. La HRTF biaural (terminología para referirse al conjunto de HRTF del oído izquierdo y del oído derecho) en sí misma, posee información de especial interés ya que las diferencias entre las HRTF de cada oído, conceden la información que nuestro sistema de audición utiliza en la percepción del campo sonoro. Por ello, la funcionalidad de la interfaz gráfica creada presenta gran provecho dentro del estudio de este campo. Las diferencias interaurales se caracterizan en amplitud y en tiempo, variando en función de la frecuencia. Mediante la transformada inversa de Fourier de la señal HRTF, se obtiene la repuesta al impulso de la cabeza, es decir, la HRIR (Head-Related Impulse Response). La cual, además de tener una gran utilidad en la creación de software o dispositivos de generación de sonido envolvente, se utiliza para obtener las diferencias ITD (Interaural Time Difference) e ILD (Interaural Time Difference), comúnmente denominados “parámetros de localización espacial”. La base de datos de HRTF contiene la información biaural de diferentes puntos de ubicación de la fuente sonora, formando una red de coordenadas esféricas que envuelve la cabeza del sujeto. Dicha red, según las medidas realizadas en la cámara anecoica de la EUITT (Escuela Universitaria de Ingeniería Técnica de Telecomunicación), presenta una precisión en elevación de 10º y en azimut de 5º. Los receptores son dos micrófonos alojados en el maniquí acústico llamado HATS (Hats and Torso Simulator) modelo 4100D de Brüel&Kjaer. Éste posee las características físicas que influyen en la percepción del entorno como son las formas del pabellón auditivo (pinna), de la cabeza, del cuello y del torso humano. Será necesario realizar los cálculos de interpolación para todos aquellos puntos no contenidos en la base de datos HRTF, este proceso es sumamente importante no solo para potenciar la capacidad de la misma sino por su utilidad para la comparación entre otras bases de datos existentes en el estudio de este ámbito. La interfaz gráfica de usuario está concebida para un manejo sencillo, claro y predecible, a la vez que interactivo. Desde el primer boceto del programa se ha tenido clara su filosofía, impuesta por las necesidades de un usuario que busca una herramienta práctica y de manejo intuitivo. Su diseño de una sola ventana reúne tanto los componentes de obtención de datos como los que hacen posible la representación gráfica de las HRTF, las HRIR y los parámetros de localización espacial, ITD e ILD. El usuario podrá ir alternando las representaciones gráficas a la vez que introduce las coordenadas de los puntos que desea visualizar, definidas por phi (elevación) y theta (azimut). Esta faceta de la interfaz es la que le otorga una gran facilidad de acceso y lectura de la información representada en ella. Además, el usuario puede introducir valores incluidos en la base de datos o valores intermedios a estos, de esta manera, se indica a la interfaz la necesidad de realizar la interpolación de los mismos. El método de interpolación escogido es el de la ponderación de la distancia inversa entre puntos. Dependiendo de los valores introducidos por el usuario se realizará una interpolación de dos o cuatro puntos, siendo éstos limítrofes al valor introducido, ya sea de phi o theta. Para añadir versatilidad a la interfaz gráfica de usuario, se ha añadido la opción de generar archivos de salida en forma de imagen de las gráficas representadas, de tal forma que el usuario pueda extraer los datos que le interese para cualquier valor de phi y theta. Se completa el presente proyecto fin de carrera con un trabajo de investigación y estudio comparativo de la función y la aplicación de las bases de datos de HRTF dentro del marco científico y de investigación. Esto ha hecho posible concentrar información relacionada a través de revistas científicas de investigación como la JAES (Journal of the Audio Engineering Society) o la ASA (Acoustical Society of America), además, del IEEE ( Institute of Electrical and Electronics Engineers) o la “Web of knowledge” entre otras. Además de realizar la búsqueda en estas fuentes, se ha optado por vías de información más comunes como Google Académico o el portal de acceso “Ingenio” a los todos los recursos electrónicos contenidos en la base de datos de la universidad. El estudio genera una ampliación en el conocimiento de la labor práctica de las HRTF. La mayoría de los estudios enfocan sus esfuerzos en mejorar la percepción del evento sonoro mediante su simulación en la escucha estéreo o multicanal. A partir de las HRTF, esto es posible mediante el análisis y el cálculo de datos como pueden ser las regresiones, siendo éstas muy útiles en la predicción de una medida basándose en la información de la actual. Otro campo de especial interés es el de la generación de sonido 3D. Mediante la base de datos HRTF es posible la simulación de una señal biaural. Se han diseñado algoritmos que son implementados en dispositivos DSP, de tal manera que por medio de retardos interaurales y de diferencias espectrales es posible llegar a un resultado óptimo de sonido envolvente, sin olvidar la importancia de los efectos de reverberación para conseguir un efecto creíble de sonido envolvente. Debido a la complejidad computacional que esto requiere, gran parte de los estudios coinciden en desarrollar sistemas más eficientes, llegando a objetivos tales como la generación de sonido 3D en tiempo real. ABSTRACT. This project involves the creation of a Graphic User Interface (GUI) in the Matlab environment which creates a graphic representation of the HRTF (Head-Related Transfer Function) database. The head transfer function is a very useful tool in the study of the capacity of human beings to perceive their sound environment, as well as their ability to localise sound sources in the area surrounding them. The binaural HRTF (terminology which refers to the HRTF group of the left and right ear) in itself possesses information of special interest seeing that the differences between the HRTF of each ear admits the information that our system of hearing uses in the perception of each sound field. For this reason, the functionality of the graphic interface created presents great benefits within the study of this field. The interaural differences are characterised in space and in time, varying depending on the frequency. By means of Fourier's transformed inverse of the HRTF signal, the response to the head impulse is obtained, in other words, the HRIR (Head-Related Impulse Response). This, as well as having a great use in the creation of software or surround sound generating devices, is used to obtain ITD differences (Interaural Time Difference) and ILD (Interaural Time Difference), commonly named “spatial localisation parameters”. The HRTF database contains the binaural information of different points of sound source location, forming a network of spherical coordinates which surround the subject's head. This network, according to the measures carried out in the anechoic chamber at the EUITT (School of Telecommunications Engineering) gives a precision in elevation of 10º and in azimuth of 5º. The receivers are two microphones placed on the acoustic mannequin called HATS (Hats and Torso Simulator) Brüel&Kjaer model 4100D. This has the physical characteristics which affect the perception of the surroundings which are the forms of the auricle (pinna), the head, neck and human torso. It will be necessary to make interpolation calculations for all those points which are not contained the HRTF database. This process is extremely important not only to strengthen the database's capacity but also for its usefulness in making comparisons with other databases that exist in the study of this field. The graphic user interface is conceived for a simple, clear and predictable use which is also interactive. Since the first outline of the program, its philosophy has been clear, based on the needs of a user who requires a practical tool with an intuitive use. Its design with only one window unites not only the components which obtain data but also those which make the graphic representation of the HRTFs possible, the hrir and the ITD and ILD spatial location parameters. The user will be able to alternate the graphic representations at the same time as entering the point coordinates that they wish to display, defined by phi (elevation) and theta (azimuth). The facet of the interface is what provides the great ease of access and reading of the information displayed on it. In addition, the user can enter values included in the database or values which are intermediate to these. It is, likewise, indicated to the interface the need to carry out the interpolation of these values. The interpolation method is the deliberation of the inverse distance between points. Depending on the values entered by the user, an interpolation of two or four points will be carried out, with these being adjacent to the entered value, whether that is phi or theta. To add versatility to the graphic user interface, the option of generating output files in the form of an image of the graphics displayed has been added. This is so that the user may extract the information that interests them for any phi and theta value. This final project is completed with a research and comparative study essay on the function and application of HRTF databases within the scientific and research framework. It has been possible to collate related information by means of scientific research magazines such as the JAES (Journal of the Audio Engineering Society), the ASA (Acoustical Society of America) as well as the IEEE (Institute of Electrical and Electronics Engineers) and the “Web of knowledge” amongst others. In addition to carrying out research with these sources, I also opted to use more common sources of information such as Academic Google and the “Ingenio” point of entry to all the electronic resources contained on the university databases. The study generates an expansion in the knowledge of the practical work of the HRTF. The majority of studies focus their efforts on improving the perception of the sound event by means of its simulation in stereo or multichannel listening. With the HRTFs, this is possible by means of analysis and calculation of data as can be the regressions. These are very useful in the prediction of a measure being based on the current information. Another field of special interest is that of the generation of 3D sound. Through HRTF databases it is possible to simulate the binaural signal. Algorithms have been designed which are implemented in DSP devices, in such a way that by means of interaural delays and wavelength differences it is possible to achieve an excellent result of surround sound, without forgetting the importance of the effects of reverberation to achieve a believable effect of surround sound. Due to the computational complexity that this requires, a great many studies agree on the development of more efficient systems which achieve objectives such as the generation of 3D sound in real time.

Distributed Web Interface for Real-Time Spatial Audio Reproduction System

SSR es el acrónimo de SoundScape Renderer (tool for real-time spatial audio reproduction providing a variety of rendering algorithms), es un programa escrito en su mayoría en C++. El programa permite al usuario escuchar tanto sonidos grabados con anterioridad como sonidos en directo. El sonido o los sonidos se oirán, desde el punto de vista del oyente, como si el sonido se produjese en el punto que el programa decida, lo interesante de este proyecto es que el sonido podrá cambiar de lugar, moverse, etc. Todo en tiempo real. Esto se consigue sin modificar el sonido al grabarlo pero sí al emitirlo, el programa calcula las variaciones necesarias para que al emitir el sonido al oyente le llegue como si el sonido realmente se generase en un punto del espacio o lo más parecido posible. La sensación de movimiento no deja de ser el punto anterior cambiando de lugar. La idea era crear una aplicación web basada en Canvas de HTML5 que se comunicará con esta interfaz de usuario remota. Así se solucionarían todos los problemas de compatibilidad ya que cualquier dispositivo con posibilidad de visualizar páginas web podría correr una aplicación basada en estándares web, por ejemplo un sistema con Windows o un móvil con navegador. El protocolo debía de ser WebSocket porque es un protocolo HTML5 y ofrece las “garantías” de latencia que una aplicación con necesidades de información en tiempo real requiere. Nos permite una comunicación full-dúplex asíncrona sin mucho payload que es justo lo que se venía a evitar al no usar polling normal de HTML. El problema que surgió fue que la interfaz de usuario de red que tenía el programa no era compatible con WebSocket debido a un handshacking inicial y obligatorio que realiza el protocolo, por lo que se necesitaba otra interfaz de red. Se decidió entonces cambiar a JSON como formato para el intercambio de mensajes. Al final el proyecto comprende no sólo la aplicación web basada en Canvas sino también un servidor funcional y la definición de una nueva interfaz de usuario de red con su protocolo añadido. ABSTRACT. This project aims to become a part of the SSR tool to extend its capabilities in the field of the access. SSR is an acronym for SoundScape Renderer, is a program mostly written in C++ that allows you to hear already recorded or live sound with a variety of sound equipment as if the sound came from a desired place in the space. Like the web-page of the SSR says surely better explained: “The SoundScape Renderer (SSR) is a tool for real-time spatial audio reproduction providing a variety of rendering algorithms.” The application can be used with a graphical interface written in Qt but has also a network interface for external applications to use it. This network interface communicates using XML messages. A good example of it is the Android client. This Android client is already working. In order to use the application should be run it by loading an audio source and the wanted environment so that the renderer knows what to do. In that moment the server binds and anyone can use the network interface. Since the network interface is documented everyone can make an application to interact with this network interface. So the application can have as many user interfaces as wanted. The part that is developed in this project has nothing to do neither with audio rendering nor even with the reproduction of the spatial audio. The part that is developed here is about the interface used in the SSR application. As it can be deduced from the title: “Distributed Web Interface for Real-Time Spatial Audio Reproduction System”, this work aims only to offer the interface via web for the SSR (“Real-Time Spatial Audio Reproduction System”). The idea is not to make a new graphical interface for SSR but to allow more types of interfaces and communication. To accomplish the objective of allowing more graphical interfaces this project is going to use a new network interface. By now the SSR application is using only XML for data interchange but this new network interface support JSON. This project comprehends the server that launch the application, the user interface and the new network interface. It is done with these modules in order to allow creating new user interfaces that can communicate with the server or new servers that can communicate with the user interface by defining a complete network interface for data interchange.

A component- and connector-based approach for end-user composite web applications development

Enabling real end-user development is the next logical stage in the evolution of Internet-wide service-based applications. Successful composite applications rely on heavyweight service orchestration technologies that raise the bar far above end-user skills. This weakness can be attributed to the fact that the composition model does not satisfy end-user needs rather than to the actual infrastructure technologies. In our opinion, the best way to overcome this weakness is to offer end-to-end composition from the user interface to service invocation, plus an understandable abstraction of building blocks and a visual composition technique empowering end users to develop their own applications. In this paper, we present a visual framework for end users, called FAST, which fulfils this objective. FAST implements a novel composition model designed to empower non-programmer end users to create and share their own self-service composite applications in a fully visual fashion. We projected the development environment implementing this model as part of the European FP7 FAST Project, which was used to validate the rationale behind our approach.

Hammer: an Android based application for end-user industrial robot programming

This paper presents a novel tablet based end-user interface for industrial robot programming (called Hammer). This application makes easier to program tasks for industrial robots like polishing, milling or grinding. It is based on the Scratch programming language, but specifically design and created for Android OS. It is a visual programming concept that allows non-skilled programmer operators to create programs. The application also allows to monitor the tasks while it is being executed by overlapping real time information through augmented reality. The application includes a teach pendant screen that can be customized according to the operator needs at every moment.

Una propuesta de modelado del estudiante basada en ontologías y diagnóstico pedagógico-cognitivo no monótono

Los recientes avances tecnológicos han encontrado un potencial campo de explotación en la educación asistida por computador. A finales de los años 90 surgió un nuevo campo de investigación denominado Entornos Virtuales Inteligentes para el Entrenamiento y/o Enseñanza (EVIEs), que combinan dos áreas de gran complejidad: Los Entornos Virtuales (EVs) y los Sistemas de Tutoría Inteligente (STIs). De este modo, los beneficios de los entornos 3D (simulación de entornos de alto riesgo o entornos de difícil uso, etc.) pueden combinarse con aquéllos de un STIs (personalización de materias y presentaciones, adaptación de la estrategia de tutoría a las necesidades del estudiante, etc.) para proporcionar soluciones educativas/de entrenamiento con valores añadidos. El Modelo del Estudiante, núcleo de un SIT, representa el conocimiento y características del estudiante, y refleja el proceso de razonamiento del estudiante. Su complejidad es incluso superior cuando los STIs se aplican a EVs porque las nuevas posibilidades de interacción proporcionadas por estos entornos deben considerarse como nuevos elementos de información clave para el modelado del estudiante, incidiendo en todo el proceso educativo: el camino seguido por el estudiante durante su navegación a través de escenarios 3D; el comportamiento no verbal tal como la dirección de la mirada; nuevos tipos de pistas e instrucciones que el módulo de tutoría puede proporcionar al estudiante; nuevos tipos de preguntas que el estudiante puede formular, etc. Por consiguiente, es necesario que la estructura de los STIs, embebida en el EVIE, se enriquezca con estos aspectos, mientras mantiene una estructura clara, estructurada, y bien definida. La mayoría de las aproximaciones al Modelo del Estudiante en STIs y en IVETs no consideran una taxonomía de posibles conocimientos acerca del estudiante suficientemente completa. Además, la mayoría de ellas sólo tienen validez en ciertos dominios o es difícil su adaptación a diferentes STIs. Para vencer estas limitaciones, hemos propuesto, en el marco de esta tesis doctoral, un nuevo mecanismo de Modelado del Estudiante basado en la Ingeniería Ontológica e inspirado en principios pedagógicos, con un modelo de datos sobre el estudiante amplio y flexible que facilita su adaptación y extensión para diferentes STIs y aplicaciones de aprendizaje, además de un método de diagnóstico con capacidades de razonamiento no monótono. El método de diagnóstico es capaz de inferir el estado de los objetivos de aprendizaje contenidos en el SIT y, a partir de él, el estado de los conocimientos del estudiante durante su proceso de aprendizaje. La aproximación almodelado del estudiante propuesta ha sido implementada e integrada en un agente software (el agente de modelado del estudiante) dentro de una plataforma software existente para el desarrollo de EVIEs denominadaMAEVIF. Esta plataforma ha sido diseñada para ser fácilmente configurable para diferentes aplicaciones de aprendizaje. El modelado del estudiante presentado ha sido implementado e instanciado para dos tipos de entornos de aprendizaje: uno para aprendizaje del uso de interfaces gráficas de usuario en una aplicación software y para un Entorno Virtual para entrenamiento procedimental. Además, se ha desarrollado una metodología para guiar en la aplicación del esta aproximación de modelado del estudiante a cada sistema concreto.---ABSTRACT---Recent technological advances have found a potential field of exploitation in computeraided education. At the end of the 90’s a new research field emerged, the so-called Intelligent Virtual Environments for Training and/or Education (IVETs), which combines two areas of great complexity: Virtual Environments (VE) and Intelligent Tutoring Systems (ITS). In this way, the benefits of 3D environments (simulation of high risk or difficult-to-use environments, etc.) may be combined with those of an ITS (content and presentation customization, adaptation of the tutoring strategy to the student requirements, etc.) in order to provide added value educational/training solutions. The StudentModel, core of an ITS, represents the student’s knowledge and characteristics, and reflects the student’s reasoning process. Its complexity is even higher when the ITSs are applied on VEs because the new interaction possibilities offered by these environments must be considered as new key information pieces for student modelling, impacting all the educational process: the path followed by the student during their navigation through 3D scenarios; non-verbal behavior such as gaze direction; new types of hints or instructions that the tutoring module can provide to the student; new question types that the student can ask, etc. Thus, it is necessary for the ITS structure, which is embedded in the IVET, to be enriched by these aspects, while keeping a clear, structured and well defined architecture. Most approaches to SM on ITSs and IVETs don’t consider a complete enough taxonomy of possible knowledge about the student. In addition, most of them have validity only in certain domains or they are hard to be adapted for different ITSs. In order to overcome these limitations, we have proposed, in the framework of this doctoral research project, a newStudentModeling mechanism that is based onOntological Engineering and inspired on pedagogical principles, with a wide and flexible data model about the student that facilitates its adaptation and extension to different ITSs and learning applications, as well as a rich diagnosis method with non-monotonic reasoning capacities. The diagnosis method is able to infer the state of the learning objectives encompassed by the ITS and, fromit, the student’s knowledge state during the student’s process of learning. The proposed student modelling approach has been implemented and integrated in a software agent (the student modeling agent) within an existing software platform for the development of IVETs called MAEVIF. This platform was designed to be easily configurable for different learning applications. The proposed student modeling has been implemented and it has been instantiated for two types of learning environments: one for learning to use the graphical user interface of a software application and a Virtual Environment for procedural training. In addition, a methodology to guide on the application of this student modeling approach to each specific system has been developed.