Interactive sensory objects for and by people with learning disabilities

This project engages people with learning disabilities to participate as co-researchers and explore museum interpretation through multisensory workshops using microcontrollers and sensors to enable alternative interactive visitor experiences in museums and heritage sites. This article describes how the project brings together artists, engineers, and experts in multimedia advocacy, as well as people with learning disabilities in the co-design of interactive multisensory objects that replicate or respond to objects of cultural significance in our national collections. Through a series of staged multi-sensory art and electronics workshops, people with learning disabilities explore how the different senses could be utilised to augment existing artefacts or create entirely new ones. The co-researchers employ multimedia advocacy tools to reflect on and to communicate their experiences and findings.

Building an Interactive, Three-dimensional Virtual World

The movement of graphics and audio programming towards three dimensions is to better simulate the way we experience our world. In this project I looked to use methods for coming closer to such simulation via realistic graphics and sound combined with a natural interface. I did most of my work on a Dell OptiPlex with an 800 MHz Pentium III processor and an NVIDlA GeForce 256 AGP Plus graphics accelerator -high end products in the consumer market as of April 2000. For graphics, I used OpenGL [1], an open·source, multi-platform set of graphics libraries that is relatively easy to use, coded in C . The basic engine I first put together was a system to place objects in a scene and to navigate around the scene in real time. Once I accomplished this, I was able to investigate specific techniques for making parts of a scene more appealing.

Software Tools For Large Scale Interactive Hydrodynamic Modeling

Running hydrodynamic models interactively allows both visual exploration and change of model state during simulation. One of the main characteristics of an interactive model is that it should provide immediate feedback to the user, for example respond to changes in model state or view settings. For this reason, such features are usually only available for models with a relatively small number of computational cells, which are used mainly for demonstration and educational purposes. It would be useful if interactive modeling would also work for models typically used in consultancy projects involving large scale simulations. This results in a number of technical challenges related to the combination of the model itself and the visualisation tools (scalability, implementation of an appropriate API for control and access to the internal state). While model parallelisation is increasingly addressed by the environmental modeling community, little effort has been spent on developing a high-performance interactive environment. What can we learn from other high-end visualisation domains such as 3D animation, gaming, virtual globes (Autodesk 3ds Max, Second Life, Google Earth) that also focus on efficient interaction with 3D environments? In these domains high efficiency is usually achieved by the use of computer graphics algorithms such as surface simplification depending on current view, distance to objects, and efficient caching of the aggregated representation of object meshes. We investigate how these algorithms can be re-used in the context of interactive hydrodynamic modeling without significant changes to the model code and allowing model operation on both multi-core CPU personal computers and high-performance computer clusters.

Arquiteturas multi-tarefas simultâneas : SEMPRE : arquitetura SMT com capacidade de execução e escalonamento de processos

O avanço tecnológico no projeto de microprocessadores, nos recentes anos, tem seguido duas tendências principais. A primeira tenta aumentar a freqüência do relógio dos mesmos usando componentes digitais e técnicas VLSI mais eficientes. A segunda tenta explorar paralelismo no nível de instrução através da reorganização dos seus componentes internos. Dentro desta segunda abordagem estão as arquiteturas multi-tarefas simultâneas, que são capazes de extrair o paralelismo existente entre e dentro de diferentes tarefas das aplicações, executando instruções de vários fluxos simultaneamente e maximizando assim a utilização do hardware. Apesar do alto custo da implementação em hardware, acredita-se no potencial destas arquiteturas para o futuro próximo, pois é previsto que em breve haverá a disponibilidade de bilhões de transistores para o desenvolvimento de circuitos integrados. Assim, a questão principal a ser encarada talvez seja: como prover instruções paralelas para uma arquitetura deste tipo? Sabe-se que a maioria das aplicações é seqüencial pois os problemas nem sempre possuem uma solução paralela e quando a solução existe os programadores nem sempre têm habilidade para ver a solução paralela. Pensando nestas questões a arquitetura SEMPRE foi projetada. Esta arquitetura executa múltiplos processos, ao invés de múltiplas tarefas, aproveitando assim o paralelismo existente entre diferentes aplicações. Este paralelismo é mais expressivo do que aquele que existe entre tarefas dentro de uma mesma aplicação devido a não existência de sincronismo ou comunicação entre elas. Portanto, a arquitetura SEMPRE aproveita a grande quantidade de processos existentes nas estações de trabalho compartilhadas e servidores de rede. Além disso, esta arquitetura provê suporte de hardware para o escalonamento de processos e instruções especiais para o sistema operacional gerenciar processos com mínimo esforço. Assim, os tempos perdidos com o escalonamento de processos e as trocas de contextos são insignificantes nesta arquitetura, provendo ainda maior desempenho durante a execução das aplicações. Outra característica inovadora desta arquitetura é a existência de um mecanismo de prébusca de processos que, trabalhando em cooperação com o escalonamento de processos, permite reduzir faltas na cache de instruções. Também, devido a essa rápida troca de contexto, a arquitetura permite a definição de uma fatia de tempo (fatia de tempo) menor do que aquela praticada pelo sistema operacional, provendo maior dinâmica na execução das aplicações. A arquitetura SEMPRE foi analisada e avaliada usando modelagem analítica e simulação dirigida por execução de programas do SPEC95. A modelagem mostrou que o escalonamento por hardware reduz os efeitos colaterais causados pela presença de processos na cache de instruções e a simulação comprovou que as diferentes características desta arquitetura podem, juntas, prover ganho de desempenho razoável sobre outras arquiteturas multi-tarefas simultâneas equivalentes, com um pequeno acréscimo de hardware, melhor aproveitando as fatias de tempo atribuídas aos processos.

Remote presence: supporting deictic gestures through a handheld multi-touch device

This thesis argues on the possibility of supporting deictic gestures through handheld multi-touch devices in remote presentation scenarios. In [1], Clark distinguishes indicative techniques of placing-for and directing-to, where placing-for refers to placing a referent into the addressee’s attention, and directing-to refers to directing the addressee’s attention towards a referent. Keynote, PowerPoint, FuzeMeeting and others support placing-for efficiently with slide transitions, and animations, but support limited to none directing-to. The traditional “pointing feature” present in some presentation tools comes as a virtual laser pointer or mouse cursor. [12, 13] have shown that the mouse cursor and laser pointer offer very little informational expressiveness and do not do justice to human communicative gestures. In this project, a prototype application was implemented for the iPad in order to explore, develop, and test the concept of pointing in remote presentations. The prototype offers visualizing and navigating the slides as well as “pointing” and zooming. To further investigate the problem and possible solutions, a theoretical framework was designed representing the relationships between the presenter’s intention and gesture and the resulting visual effect (cursor) that enables the audience members to interpret the meaning of the effect and the presenter’s intention. Two studies were performed to investigate people’s appreciation of different ways of presenting remotely. An initial qualitative study was performed at The Hague, followed by an online quantitative user experiment. The results indicate that subjects found pointing to be helpful in understanding and concentrating, while the detached video feed of the presenter was considered to be distracting. The positive qualities of having the video feed were the emotion and social presence that it adds to the presentations. For a number of subjects, pointing displayed some of the same social and personal qualities [2] that video affords, while less intensified. The combination of pointing and video proved to be successful with 10-out-of-19 subjects scoring it the highest while pointing example came at a close 8-out-of-19. Video was the least preferred with only one subject preferring it. We suggest that the research performed here could provide a basis for future research and possibly be applied in a variety of distributed collaborative settings.

Automatic Orientation of Multi-Scale Terrestrial Images for 3D Reconstruction

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

User interaction widgets for interactive theorem proving

Matita (that means pencil in Italian) is a new interactive theorem prover under development at the University of Bologna. When compared with state-of-the-art proof assistants, Matita presents both traditional and innovative aspects. The underlying calculus of the system, namely the Calculus of (Co)Inductive Constructions (CIC for short), is well-known and is used as the basis of another mainstream proof assistant—Coq—with which Matita is to some extent compatible. In the same spirit of several other systems, proof authoring is conducted by the user as a goal directed proof search, using a script for storing textual commands for the system. In the tradition of LCF, the proof language of Matita is procedural and relies on tactic and tacticals to proceed toward proof completion. The interaction paradigm offered to the user is based on the script management technique at the basis of the popularity of the Proof General generic interface for interactive theorem provers: while editing a script the user can move forth the execution point to deliver commands to the system, or back to retract (or “undo”) past commands. Matita has been developed from scratch in the past 8 years by several members of the Helm research group, this thesis author is one of such members. Matita is now a full-fledged proof assistant with a library of about 1.000 concepts. Several innovative solutions spun-off from this development effort. This thesis is about the design and implementation of some of those solutions, in particular those relevant for the topic of user interaction with theorem provers, and of which this thesis author was a major contributor. Joint work with other members of the research group is pointed out where needed. The main topics discussed in this thesis are briefly summarized below. Disambiguation. Most activities connected with interactive proving require the user to input mathematical formulae. Being mathematical notation ambiguous, parsing formulae typeset as mathematicians like to write down on paper is a challenging task; a challenge neglected by several theorem provers which usually prefer to fix an unambiguous input syntax. Exploiting features of the underlying calculus, Matita offers an efficient disambiguation engine which permit to type formulae in the familiar mathematical notation. Step-by-step tacticals. Tacticals are higher-order constructs used in proof scripts to combine tactics together. With tacticals scripts can be made shorter, readable, and more resilient to changes. Unfortunately they are de facto incompatible with state-of-the-art user interfaces based on script management. Such interfaces indeed do not permit to position the execution point inside complex tacticals, thus introducing a trade-off between the usefulness of structuring scripts and a tedious big step execution behavior during script replaying. In Matita we break this trade-off with tinycals: an alternative to a subset of LCF tacticals which can be evaluated in a more fine-grained manner. Extensible yet meaningful notation. Proof assistant users often face the need of creating new mathematical notation in order to ease the use of new concepts. The framework used in Matita for dealing with extensible notation both accounts for high quality bidimensional rendering of formulae (with the expressivity of MathMLPresentation) and provides meaningful notation, where presentational fragments are kept synchronized with semantic representation of terms. Using our approach interoperability with other systems can be achieved at the content level, and direct manipulation of formulae acting on their rendered forms is possible too. Publish/subscribe hints. Automation plays an important role in interactive proving as users like to delegate tedious proving sub-tasks to decision procedures or external reasoners. Exploiting the Web-friendliness of Matita we experimented with a broker and a network of web services (called tutors) which can try independently to complete open sub-goals of a proof, currently being authored in Matita. The user receives hints from the tutors on how to complete sub-goals and can interactively or automatically apply them to the current proof. Another innovative aspect of Matita, only marginally touched by this thesis, is the embedded content-based search engine Whelp which is exploited to various ends, from automatic theorem proving to avoiding duplicate work for the user. We also discuss the (potential) reusability in other systems of the widgets presented in this thesis and how we envisage the evolution of user interfaces for interactive theorem provers in the Web 2.0 era.

Theoretical and implementation aspects in the mechanization of the metatheory of programming languages

Interactive theorem provers are tools designed for the certification of formal proofs developed by means of man-machine collaboration. Formal proofs obtained in this way cover a large variety of logical theories, ranging from the branches of mainstream mathematics, to the field of software verification. The border between these two worlds is marked by results in theoretical computer science and proofs related to the metatheory of programming languages. This last field, which is an obvious application of interactive theorem proving, poses nonetheless a serious challenge to the users of such tools, due both to the particularly structured way in which these proofs are constructed, and to difficulties related to the management of notions typical of programming languages like variable binding. This thesis is composed of two parts, discussing our experience in the development of the Matita interactive theorem prover and its use in the mechanization of the metatheory of programming languages. More specifically, part I covers: - the results of our effort in providing a better framework for the development of tactics for Matita, in order to make their implementation and debugging easier, also resulting in a much clearer code; - a discussion of the implementation of two tactics, providing infrastructure for the unification of constructor forms and the inversion of inductive predicates; we point out interactions between induction and inversion and provide an advancement over the state of the art. In the second part of the thesis, we focus on aspects related to the formalization of programming languages. We describe two works of ours: - a discussion of basic issues we encountered in our formalizations of part 1A of the Poplmark challenge, where we apply the extended inversion principles we implemented for Matita; - a formalization of an algebraic logical framework, posing more complex challenges, including multiple binding and a form of hereditary substitution; this work adopts, for the encoding of binding, an extension of Masahiko Sato's canonical locally named representation we designed during our visit to the Laboratory for Foundations of Computer Science at the University of Edinburgh, under the supervision of Randy Pollack.

Learning for sustainability - LforS: Simulation games - a creative tool for interactive learning

Description of simulation and training games as tool for awareness and capacity development in multi steakeholder processes

Lag Camera: A Moving Multi-Camera Array for Scene-Acquisition

Many applications, such as telepresence, virtual reality, and interactive walkthroughs, require a three-dimensional(3D)model of real-world environments. Methods, such as lightfields, geometric reconstruction and computer vision use cameras to acquire visual samples of the environment and construct a model. Unfortunately, obtaining models of real-world locations is a challenging task. In particular, important environments are often actively in use, containing moving objects, such as people entering and leaving the scene. The methods previously listed have difficulty in capturing the color and structure of the environment while in the presence of moving and temporary occluders. We describe a class of cameras called lag cameras. The main concept is to generalize a camera to take samples over space and time. Such a camera, can easily and interactively detect moving objects while continuously moving through the environment. Moreover, since both the lag camera and occluder are moving, the scene behind the occluder is captured by the lag camera even from viewpoints where the occluder lies in between the lag camera and the hidden scene. We demonstrate an implementation of a lag camera, complete with analysis and captured environments.

Interactive Augmentation of Live Images using a HDR Stereo Camera

Adding virtual objects to real environments plays an important role in todays computer graphics: Typical examples are virtual furniture in a real room and virtual characters in real movies. For a believable appearance, consistent lighting of the virtual objects is required. We present an augmented reality system that displays virtual objects with consistent illumination and shadows in the image of a simple webcam. We use two high dynamic range video cameras with fisheye lenses permanently recording the environment illumination. A sampling algorithm selects a few bright parts in one of the wide angle images and the corresponding points in the second camera image. The 3D position can then be calculated using epipolar geometry. Finally, the selected point lights are used in a multi pass algorithm to draw the virtual object with shadows. To validate our approach, we compare the appearance and shadows of the synthetic objects with real objects.

User experience with immersive and interactive media = Experiencia de usuario en entornos 3D inmersivos e interactivos

Estudios recientes promueven la integración de estímulos multisensoriales en activos multimedia con el fin de mejorar la experiencia de usuario mediante la estimulación de nuevos sentidos, más allá de la tradicional experiencia audiovisual. Del mismo modo, varios trabajos proponen la introducción de componentes de interacción capaces de complementar con nuevas características, funcionalidades y/o información la experiencia multimedia. Efectos sensoriales basados en el uso de nuevas técnicas de audio, olores, viento, vibraciones y control de la iluminación, han demostrado tener un impacto favorable en la sensación de Presencia, en el disfrute de la experiencia multimedia y en la calidad, relevancia y realismo de la misma percibidos por el usuario. Asimismo, los servicios basados en dos pantallas y la manipulación directa de (elementos en) la escena de video tienen el potencial de mejorar la comprensión, la concentración y la implicación proactiva del usuario en la experiencia multimedia. El deporte se encuentra entre los géneros con mayor potencial para integrar y explotar éstas soluciones tecnológicas. Trabajos previos han demostrado asimismo la viabilidad técnica de integrar éstas tecnologías con los estándares actualmente adoptados a lo largo de toda la cadena de transmisión de televisión. De este modo, los sistemas multimedia enriquecidos con efectos sensoriales, los servicios interactivos multiplataforma y un mayor control del usuario sobre la escena de vídeo emergen como nuevas formas de llevar la multimedia immersiva e interactiva al mercado de consumo de forma no disruptiva. Sin embargo, existen numerosas interrogantes relativas a los efectos sensoriales y/o soluciones interactivas más adecuadas para complementar un contenido audiovisual determinado o a la mejor manera de de integrar y combinar dichos componentes para mejorar la experiencia de usuario de un segmento de audiencia objetivo. Además, la evidencia científica sobre el impacto de factores humanos en la experiencia de usuario con estas nuevas formas de immersión e interacción en el contexto multimedia es aún insuficiente y en ocasiones, contradictoria. Así, el papel de éstos factores en el potencial de adopción de éstas tecnologías ha sido amplia-mente ignorado. La presente tesis analiza el impacto del audio binaural, efectos sensoriales (de iluminación y olfativos), interacción con objetos 3D integrados en la escena de vídeo e interacción con contenido adicional utilizando una segunda pantalla en la experiencia de usuario con contenidos de deporte. La posible influencia de dichos componentes en las variables dependientes se explora tanto a nivel global (efecto promedio) como en función de las características de los usuarios (efectos heterogéneos). Para ello, se ha llevado a cabo un experimento con usuarios orientado a explorar la influencia de éstos componentes immersivos e interactivos en dos grandes dimensiones de la experiencia multimedia: calidad y Presencia. La calidad de la experiencia multimedia se analiza en términos de las posibles variaciones asociadas a la calidad global y a la calidad del contenido, la imagen, el audio, los efectos sensoriales, la interacción con objetos 3D y la interacción con la segunda pantalla. El posible impacto en la Presencia considera dos de las dimensiones definidas por el cuestionario ITC-SOPI: Presencia Espacial (Spatial Presence) e Implicación (Engagement). Por último, los individuos son caracterizados teniendo en cuenta los siguientes atributos afectivos, cognitivos y conductuales: preferencias y hábitos en relación con el contenido, grado de conocimiento de las tecnologías integradas en el sistema, tendencia a involucrarse emocionalmente, tendencia a concentrarse en una actividad bloqueando estímulos externos y los cinco grandes rasgos de la personalidad: extroversión, amabilidad, responsabilidad, inestabilidad emocional y apertura a nuevas experiencias. A nivel global, nuestro estudio revela que los participantes prefieren el audio binaural frente al sistema estéreo y que los efectos sensoriales generan un aumento significativo del nivel de Presencia Espacial percibido por los usuarios. Además, las manipulaciones experimentales realizadas permitieron identificar una gran variedad de efectos heterogéneos. Un resultado interesante es que dichos efectos no se encuentran distribuidos de forma equitativa entre las medidas de calidad y Presencia. Nuestros datos revelan un impacto generalizado del audio binaural en la mayoría de las medidas de calidad y Presencia analizadas. En cambio, la influencia de los efectos sensoriales y de la interacción con la segunda pantalla se concentran en las medidas de Presencia y calidad, respectivamente. La magnitud de los efectos heterogéneos identificados está modulada por las siguientes características personales: preferencias en relación con el contenido, frecuencia con la que el usuario suele ver contenido similar, conocimiento de las tecnologías integradas en el demostrador, sexo, tendencia a involucrarse emocionalmente, tendencia a a concentrarse en una actividad bloqueando estímulos externos y niveles de amabilidad, responsabilidad y apertura a nuevas experiencias. Las características personales consideradas en nuestro experimento explicaron la mayor parte de la variación en las variables dependientes, confirmando así el importante (y frecuentemente ignorado) papel de las diferencias individuales en la experiencia multimedia. Entre las características de los usuarios con un impacto más generalizado se encuentran las preferencias en relación con el contenido, el grado de conocimiento de las tecnologías integradas en el sistema y la tendencia a involucrarse emocionalmente. En particular, los primeros dos factores parecen generar un conflicto de atención hacia el contenido versus las características/elementos técnicos del sistema, respectivamente. Asimismo, la experiencia multimedia de los fans del fútbol parece estar modulada por procesos emociona-les, mientras que para los no-fans predominan los procesos cognitivos, en particular aquellos directamente relacionados con la percepción de calidad. Abstract Recent studies encourage the integration of multi-sensorial stimuli into multimedia assets to enhance the user experience by stimulating other senses beyond sight and hearing. Similarly, the introduction of multi-modal interaction components complementing with new features, functionalities and/or information the multimedia experience is promoted. Sensory effects as odor, wind, vibration and light effects, as well as an enhanced audio quality, have been found to favour media enjoyment and to have a positive influence on the sense of Presence and on the perceived quality, relevance and reality of a multimedia experience. Two-screen services and a direct manipulation of (elements in) the video scene have the potential to enhance user comprehension, engagement and proactive involvement of/in the media experience. Sports is among the genres that could benefit the most from these solutions. Previous works have demonstrated the technical feasibility of implementing and deploying end-to-end solutions integrating these technologies into legacy systems. Thus, sensorially-enhanced media, two-screen services and an increased user control over the displayed scene emerge as means to deliver a new form of immersive and interactive media experiences to the mass market in a non-disruptive manner. However, many questions remain concerning issues as the specific interactive solutions or sensory effects that can better complement a given audiovisual content or the best way in which to integrate and combine them to enhance the user experience of a target audience segment. Furthermore, scientific evidence on the impact of human factors on the user experience with these new forms of immersive and interactive media is still insufficient and sometimes, contradictory. Thus, the role of these factors on the potential adoption of these technologies has been widely ignored. This thesis analyzes the impact of binaural audio, sensory (light and olfactory) effects, interaction with 3D objects integrated into the video scene and interaction with additional content using a second screen on the sports media experience. The potential influence of these components on the dependent variables is explored both at the overall level (average effect) and as a function of users’ characteristics (heterogeneous effects). To these aims, we conducted an experimental study exploring the influence of these immersive and interactive elements on the quality and Presence dimensions of the media experience. Along the quality dimension, we look for possible variations on the quality scores as-signed to the overall media experience and to the media components content, image, audio, sensory effects, interaction with 3D objects and interaction using the tablet device. The potential impact on Presence is analyzed by looking at two of the four dimensions defined by the ITC-SOPI questionnaire, namely Spatial Presence and Engagement. The users’ characteristics considered encompass the following personal affective, cognitive and behavioral attributes: preferences and habits in relation to the content, knowledge of the involved technologies, tendency to get emotionally involved and tendency to get absorbed in an activity and block out external distractors and the big five personality traits extraversion, agreeableness, conscientiousness, neuroticism and openness to experience. At the overall level, we found that participants preferred binaural audio than standard stereo audio and that sensory effects increase significantly the level of Spatial Presence. Several heterogeneous effects were also revealed as a result of our experimental manipulations. Interestingly, these effects were not equally distributed across the quality and Presence measures analyzed. Whereas binaural audio was foud to have an influence on the majority of the quality and Presence measures considered, the effects of sensory effects and of interaction with additional content through the tablet device concentrate mainly on the dimensions of Presence and on quality measures, respectively. The magnitude of these effects was modulated by individual’s characteristics, such as: preferences in relation to the content, frequency of viewing similar content, knowledge of involved technologies, gender, tendency to get emotionally involved, tendency to absorption and levels of agreeableness, conscientiousness and openness to experience. The personal characteristics collected in our experiment explained most of the variation in the dependent variables, confirming the frequently neglected role of individual differences on the media experience. Preferences in relation to the content, knowledge of involved technologies and tendency to get emotionally involved were among the user variables with the most generalized influence. In particular, the former two features seem to present a conflict in the allocation of attentional resources towards the media content versus the technical features of the system, respectively. Additionally, football fans’ experience seems to be modulated by emotional processes whereas for not fans, cognitive processes (and in particular those related to quality judgment) prevail.

Multi-function controllers for a hydraulic pump

This project has been undertaken for Hamworthy Hydraulics Limited. Its objective was to design and develop a controller package for a variable displacement, hydraulic pump for use mainly on mobile earth moving machinery. A survey was undertaken of control options used in practice and from this a design specification was formulated, the successful implementation of which would give Hamworthy an advantage over its competitors. Two different modes for the controller were envisaged. One consisted of using conventional hydro-mechanics and the other was based upon a microprocessor. To meet short term customer prototype requirements the first section of work was the realisation of the hydro-mechanical system. Mathematical models were made to evaluate controller stability and hence aid their design. The final package met the requirements of the specification and a single version could operate all sizes of variable displacement pumps in the Hamworthy range. The choice of controller options and combinations totalled twenty-four. The hydro-mechanical controller was complex and it was realised that a micro-processor system would allow all options to be implemented with just one design of hardware, thus greatly simplifying production. The final section of this project was to determine whether such a design was feasible. This entailed finding cheap, reliable transducers, using mathematical models to predict electro-hydraulic interface stability, testing such interfaces and finally incorporating a micro-processor in an interactive control loop. The study revealed that such a system was technically possible but it would cost 60% more than its hydro-mechanical counterpart. It was therefore concluded that, in the short term, for the markets considered, the hydro-mechanical design was the better solution. Regarding the micro-processor system the final conclusion was that, because the relative costs of the two systems are decreasing, the electro-hydraulic controller will gradually become more attractive and therefore Hamworthy should continue with its development.

Interactive semantics

Much research pursues machine intelligence through better representation of semantics. What is semantics? People in different areas view semantics from different facets although it accompanies interaction through civilization. Some researchers believe that humans have some innate structure in mind for processing semantics. Then, what the structure is like? Some argue that humans evolve a structure for processing semantics through constant learning. Then, how the process is like? Humans have invented various symbol systems to represent semantics. Can semantics be accurately represented? Turing machines are good at processing symbols according to algorithms designed by humans, but they are limited in ability to process semantics and to do active interaction. Super computers and high-speed networks do not help solve this issue as they do not have any semantic worldview and cannot reflect themselves. Can future cyber-society have some semantic images that enable machines and individuals (humans and agents) to reflect themselves and interact with each other with knowing social situation through time? This paper concerns these issues in the context of studying an interactive semantics for the future cyber-society. It firstly distinguishes social semantics from natural semantics, and then explores the interactive semantics in the category of social semantics. Interactive semantics consists of an interactive system and its semantic image, which co-evolve and influence each other. The semantic worldview and interactive semantic base are proposed as the semantic basis of interaction. The process of building and explaining semantic image can be based on an evolving structure incorporating adaptive multi-dimensional classification space and self-organized semantic link network. A semantic lens is proposed to enhance the potential of the structure and help individuals build and retrieve semantic images from different facets, abstraction levels and scales through time.

Video-based, real-time multi-view stereo

We investigate the problem of obtaining a dense reconstruction in real-time, from a live video stream. In recent years, multi-view stereo (MVS) has received considerable attention and a number of methods have been proposed. However, most methods operate under the assumption of a relatively sparse set of still images as input and unlimited computation time. Video based MVS has received less attention despite the fact that video sequences offer significant benefits in terms of usability of MVS systems. In this paper we propose a novel video based MVS algorithm that is suitable for real-time, interactive 3d modeling with a hand-held camera. The key idea is a per-pixel, probabilistic depth estimation scheme that updates posterior depth distributions with every new frame. The current implementation is capable of updating 15 million distributions/s. We evaluate the proposed method against the state-of-the-art real-time MVS method and show improvement in terms of accuracy. © 2011 Elsevier B.V. All rights reserved.