Applying semiotic analysis to the design and modeling of distributed multimedia systems

Semiotics is the study of signs. Application of semiotics in information systems design is based on the notion that information systems are organizations within which agents deploy signs in the form of actions according to a set of norms. An analysis of the relationships among the agents, their actions and the norms would give a better specification of the system. Distributed multimedia systems (DMMS) could be viewed as a system consisted of many dynamic, self-controlled normative agents engaging in complex interaction and processing of multimedia information. This paper reports the work of applying the semiotic approach to the design and modeling of DMMS, with emphasis on using semantic analysis under the semiotic framework. A semantic model of DMMS describing various components and their ontological dependencies is presented, which then serves as a design model and implemented in a semantic database. Benefits of using the semantic database are discussed with reference to various design scenarios.

Coming of age: a review of embodiment and the neuroscience of semantics

Over the last decade, there has been an increasing body of work that explores whether sensory and motor information is a necessary part of semantic representation and processing. This is the embodiment hypothesis. This paper presents a theoretical review of this work that is intended to be useful for researchers in the neurosciences and neuropsychology. Beginning with a historical perspective, relevant theories are placed on a continuum from strongly embodied to completely unembodied representations. Predictions are derived and neuroscientific and neuropsychological evidence that could support different theories is reviewed; finally, criticisms of embodiment are discussed. We conclude that strongly embodied and completely disembodied theories are not supported, and that the remaining theories agree that semantic representation involves some form of Convergence Zones (Damasio, 1989) and the activation of modal content. For the future, research must carefully define the boundaries of semantic processing and tackle the representation of abstract entities.

Modeling building semantics: providing feedback and sustainability

Even minor changes in user activity can bring about significant energy savings within built space. Many building performance assessment methods have been developed, however these often disregard the impact of user behavior (i.e. the social, cultural and organizational aspects of the building). Building users currently have limited means of determining how sustainable they are, in context of the specific building structure and/or when compared to other users performing similar activities, it is therefore easy for users to dismiss their energy use. To support sustainability, buildings must be able to monitor energy use, identify areas of potential change in the context of user activity and provide contextually relevant information to facilitate persuasion management. If the building is able to provide users with detailed information about how specific user activity that is wasteful, this should provide considerable motivation to implement positive change. This paper proposes using a dynamic and temporal semantic model, to populate information within a model of persuasion, to manage user change. By semantically mapping a building, and linking this to persuasion management we suggest that: i) building energy use can be monitored and analyzed over time; ii) persuasive management can be facilitated to move user activity towards sustainability.

Distributed systems and multimedia

Edited book to support level 3 undergraduate understanding.

Defining user perception of distributed multimedia quality

This article presents the results of a study that explored the human side of the multimedia experience. We propose a model that assesses quality variation from three distinct levels: the network, the media and the content levels; and from two views: the technical and the user perspective. By facilitating parameter variation at each of the quality levels and from each of the perspectives, we were able to examine their impact on user quality perception. Results show that a significant reduction in frame rate does not proportionally reduce the user's understanding of the presentation independent of technical parameters, that multimedia content type significantly impacts user information assimilation, user level of enjoyment, and user perception of quality, and that the device display type impacts user information assimilation and user perception of quality. Finally, to ensure the transfer of information, low-level abstraction (network-level) parameters, such as delay and jitter, should be adapted; to maintain the user's level of enjoyment, high-level abstraction quality parameters (content-level), such as the appropriate use of display screens, should be adapted.

Distributed multimedia quality: the user perspective

Distributed multimedia supports a symbiotic infotainment duality, i.e. the ability to transfer information to the user, yet also provide the user with a level of satisfaction. As multimedia is ultimately produced for the education and / or enjoyment of viewers, the user’s-perspective concerning the presentation quality is surely of equal importance as objective Quality of Service (QoS) technical parameters, to defining distributed multimedia quality. In order to extensively measure the user-perspective of multimedia video quality, we introduce an extended model of distributed multimedia quality that segregates quality into three discrete levels: the network-level, the media-level and content-level, using two distinct quality perspectives: the user-perspective and the technical-perspective. Since experimental questionnaires do not provide continuous monitoring of user attention, eye tracking was used in our study in order to provide a better understanding of the role that the human element plays in the reception, analysis and synthesis of multimedia data. Results showed that video content adaptation, results in disparity in user video eye-paths when: i) no single / obvious point of focus exists; or ii) when the point of attention changes dramatically. Accordingly, appropriate technical- and user-perspective parameter adaptation is implemented, for all quality abstractions of our model, i.e. network-level (via simulated delay and jitter), media-level (via a technical- and user-perspective manipulated region-of-interest attentive display) and content-level (via display-type and video clip-type). Our work has shown that user perception of distributed multimedia quality cannot be achieved by means of purely technical-perspective QoS parameter adaptation.