Automatic 3DTV Quality Assessment Based On Depth Perception Analysis

Quality assessment is a key factor for stereoscopic 3D video content as some observers are affected by visual discomfort in the eye when viewing 3D video, especially when combining positive and negative parallax with fast motion. In this paper, we propose techniques to assess objective quality related to motion and depth maps, which facilitate depth perception analysis. Subjective tests were carried out in order to understand the source of the problem. Motion is an important feature affecting 3D experience but also often the cause of visual discomfort. The automatic algorithm developed tries to quantify the impact on viewer experience when common cases of discomfort occur, such as high-motion sequences, scene changes with abrupt parallax changes, or complete absence of stereoscopy, with a goal of preventing the viewer from having a bad stereoscopic experience.

Visual attention and perception models for assessing quality in 2D and 3D stereoscopic video

La medida de calidad de vídeo sigue siendo necesaria para definir los criterios que caracterizan una señal que cumpla los requisitos de visionado impuestos por el usuario. Las nuevas tecnologías, como el vídeo 3D estereoscópico o formatos más allá de la alta definición, imponen nuevos criterios que deben ser analizadas para obtener la mayor satisfacción posible del usuario. Entre los problemas detectados durante el desarrollo de esta tesis doctoral se han determinado fenómenos que afectan a distintas fases de la cadena de producción audiovisual y tipo de contenido variado. En primer lugar, el proceso de generación de contenidos debe encontrarse controlado mediante parámetros que eviten que se produzca el disconfort visual y, consecuentemente, fatiga visual, especialmente en lo relativo a contenidos de 3D estereoscópico, tanto de animación como de acción real. Por otro lado, la medida de calidad relativa a la fase de compresión de vídeo emplea métricas que en ocasiones no se encuentran adaptadas a la percepción del usuario. El empleo de modelos psicovisuales y diagramas de atención visual permitirían ponderar las áreas de la imagen de manera que se preste mayor importancia a los píxeles que el usuario enfocará con mayor probabilidad. Estos dos bloques se relacionan a través de la definición del término saliencia. Saliencia es la capacidad del sistema visual para caracterizar una imagen visualizada ponderando las áreas que más atractivas resultan al ojo humano. La saliencia en generación de contenidos estereoscópicos se refiere principalmente a la profundidad simulada mediante la ilusión óptica, medida en términos de distancia del objeto virtual al ojo humano. Sin embargo, en vídeo bidimensional, la saliencia no se basa en la profundidad, sino en otros elementos adicionales, como el movimiento, el nivel de detalle, la posición de los píxeles o la aparición de caras, que serán los factores básicos que compondrán el modelo de atención visual desarrollado. Con el objetivo de detectar las características de una secuencia de vídeo estereoscópico que, con mayor probabilidad, pueden generar disconfort visual, se consultó la extensa literatura relativa a este tema y se realizaron unas pruebas subjetivas preliminares con usuarios. De esta forma, se llegó a la conclusión de que se producía disconfort en los casos en que se producía un cambio abrupto en la distribución de profundidades simuladas de la imagen, aparte de otras degradaciones como la denominada “violación de ventana”. A través de nuevas pruebas subjetivas centradas en analizar estos efectos con diferentes distribuciones de profundidades, se trataron de concretar los parámetros que definían esta imagen. Los resultados de las pruebas demuestran que los cambios abruptos en imágenes se producen en entornos con movimientos y disparidades negativas elevadas que producen interferencias en los procesos de acomodación y vergencia del ojo humano, así como una necesidad en el aumento de los tiempos de enfoque del cristalino. En la mejora de las métricas de calidad a través de modelos que se adaptan al sistema visual humano, se realizaron también pruebas subjetivas que ayudaron a determinar la importancia de cada uno de los factores a la hora de enmascarar una determinada degradación. Los resultados demuestran una ligera mejora en los resultados obtenidos al aplicar máscaras de ponderación y atención visual, los cuales aproximan los parámetros de calidad objetiva a la respuesta del ojo humano. ABSTRACT Video quality assessment is still a necessary tool for defining the criteria to characterize a signal with the viewing requirements imposed by the final user. New technologies, such as 3D stereoscopic video and formats of HD and beyond HD oblige to develop new analysis of video features for obtaining the highest user’s satisfaction. Among the problems detected during the process of this doctoral thesis, it has been determined that some phenomena affect to different phases in the audiovisual production chain, apart from the type of content. On first instance, the generation of contents process should be enough controlled through parameters that avoid the occurrence of visual discomfort in observer’s eye, and consequently, visual fatigue. It is especially necessary controlling sequences of stereoscopic 3D, with both animation and live-action contents. On the other hand, video quality assessment, related to compression processes, should be improved because some objective metrics are adapted to user’s perception. The use of psychovisual models and visual attention diagrams allow the weighting of image regions of interest, giving more importance to the areas which the user will focus most probably. These two work fields are related together through the definition of the term saliency. Saliency is the capacity of human visual system for characterizing an image, highlighting the areas which result more attractive to the human eye. Saliency in generation of 3DTV contents refers mainly to the simulated depth of the optic illusion, i.e. the distance from the virtual object to the human eye. On the other hand, saliency is not based on virtual depth, but on other features, such as motion, level of detail, position of pixels in the frame or face detection, which are the basic features that are part of the developed visual attention model, as demonstrated with tests. Extensive literature involving visual comfort assessment was looked up, and the development of new preliminary subjective assessment with users was performed, in order to detect the features that increase the probability of discomfort to occur. With this methodology, the conclusions drawn confirmed that one common source of visual discomfort was when an abrupt change of disparity happened in video transitions, apart from other degradations, such as window violation. New quality assessment was performed to quantify the distribution of disparities over different sequences. The results confirmed that abrupt changes in negative parallax environment produce accommodation-vergence mismatches derived from the increasing time for human crystalline to focus the virtual objects. On the other side, for developing metrics that adapt to human visual system, additional subjective tests were developed to determine the importance of each factor, which masks a concrete distortion. Results demonstrated slight improvement after applying visual attention to objective metrics. This process of weighing pixels approximates the quality results to human eye’s response.

Perceptual motion standstill in rapidly moving chromatic displays

In motion standstill, a quickly moving object appears to stand still, and its details are clearly visible. It is proposed that motion standstill can occur when the spatiotemporal resolution of the shape and color systems exceeds that of the motion systems. For moving red-green gratings, the first- and second-order motion systems fail when the grating is isoluminant. The third-order motion system fails when the green/red saturation ratio produces isosalience (equal distinctiveness of red and green). When a variety of high-contrast red-green gratings, with different spatial frequencies and speeds, were made isoluminant and isosalient, the perception of motion standstill was so complete that motion direction judgments were at chance levels. Speed ratings also indicated that, within a narrow range of luminance contrasts and green/red saturation ratios, moving stimuli were perceived as absolutely motionless. The results provide further evidence that isoluminant color motion is perceived only by the third-order motion system, and they have profound implications for the nature of shape and color perception.

The effect of anticoagulation on the restoration of range of motion after total knee arthroplasty: Enoxaparin versus aspirin

Anticoagulation used for thromboembolic prophylaxis following total knee arthroplasty (TKA) could interfere with movement. This study compares the effect of 2 anticoagulants, enoxaparin and aspirin, on restoration of range of motion (ROM) after TKA. Two groups of 75 consecutive patients, matched for age, arthritic severity, and preoperative ROM, underwent TKA. Flexion and extension milestone measures were recorded daily. Results show a highly statistically significant difference (P

Plaid motion rivalry: Correlates with binocular rivalry and positive mood state

Recently Hupe and Rubin (2003, Vision Research 43 531 - 548) re-introduced the plaid as a form of perceptual rivalry by using two sets of drifting gratings behind a circular aperture to produce quasi-regular perceptual alternations between a coherent moving plaid of diamond-shaped intersections and the two sets of component 'sliding' gratings. We call this phenomenon plaid motion rivalry (PMR), and have compared its temporal dynamics with those of binocular rivalry in a sample of subjects covering a wide range of perceptual alternation rates. In support of the proposal that all rivalries may be mediated by a common switching mechanism, we found a high correlation between alternation rates induced by PMR and binocular rivalry. In keeping with a link discovered between the phase of rivalry and mood, we also found a link between PMR and an individual's mood state that is consistent with suggestions that each opposing phase of rivalry is associated with one or the other hemisphere, with the 'diamonds' phase of PMR linked with the 'positive' left hemisphere.

Dynamic facial expressions evoke distinct activation in the face perception network:a connectivity analysis study

Very little is known about the neural structures involved in the perception of realistic dynamic facial expressions. In the present study, a unique set of naturalistic dynamic facial emotional expressions was created. Through fMRI and connectivity analysis, a dynamic face perception network was identified, which is demonstrated to extend Haxby et al.'s [Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. The distributed human neural system for face perception. Trends in Cognitive Science, 4, 223–233, 2000] distributed neural system for face perception. This network includes early visual regions, such as the inferior occipital gyrus, which is identified as insensitive to motion or affect but sensitive to the visual stimulus, the STS, identified as specifically sensitive to motion, and the amygdala, recruited to process affect. Measures of effective connectivity between these regions revealed that dynamic facial stimuli were associated with specific increases in connectivity between early visual regions, such as the inferior occipital gyrus and the STS, along with coupling between the STS and the amygdala, as well as the inferior frontal gyrus. These findings support the presence of a distributed network of cortical regions that mediate the perception of different dynamic facial expressions.

Motion, flash, and flicker:A unified spatiotemporal model of perceived edge sharpening

Blurred edges appear sharper in motion than when they are stationary. We proposed a model of this motion sharpening that invokes a local, nonlinear contrast transducer function (Hammett et al, 1998 Vision Research 38 2099-2108). Response saturation in the transducer compresses or 'clips' the input spatial waveform, rendering the edges as sharper. To explain the increasing distortion of drifting edges at higher speeds, the degree of nonlinearity must increase with speed or temporal frequency. A dynamic contrast gain control before the transducer can account for both the speed dependence and approximate contrast invariance of motion sharpening (Hammett et al, 2003 Vision Research, in press). We show here that this model also predicts perceived sharpening of briefly flashed and flickering edges, and we show that the model can account fairly well for experimental data from all three modes of presentation (motion, flash, and flicker). At moderate durations and lower temporal frequencies the gain control attenuates the input signal, thus protecting it from later compression by the transducer. The gain control is somewhat sluggish, and so it suffers both a slow onset, and loss of power at high temporal frequencies. Consequently, brief presentations and high temporal frequencies of drift and flicker are less protected from distortion, and show greater perceptual sharpening.

Contextual effects in speed perception may occur at an early stage of processing

How does nearby motion affect the perceived speed of a target region? When a central drifting Gabor patch is surrounded by translating noise, its speed can be misperceived over a fourfold range. Typically, when a surround moves in the same direction, perceived centre speed is reduced; for opposite-direction surrounds it increases. Measuring this illusion for a variety of surround properties reveals that the motion context effects are a saturating function of surround speed (Experiment I) and contrast (Experiment II). Our analyses indicate that the effects are consistent with a subtractive process, rather than with speed being averaged over area. In Experiment III we exploit known properties of the motion system to ask where these surround effects impact. Using 2D plaid stimuli, we find that surround-induced shifts in perceived speed of one plaid component produce substantial shifts in perceived plaid direction. This indicates that surrounds exert their influence early in processing, before pattern motion direction is computed. These findings relate to ongoing investigations of surround suppression for direction discrimination, and are consistent with single-cell findings of direction-tuned suppressive and facilitatory interactions in primary visual cortex (V1).

A multimodal investigation of dynamic face perception using functional magnetic resonance imaging and magnetoencephalography

Motion is an important aspect of face perception that has been largely neglected to date. Many of the established findings are based on studies that use static facial images, which do not reflect the unique temporal dynamics available from seeing a moving face. In the present thesis a set of naturalistic dynamic facial emotional expressions was purposely created and used to investigate the neural structures involved in the perception of dynamic facial expressions of emotion, with both functional Magnetic Resonance Imaging (fMRI) and Magnetoencephalography (MEG). Through fMRI and connectivity analysis, a dynamic face perception network was identified, which is demonstrated to extend the distributed neural system for face perception (Haxby et al.,2000). Measures of effective connectivity between these regions revealed that dynamic facial stimuli were associated with specific increases in connectivity between early visual regions, such as inferior occipital gyri and superior temporal sulci, along with coupling between superior temporal sulci and amygdalae, as well as with inferior frontal gyri. MEG and Synthetic Aperture Magnetometry (SAM) were used to examine the spatiotemporal profile of neurophysiological activity within this dynamic face perception network. SAM analysis revealed a number of regions showing differential activation to dynamic versus static faces in the distributed face network, characterised by decreases in cortical oscillatory power in the beta band, which were spatially coincident with those regions that were previously identified with fMRI. These findings support the presence of a distributed network of cortical regions that mediate the perception of dynamic facial expressions, with the fMRI data providing information on the spatial co-ordinates paralleled by the MEG data, which indicate the temporal dynamics within this network. This integrated multimodal approach offers both excellent spatial and temporal resolution, thereby providing an opportunity to explore dynamic brain activity and connectivity during face processing.

A psychophysical assessment of visual motion processing among high-functioning persons with autism

Thèse numérisée par la Direction des bibliothèques de l'Université de Montréal.

A psychophysical assessment of visual motion processing among high-functioning persons with autism

Thèse numérisée par la Direction des bibliothèques de l'Université de Montréal.

The interplay between social motivation, social experience, and developmental neural specialization for social perception

From birth, infants preferentially attend to human motion, which allows them to learn to interpret other peoples’ facial expressions and mental states. Evidence from adults shows that selectivity of the amygdala and the posterior superior temporal sulcus (pSTS) to biological motion correlates with social network size. Social motivation—one’s desire to orient to the social world, to seek and find reward in social interaction, and to maintain social relationships—may also contribute to neural specialization for biological motion and to social network characteristics. The current study aimed to determine whether neural selectivity for biological motion relates to social network characteristics, and to gain preliminary evidence as to whether social motivation plays a role in this relation. Findings suggest that neural selectivity for biological motion in the pSTS is positively related to social network size in middle childhood and that this relation is moderated by social motivation.

A single theoretical framework for circular features processing in humans: orientation and direction of motion compared

Common computational principles underlie processing of various visual features in the cortex. They are considered to create similar patterns of contextual modulations in behavioral studies for different features as orientation and direction of motion. Here, I studied the possibility that a single theoretical framework, implemented in different visual areas, of circular feature coding and processing could explain these similarities in observations. Stimuli were created that allowed direct comparison of the contextual effects on orientation and motion direction with two different psychophysical probes: changes in weak and strong signal perception. One unique simplified theoretical model of circular feature coding including only inhibitory interactions, and decoding through standard vector average, successfully predicted the similarities in the two domains, while different feature population characteristics explained well the differences in modulation on both experimental probes. These results demonstrate how a single computational principle underlies processing of various features across the cortices.

A New Sense of Motion: Framing the Idea of the Archive in the New Information Order

In evolution: that there are a myriad of ways a diversity of folks can archive digital documents (perhaps as print).Documents can only move through containers, and those containers rely on perception to be used.Further, the decision of what constitutes an archive is up to the archon.

Perception Of Oral And Maxillofacial Surgeons, Orthodontists And Laypersons In Relation To The Harmony Of The Smile.

Harmony is one of the main objectives in surgical and orthodontic treatment and this harmony must be present in the smile, as well as in the face. The aim of the present study was to assess the perceptions of professionals and laypersons in relation to the harmony of the smile of patients with or without vertical maxillary alterations. Sixty observers (oral and maxillofacial surgeons, orthodontists and laypersons) reported the degree of harmony of six smiles using an objective questionnaire and the participants indicated if there was a need for corrective surgery or not. The classification of observers was recorded on a Likert scale from 1 to 5. Mixed regression was used to determine differences between the three groups. Statistically significant differences were found only for the harmony of the smile between the oral and maxillofacial surgeons and laypersons, with laypersons being more critical when assessing the smile. There was no statistical difference between the other groups for the harmony of the smile or the indication of corrective surgery. The patterns of greater or lesser harmony determined by observers during the smile were similar to those found in the literature as the ideal standard in relation to vertical maxillary positioning. Laypersons had a tendency to be more critical in relation to facial harmony than surgeons, although no statistical differences were found in the other groups in relation to the harmony of the smile or indication for the corrective surgery. In addition, the patterns of greater or lesser harmony of the smile determined by the participants were similar to those found in the literature as the ideal standard in relation to vertical maxillary positioning. Overall, the present study demonstrates that adequate interaction between surgeons, orthodontists and laypersons is essential in order to achieve facial harmony with orthodontic and/or surgical treatment. Opinion of specialists and laypersons about the smile in relation to the vertical positioning of the maxilla.