Improved matching criterion for frame rate upconversion with trilateral filtering

Frame rate upconversion (FRUC) is an important post-processing technique to enhance the visual quality of low frame rate video. A major, recent advance in this area is FRUC based on trilateral filtering which novelty mainly derives from the combination of an edge-based motion estimation block matching criterion with the trilateral filter. However, there is still room for improvement, notably towards reducing the size of the uncovered regions in the initial estimated frame, this means the estimated frame before trilateral filtering. In this context, proposed is an improved motion estimation block matching criterion where a combined luminance and edge error metric is weighted according to the motion vector components, notably to regularise the motion field. Experimental results confirm that significant improvements are achieved for the final interpolated frames, reaching PSNR gains up to 2.73 dB, on average, regarding recent alternative solutions, for video content with varied motion characteristics.

VSImG: A high frame rate bitmap based display system for neuroscience research

This paper describes a visual stimulus generator (VSImG) capable of displaying a gray-scale, 256 x 256 x 8 bitmap image with a frame rate of 500 Hz using a boustrophedonic scanning technique. It is designed for experiments with motion-sensitive neurons of the fly`s visual system, where the flicker fusion frequency of the photoreceptors can reach up to 500 Hz. Devices with such a high frame rate are not commercially available, but are required, if sensory systems with high flicker fusion frequency are to be studied. The implemented hardware approach gives us complete real-time control of the displacement sequence and provides all the signals needed to drive an electrostatic deflection display. With the use of analog signals, very small high-resolution displacements, not limited by the image`s pixel size can be obtained. Very slow image displacements with visually imperceptible steps can also be generated. This can be of interest for other vision research experiments. Two different stimulus files can be used simultaneously, allowing the system to generate X-Y displacements on one display or independent movements on two displays as long as they share the same bitmap image. (C) 2011 Elsevier B.V. All rights reserved.

Vertical CNT-Si Photodiode Array

Agências financiadoras: National Natural Science Foundation of China - 61204077; Shenzhen Science and Technology Innovation Commission - JCYJ20120614150521967

Workload balancing in distributed virtual reality environments

Virtual Reality (VR) has grown to become state-of-theart technology in many business- and consumer oriented E-Commerce applications. One of the major design challenges of VR environments is the placement of the rendering process. The rendering process converts the abstract description of a scene as contained in an object database to an image. This process is usually done at the client side like in VRML [1] a technology that requires the client’s computational power for smooth rendering. The vision of VR is also strongly connected to the issue of Quality of Service (QoS) as the perceived realism is subject to an interactive frame rate ranging from 10 to 30 frames-per-second (fps), real-time feedback mechanisms and realistic image quality. These requirements overwhelm traditional home computers or even high sophisticated graphical workstations over their limits. Our work therefore introduces an approach for a distributed rendering architecture that gracefully balances the workload between the client and a clusterbased server. We believe that a distributed rendering approach as described in this paper has three major benefits: It reduces the clients workload, it decreases the network traffic and it allows to re-use already rendered scenes.

Development of model observers applied to 3D breast tomosynthesis microcalcifications and masses

The development of model observers for mimicking human detection strategies has followed from symmetric signals in simple noise to increasingly complex backgrounds. In this study we implement different model observers for the complex task of detecting a signal in a 3D image stack. The backgrounds come from real breast tomosynthesis acquisitions and the signals were simulated and reconstructed within the volume. Two different tasks relevant to the early detection of breast cancer were considered: detecting an 8 mm mass and detecting a cluster of microcalcifications. The model observers were calculated using a channelized Hotelling observer (CHO) with dense difference-of-Gaussian channels, and a modified (Partial prewhitening [PPW]) observer which was adapted to realistic signals which are not circularly symmetric. The sustained temporal sensitivity function was used to filter the images before applying the spatial templates. For a frame rate of five frames per second, the only CHO that we calculated performed worse than the humans in a 4-AFC experiment. The other observers were variations of PPW and outperformed human observers in every single case. This initial frame rate was a rather low speed and the temporal filtering did not affect the results compared to a data set with no human temporal effects taken into account. We subsequently investigated two higher speeds at 5, 15 and 30 frames per second. We observed that for large masses, the two types of model observers investigated outperformed the human observers and would be suitable with the appropriate addition of internal noise. However, for microcalcifications both only the PPW observer consistently outperformed the humans. The study demonstrated the possibility of using a model observer which takes into account the temporal effects of scrolling through an image stack while being able to effectively detect a range of mass sizes and distributions.

Wall motion estimation in intracranial aneurysms

The quantification of wall motion in cerebral aneurysms is becoming important owing to its potential connection to rupture, and as a way to incorporate the effects of vascular compliance in computational fluid dynamics (CFD) simulations.Most of papers report values obtained with experimental phantoms, simulated images, or animal models, but the information for real patients is limited. In this paper, we have combined non-rigid registration (IR) with signal processing techniques to measure pulsation in real patients from high frame rate digital subtraction angiography (DSA). We have obtained physiological meaningful waveforms with amplitudes in therange 0mm-0.3mm for a population of 18 patients including ruptured and unruptured aneurysms. Statistically significant differences in pulsation were found according to the rupture status, in agreement with differences in biomechanical properties reported in the literature.

Trinocular System for 3D motion and Dense Stucture Estimation

The relief of the seafloor is an important source of data for many scientists. In this paper we present an optical system to deal with underwater 3D reconstruction. This system is formed by three cameras that take images synchronously in a constant frame rate scheme. We use the images taken by these cameras to compute dense 3D reconstructions. We use Bundle Adjustment to estimate the motion ofthe trinocular rig. Given the path followed by the system, we get a dense map of the observed scene by registering the different dense local reconstructions in a unique and bigger one

Real-time Imaging and Mosaicking of Planar Surfaces

Laajojen pintojen kuvaaminen rajoitetussa työskentelytilassa riittävällä kuvatarkkuudella voi olla vaikeaa. Kuvaaminen on suoritettava osissa ja osat koottava saumattomaksi kokonaisnäkymäksi eli mosaiikkikuvaksi. Kuvauslaitetta käsin siirtelevän käyttäjän on saatava välitöntä palautetta, jotta mosaiikkiin ei jäisi aukkoja ja työ olisi nopeaa. Työn tarkoituksena oli rakentaa pieni, kannettava ja tarkka kuvauslaite paperi- ja painoteollisuuden tarpeisiin sekä kehittää palautteen antamiseen menetelmä, joka koostaaja esittää karkeaa mosaiikkikuvaa tosiajassa. Työssä rakennettiin kaksi kuvauslaitetta: ensimmäinen kuluttajille ja toinen teollisuuteen tarkoitetuista osista. Kuvamateriaali käsiteltiin tavallisella pöytätietokoneella. Videokuvien välinen liike laskettiin yksinkertaisella seurantamenetelmällä ja mosaiikkikuvaa koottiin kameroiden kuvanopeudella. Laskennallista valaistuksenkorjausta tutkittiin ja kehitetty menetelmä otettiin käyttöön. Ensimmäisessä kuvauslaitteessa on ongelmia valaistuksen ja linssivääristymien kanssa tuottaen huonolaatuisia mosaiikkikuvia. Toisessa kuvauslaitteessa nämä ongelmat on korjattu. Seurantamenetelmä toimii hyvin ottaen huomioon sen yksinkertaisuuden ja siihen ehdotetaan monia parannuksia. Työn tulokset osoittavat, että tosiaikainen mosaiikkikuvan koostaminen megapikselin kuvamateriaalista on mahdollista kuluttajille tarkoitetulla tietokonelaitteistolla.

Extension of ultrasound Fourier slice Imaging theory to sectorial acquisition

Ultrasound image reconstruction from the echoes received by an ultrasound probe after the transmission of diverging waves is an active area of research because of its capacity to insonify at ultra-high frame rate with large regions of interest using small phased arrays as the ones used in echocardiography. Current state-of-the-art techniques are based on the emission of diverging waves and the use of delay and sum strategies applied on the received signals to reconstruct the desired image (DW/DAS). Recently, we have introduced the concept of Ultrasound Fourier Slice Imaging (UFSI) theory for the reconstruction of ultrafast imaging for linear acquisition. In this study, we extend this theory to sectorial acquisition thanks to the introduction of an explicit and invertible spatial transform. Starting from a diverging wave, we show that the direct use of UFSI theory along with the application of the proposed spatial transform allows reconstructing the insonified medium in the conventional Cartesian space. Simulations and experiments reveal the capacity of this new approach in obtaining competitive quality of ultrafast imaging when compared with the current reference method.

Vector flow mapping using plane wave ultrasound imaging

Les diagnostics cliniques des maladies cardio-vasculaires sont principalement effectués à l’aide d’échographies Doppler-couleur malgré ses restrictions : mesures de vélocité dépendantes de l’angle ainsi qu’une fréquence d’images plus faible à cause de focalisation traditionnelle. Deux études, utilisant des approches différentes, adressent ces restrictions en utilisant l’imagerie à onde-plane, post-traitée avec des méthodes de délai et sommation et d’autocorrélation. L’objectif de la présente étude est de ré-implémenté ces méthodes pour analyser certains paramètres qui affecte la précision des estimations de la vélocité du flux sanguin en utilisant le Doppler vectoriel 2D. À l’aide d’expériences in vitro sur des flux paraboliques stationnaires effectuées avec un système Verasonics, l’impact de quatre paramètres sur la précision de la cartographie a été évalué : le nombre d’inclinaisons par orientation, la longueur d’ensemble pour les images à orientation unique, le nombre de cycles par pulsation, ainsi que l’angle de l’orientation pour différents flux. Les valeurs optimales sont de 7 inclinaisons par orientation, une orientation de ±15° avec 6 cycles par pulsation. La précision de la reconstruction est comparable à l’échographie Doppler conventionnelle, tout en ayant une fréquence d’image 10 à 20 fois supérieure, permettant une meilleure caractérisation des transitions rapides qui requiert une résolution temporelle élevée.

Ultrafast Echocardiography

Grâce à son accessibilité, sa polyvalence et sa sécurité, l'échocardiographie est devenue la technique d'imagerie la plus utilisée pour évaluer la fonction cardiaque. Au vu du succès de l'échographie ultrarapide par ondes planes des techniques similaires pour augmenter la résolution temporelle en échocardiographie ont été mise en oeuvre. L’augmentation de la résolution temporelle de l’échographie cardiaque au-delà des valeurs actuellement atteignables (~ 60 à 80 images par secondes), pourrait être utilisé pour améliorer d’autres caractéristiques de l'échocardiographie, comme par exemple élargir la plage de vitesses détectables en imagerie Doppler couleur limitées par la valeur de Nyquist. Nous avons étudié l'échocardiographie ultrarapide en utilisant des fronts d’ondes ultrasonores divergentes. La résolution temporelle atteinte par la méthode d'ondes divergentes a permis d’améliorer les capacités des modes d’échocardiographie en mode B et en Doppler couleur. La résolution temporelle de la méthode mode B a été augmentée jusqu'à 633 images par secondes, tout en gardant une qualité d'image comparable à celle de la méthode d’échocardiographie conventionnelle. La vitesse de Nyquist de la méthode Doppler couleur a été multipliée jusqu'à 6 fois au delà de la limite conventionnelle en utilisant une technique inspirée de l’imagerie radar; l’implémentation de cette méthode n’aurait pas été possible sans l’utilisation de fronts d’ondes divergentes. Les performances avantageuses de la méthode d'échocardiographie ultrarapide sont supportées par plusieurs résultats in vitro et in vivo inclus dans ce manuscrit.

VLSI architecture for motion estimation in underwater imaging

El treball desenvolupat en aquesta tesi aprofundeix i aporta solucions innovadores en el camp orientat a tractar el problema de la correspondència en imatges subaquàtiques. En aquests entorns, el que realment complica les tasques de processat és la falta de contorns ben definits per culpa d'imatges esborronades; un fet aquest que es deu fonamentalment a il·luminació deficient o a la manca d'uniformitat dels sistemes d'il·luminació artificials. Els objectius aconseguits en aquesta tesi es poden remarcar en dues grans direccions. Per millorar l'algorisme d'estimació de moviment es va proposar un nou mètode que introdueix paràmetres de textura per rebutjar falses correspondències entre parells d'imatges. Un seguit d'assaigs efectuats en imatges submarines reals han estat portats a terme per seleccionar les estratègies més adients. Amb la finalitat d'aconseguir resultats en temps real, es proposa una innovadora arquitectura VLSI per la implementació d'algunes parts de l'algorisme d'estimació de moviment amb alt cost computacional.

Defining user perception of distributed multimedia quality

This article presents the results of a study that explored the human side of the multimedia experience. We propose a model that assesses quality variation from three distinct levels: the network, the media and the content levels; and from two views: the technical and the user perspective. By facilitating parameter variation at each of the quality levels and from each of the perspectives, we were able to examine their impact on user quality perception. Results show that a significant reduction in frame rate does not proportionally reduce the user's understanding of the presentation independent of technical parameters, that multimedia content type significantly impacts user information assimilation, user level of enjoyment, and user perception of quality, and that the device display type impacts user information assimilation and user perception of quality. Finally, to ensure the transfer of information, low-level abstraction (network-level) parameters, such as delay and jitter, should be adapted; to maintain the user's level of enjoyment, high-level abstraction quality parameters (content-level), such as the appropriate use of display screens, should be adapted.

Infotainment across access devices: the perceptual impact of multimedia QoS

The introduction of multimedia on pervasive and mobile communication devices raises a number of perceptual quality issues. However, limited work has been done examining the 3-way interaction between use of equipment, user perceptual quality and quality of service. Our work measures user perceptual quality with the quality of perception (QoP) metrics which comprises levels of informational transfer (objective) and user satisfaction (subjective) when users are presented with multimedia video clips at three different frame rates, using four different display devices. Finally, our results will show that variation in frame-rate does not impact a user’s level of information assimilation (IA), however, does impact a users’ perception of multimedia video ‘quality’.

Inexpensive solution for real-time video and image stitching

Image stitching is the process of joining several images to obtain a bigger view of a scene. It is used, for example, in tourism to transmit to the viewer the sensation of being in another place. I am presenting an inexpensive solution for automatic real time video and image stitching with two web cameras as the video/image sources. The proposed solution relies on the usage of several markers in the scene as reference points for the stitching algorithm. The implemented algorithm is divided in four main steps, the marker detection, camera pose determination (in reference to the markers), video/image size and 3d transformation, and image translation. Wii remote controllers are used to support several steps in the process. The built‐in IR camera provides clean marker detection, which facilitates the camera pose determination. The only restriction in the algorithm is that markers have to be in the field of view when capturing the scene. Several tests where made to evaluate the final algorithm. The algorithm is able to perform video stitching with a frame rate between 8 and 13 fps. The joining of the two videos/images is good with minor misalignments in objects at the same depth of the marker,misalignments in the background and foreground are bigger. The capture process is simple enough so anyone can perform a stitching with a very short explanation. Although real‐time video stitching can be achieved by this affordable approach, there are few shortcomings in current version. For example, contrast inconsistency along the stitching line could be reduced by applying a color correction algorithm to every source videos. In addition, the misalignments in stitched images due to camera lens distortion could be eased by optical correction algorithm. The work was developed in Apple’s Quartz Composer, a visual programming environment. A library of extended functions was developed using Xcode tools also from Apple.