Object and Scene Classification: what does a Supervised Approach Provide us?

Given a set of images of scenes containing different object categories (e.g. grass, roads) our objective is to discover these objects in each image, and to use this object occurrences to perform a scene classification (e.g. beach scene, mountain scene). We achieve this by using a supervised learning algorithm able to learn with few images to facilitate the user task. We use a probabilistic model to recognise the objects and further we classify the scene based on their object occurrences. Experimental results are shown and evaluated to prove the validity of our proposal. Object recognition performance is compared to the approaches of He et al. (2004) and Marti et al. (2001) using their own datasets. Furthermore an unsupervised method is implemented in order to evaluate the advantages and disadvantages of our supervised classification approach versus an unsupervised one

Anàlisi de l'aplicació GeoMedia 3D

Aquest treball es centra en la utilització i aplicació de les tecnologies dels sistemes d'informació geogràfica. En concret es realitza una pràctica sobre la ciutat de Lleida per poder veure certs edificis que compleixen una sèrie de premisses. En el projecte es fa una anàlisi de l'aplicació GeoMedia 3D.

Development of model observers applied to 3D breast tomosynthesis microcalcifications and masses

The development of model observers for mimicking human detection strategies has followed from symmetric signals in simple noise to increasingly complex backgrounds. In this study we implement different model observers for the complex task of detecting a signal in a 3D image stack. The backgrounds come from real breast tomosynthesis acquisitions and the signals were simulated and reconstructed within the volume. Two different tasks relevant to the early detection of breast cancer were considered: detecting an 8 mm mass and detecting a cluster of microcalcifications. The model observers were calculated using a channelized Hotelling observer (CHO) with dense difference-of-Gaussian channels, and a modified (Partial prewhitening [PPW]) observer which was adapted to realistic signals which are not circularly symmetric. The sustained temporal sensitivity function was used to filter the images before applying the spatial templates. For a frame rate of five frames per second, the only CHO that we calculated performed worse than the humans in a 4-AFC experiment. The other observers were variations of PPW and outperformed human observers in every single case. This initial frame rate was a rather low speed and the temporal filtering did not affect the results compared to a data set with no human temporal effects taken into account. We subsequently investigated two higher speeds at 5, 15 and 30 frames per second. We observed that for large masses, the two types of model observers investigated outperformed the human observers and would be suitable with the appropriate addition of internal noise. However, for microcalcifications both only the PPW observer consistently outperformed the humans. The study demonstrated the possibility of using a model observer which takes into account the temporal effects of scrolling through an image stack while being able to effectively detect a range of mass sizes and distributions.

A 2D/3D image analysis system to track fluorescently labeled structures in rod-shaped cells: application to measure spindle pole asymmetry during mitosis.

BACKGROUND: The yeast Schizosaccharomyces pombe is frequently used as a model for studying the cell cycle. The cells are rod-shaped and divide by medial fission. The process of cell division, or cytokinesis, is controlled by a network of signaling proteins called the Septation Initiation Network (SIN); SIN proteins associate with the SPBs during nuclear division (mitosis). Some SIN proteins associate with both SPBs early in mitosis, and then display strongly asymmetric signal intensity at the SPBs in late mitosis, just before cytokinesis. This asymmetry is thought to be important for correct regulation of SIN signaling, and coordination of cytokinesis and mitosis. In order to study the dynamics of organelles or large protein complexes such as the spindle pole body (SPB), which have been labeled with a fluorescent protein tag in living cells, a number of the image analysis problems must be solved; the cell outline must be detected automatically, and the position and signal intensity associated with the structures of interest within the cell must be determined. RESULTS: We present a new 2D and 3D image analysis system that permits versatile and robust analysis of motile, fluorescently labeled structures in rod-shaped cells. We have designed an image analysis system that we have implemented as a user-friendly software package allowing the fast and robust image-analysis of large numbers of rod-shaped cells. We have developed new robust algorithms, which we combined with existing methodologies to facilitate fast and accurate analysis. Our software permits the detection and segmentation of rod-shaped cells in either static or dynamic (i.e. time lapse) multi-channel images. It enables tracking of two structures (for example SPBs) in two different image channels. For 2D or 3D static images, the locations of the structures are identified, and then intensity values are extracted together with several quantitative parameters, such as length, width, cell orientation, background fluorescence and the distance between the structures of interest. Furthermore, two kinds of kymographs of the tracked structures can be established, one representing the migration with respect to their relative position, the other representing their individual trajectories inside the cell. This software package, called "RodCellJ", allowed us to analyze a large number of S. pombe cells to understand the rules that govern SIN protein asymmetry. CONCLUSIONS: "RodCell" is freely available to the community as a package of several ImageJ plugins to simultaneously analyze the behavior of a large number of rod-shaped cells in an extensive manner. The integration of different image-processing techniques in a single package, as well as the development of novel algorithms does not only allow to speed up the analysis with respect to the usage of existing tools, but also accounts for higher accuracy. Its utility was demonstrated on both 2D and 3D static and dynamic images to study the septation initiation network of the yeast Schizosaccharomyces pombe. More generally, it can be used in any kind of biological context where fluorescent-protein labeled structures need to be analyzed in rod-shaped cells. AVAILABILITY: RodCellJ is freely available under http://bigwww.epfl.ch/algorithms.html, (after acceptance of the publication).

Temporal SNR characteristics in segmented 3D-EPI at 7T.

Three-dimensional segmented echo planar imaging (3D-EPI) is a promising approach for high-resolution functional magnetic resonance imaging, as it provides an increased signal-to-noise ratio (SNR) at similar temporal resolution to traditional multislice 2D-EPI readouts. Recently, the 3D-EPI technique has become more frequently used and it is important to better understand its implications for fMRI. In this study, the temporal SNR characteristics of 3D-EPI with varying numbers of segments are studied. It is shown that, in humans, the temporal variance increases with the number of segments used to form the EPI acquisition and that for segmented acquisitions, the maximum available temporal SNR is reduced compared to single shot acquisitions. This reduction with increased segmentation is not found in phantom data and thus likely due to physiological processes. When operating in the thermal noise dominated regime, fMRI experiments with a motor task revealed that the 3D variant outperforms the 2D-EPI in terms of temporal SNR and sensitivity to detect activated brain regions. Thus, the theoretical SNR advantage of a segmented 3D-EPI sequence for fMRI only exists in a low SNR situation. However, other advantages of 3D-EPI, such as the application of parallel imaging techniques in two dimensions and the low specific absorption rate requirements, may encourage the use of the 3D-EPI sequence for fMRI in situations with higher SNR.

Multi-scale label interaction to improve object detection and segmentation

The project aims at advancing the state of the art in the use of context information for classification of image and video data. The use of context in the classification of images has been showed of great importance to improve the performance of actual object recognition systems. In our project we proposed the concept of Multi-scale Feature Labels as a general and compact method to exploit the local and global context. The feature extraction from the discriminative probability or classification confidence label field is of great novelty. Moreover the use of a multi-scale representation of the feature labels lead to a compact and efficient description of the context. The goal of the project has been also to provide a general-purpose method and prove its suitability in different image/video analysis problem. The two-year project generated 5 journal publications (plus 2 under submission), 10 conference publications (plus 2 under submission) and one patent (plus 1 pending). Of these publications, a relevant number make use of the main result of this project to improve the results in detection and/or segmentation of objects.

Creación de una demo 3D mediante gráficos procedurales

Aquest projecte consisteix en el desenvolupament d’una demo 3D utilitzant exclusivament gràfics procedurals per tal d’avaluar la seva viabilitat en aplicacions més complexes com els videojocs. En aquesta aplicació es genera un terreny aleatori explorable amb vegetació i textures creades proceduralment.

FIB-SEM tomography in biology.

Three-dimensional information is much easier to understand than a set of two-dimensional images. Therefore a layman is thrilled by the pseudo-3D image taken in a scanning electron microscope (SEM) while, when seeing a transmission electron micrograph, his imagination is challenged. First approaches to gain insight in the third dimension were to make serial microtome sections of a region of interest (ROI) and then building a model of the object. Serial microtome sectioning is a tedious and skill-demanding work and therefore seldom done. In the last two decades with the increase of computer power, sophisticated display options, and the development of new instruments, an SEM with a built-in microtome as well as a focused ion beam scanning electron microscope (FIB-SEM), serial sectioning, and 3D analysis has become far easier and faster.Due to the relief like topology of the microtome trimmed block face of resin-embedded tissue, the ROI can be searched in the secondary electron mode, and at the selected spot, the ROI is prepared with the ion beam for 3D analysis. For FIB-SEM tomography, a thin slice is removed with the ion beam and the newly exposed face is imaged with the electron beam, usually by recording the backscattered electrons. The process, also called "slice and view," is repeated until the desired volume is imaged.As FIB-SEM allows 3D imaging of biological fine structure at high resolution of only small volumes, it is crucial to perform slice and view at carefully selected spots. Finding the region of interest is therefore a prerequisite for meaningful imaging. Thin layer plastification of biofilms offers direct access to the original sample surface and allows the selection of an ROI for site-specific FIB-SEM tomography just by its pronounced topographic features.

Intracellular nanomanipulation by a photonic-force microscope with real-time acquisition of a 3D stiffness matrix.

A traditional photonic-force microscope (PFM) results in huge sets of data, which requires tedious numerical analysis. In this paper, we propose instead an analog signal processor to attain real-time capabilities while retaining the richness of the traditional PFM data. Our system is devoted to intracellular measurements and is fully interactive through the use of a haptic joystick. Using our specialized analog hardware along with a dedicated algorithm, we can extract the full 3D stiffness matrix of the optical trap in real time, including the off-diagonal cross-terms. Our system is also capable of simultaneously recording data for subsequent offline analysis. This allows us to check that a good correlation exists between the classical analysis of stiffness and our real-time measurements. We monitor the PFM beads using an optical microscope. The force-feedback mechanism of the haptic joystick helps us in interactively guiding the bead inside living cells and collecting information from its (possibly anisotropic) environment. The instantaneous stiffness measurements are also displayed in real time on a graphical user interface. The whole system has been built and is operational; here we present early results that confirm the consistency of the real-time measurements with offline computations.

Patient-specific computational hemodynamics of intracranial aneurysms from 3D rotational angiography and CT angiography: an in vivo reproducibility study

Patient-specific simulations of the hemodynamics in intracranial aneurysms can be constructed by using image-based vascular models and CFD techniques. This work evaluates the impact of the choice of imaging technique on these simulations

Temporal diffeomorphic free-form deformation: application to motion and strain estimation from 3D echocardiography

This paper presents a new registration algorithm, called Temporal Di eomorphic Free Form Deformation (TDFFD), and its application to motion and strain quanti cation from a sequence of 3D ultrasound (US) images. The originality of our approach resides in enforcing time consistency by representing the 4D velocity eld as the sum of continuous spatiotemporal B-Spline kernels. The spatiotemporal displacement eld is then recovered through forward Eulerian integration of the non-stationary velocity eld. The strain tensor iscomputed locally using the spatial derivatives of the reconstructed displacement eld. The energy functional considered in this paper weighs two terms: the image similarity and a regularization term. The image similarity metric is the sum of squared di erences between the intensities of each frame and a reference one. Any frame in the sequence can be chosen as reference. The regularization term is based on theincompressibility of myocardial tissue. TDFFD was compared to pairwise 3D FFD and 3D+t FFD, bothon displacement and velocity elds, on a set of synthetic 3D US images with di erent noise levels. TDFFDshowed increased robustness to noise compared to these two state-of-the-art algorithms. TDFFD also proved to be more resistant to a reduced temporal resolution when decimating this synthetic sequence. Finally, this synthetic dataset was used to determine optimal settings of the TDFFD algorithm. Subsequently, TDFFDwas applied to a database of cardiac 3D US images of the left ventricle acquired from 9 healthy volunteers and 13 patients treated by Cardiac Resynchronization Therapy (CRT). On healthy cases, uniform strain patterns were observed over all myocardial segments, as physiologically expected. On all CRT patients, theimprovement in synchrony of regional longitudinal strain correlated with CRT clinical outcome as quanti ed by the reduction of end-systolic left ventricular volume at follow-up (6 and 12 months), showing the potential of the proposed algorithm for the assessment of CRT.

Magnetization transfer-based 3D visualization of foot peripheral nerves.

PURPOSE: To investigate magnetization transfer (MT) effects as a new source of contrast for imaging and tracking of peripheral foot nerves. MATERIALS AND METHODS: Two sets of 3D spoiled gradient-echo images acquired with and without a saturation pulse were used to generate MT ratio (MTR) maps of 260 μm in-plane resolution for eight volunteers at 3T. Scan parameters were adjusted to minimize signal loss due to T2 dephasing, and a dedicated coil was used to improve the inherently low signal-to-noise ratio of small voxels. Resulting MTR values in foot nerves were compared with those in surrounding muscle tissue. RESULTS: Average MTR values for muscle (45.5 ± 1.4%) and nerve (21.4 ± 3.1%) were significantly different (P < 0.0001). In general, the difference in MTR values was sufficiently large to allow for intensity-based segmentation and tracking of foot nerves in individual subjects. This procedure was termed MT-based 3D visualization. CONCLUSION: The MTR serves as a new source of contrast for imaging of peripheral foot nerves and provides a means for high spatial resolution tracking of these structures. The proposed methodology is directly applicable on standard clinical MR scanners and could be applied to systemic pathologies, such as diabetes.

Sex differences: From mental rotation to object location memory, an ERP study

Sex differences in cognition have been largely investigated. The most consistent sex differences favoring females are observed in object location memory involving the left hemisphere whereas the most consistent sex differences favoring males are observed in tasks that require mental rotation involving the right hemisphere. Here we used a task involving these two abilities to see the impact of mental rotation on object location memory. To that end we used a combination of behavioral and event-related potential (ERP) electroencephalography (EEG) measures.A computer screen displayed a square frame of 4 pairs of images (a "teddy" bear, a shoe, an umbrella and a lamp) randomly arranged around a central fixation cross. After a 10-second interval for memorization, images disappeared and were replaced by a test frame with no image but a random pair of two locations marked in black. In addition, this test frame was randomly displayed either in the original orientation (0° rotation) or in the rotated one (90° clockwise - CW - or 90° counterclockwise - CCW). Preceding the test frame, an arrow indicating the presence or the absence of rotation of the frame was displayed on the screen. The task of the participants (15 females and 15 males) was to determine if two marked locations corresponded or not to a pair of identical images. Each response was followed by feedback.Findings showed no significant sex differences in the performance of the original orientation. In comparison with this position, the rotation of the frame produced an equal decrease of male and female performance. In addition, this decrease was significantly higher when the rotation of the frame was in a CCW direction. We further assessed the ERP when the arrow indicated the direction of rotation as stimulus-onset, during four time windows representing major components C1, P1, N1 and N2. Although no sex differences were observed in performance, brain activities differed according to sex. Enhanced amplitudes were found for the CCW compared to CW rotation over the right posterior areas for the P1, N1 and N2 components for men as well as for women. Major topographical differences related to sex were measured for the CW rotation condition as marked lateralized amplitude: left-hemisphere amplitude larger than right one was measured during P1 time range for men. These similar patterns prolonged from P1 to N1 for women. Early distinctions were found in interaction with sex between CCW and CW waveform amplitudes, expressing over anterior electrode sites during C1 time range (0-50 ms post-stimulus).In conclusion (i) women do not outperform men in object location memory in this study (absence of rotation condition); (ii) mental rotation, in particular the direction of rotation, influences performance on object location memory; (iii) CCW rotation is associated with activity in the right parietal hemisphere whereas the CW rotation involves the left parietal hemisphere; (iv) this last effect is less pronounced in males, which could explain why greater involvement of right parietal areas in men and of bilateral posterior areas in women is generally reported in mental rotation tasks; and (v) the early distinctions between both directions of rotation located over anterior sites could be related to sex differences in their respective involvement of control mechanisms.