The influence of subpixel measurement on digital camera calibration

The aim of this work is to evaluate the influence of point measurements in images, with subpixel accuracy, and its contribution in the calibration of digital cameras. Also, the effect of subpixel measurements in 3D coordinates of check points in the object space will be evaluated. With this purpose, an algorithm that allows subpixel accuracy was implemented for semi-automatic determination of points of interest, based on Fõrstner operator. Experiments were accomplished with a block of images acquired with the multispectral camera DuncanTech MS3100-CIR. The influence of subpixel measurements in the adjustment by Least Square Method (LSM) was evaluated by the comparison of estimated standard deviation of parameters in both situations, with manual measurement (pixel accuracy) and with subpixel estimation. Additionally, the influence of subpixel measurements in the 3D reconstruction was also analyzed. Based on the obtained results, i.e., on the quantification of the standard deviation reduction in the Inner Orientation Parameters (IOP) and also in the relative error of the 3D reconstruction, it was shown that measurements with subpixel accuracy are relevant for some tasks in Photogrammetry, mainly for those in which the metric quality is of great relevance, as Camera Calibration.

Exterior orientation of CBERS-2B imagery using multi-feature control and orbital data

The major contribution of this paper relates to the practical advantages of combining Ground Control Points (GCPs), Ground Control Lines (GCLs) and orbital data to estimate the exterior orientation parameters of images collected by CBERS-2B (China-Brazil Earth Resources Satellite) HRC (High-resolution Camera) and CCD (High-resolution CCD Camera) sensors. Although the CBERS-2B is no longer operational, its images are still being used in Brazil, and the next generations of the CBERS satellite will have sensors with similar technical features, which motivates the study presented in this paper. The mathematical models that relate the object and image spaces are based on collinearity (for points) and coplanarity (for lines) conditions. These models were created in an in-house developed software package called TMS (Triangulation with Multiple Sensors) with multi-feature control (GCPs and GCLs). Experiments on a block of four CBERS-2B HRC images and on one CBERS-2B CCD image were performed using both models. It was observed that the combination of GCPs and GCLs provided better bundle block adjustment results than conventional bundle adjustment using only GCPs. The results also demonstrate the advantages of using primarily orbital data when the number of control entities is reduced. © 2013 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS).

Flares and variability from Sagittarius A(star): five nights of simultaneous multi-wavelength observations

Aims. We report on simultaneous observations and modeling of mid-infrared (MIR), near-infrared (NIR), and submillimeter (sub-mm) emission of the source Sgr A * associated with the supermassive black hole at the center of our Galaxy. Our goal was to monitor the activity of Sgr A* at different wavelengths in order to constrain the emitting processes and gain insight into the nature of the close environment of Sgr A*. Methods. We used the MIR instrument VISIR in the BURST imaging mode, the adaptive optics assisted NIR camera NACO, and the sub-mm antenna APEX to monitor Sgr A* over several nights in July 2007. Results. The observations reveal remarkable variability in the NIR and sub-mm during the five nights of observation. No source was detected in the MIR, but we derived the lowest upper limit for a flare at 8.59 mu m (22.4 mJy with A(8.59 mu m) = 1.6 +/- 0.5). This observational constraint makes us discard the observed NIR emission as coming from a thermal component emitting at sub-mm frequencies. Moreover, comparison of the sub-mm and NIR variability shows that the highest NIR fluxes (flares) are coincident with the lowest sub-mm levels of our five-night campaign involving three flares. We explain this behavior by a loss of electrons to the system and/or by a decrease in the magnetic field, as might conceivably occur in scenarios involving fast outflows and/or magnetic reconnection.

Framing Death. La morte in diretta, tra cinema e media digitali

Come dimostrano i sempre più numerosi casi di cronaca riportati dai notiziari, la preoccupazione per la gestione delle immagini di morte si configura come un nodo centrale che coinvolge spettatori, produttori di contenuti e broadcaster, dato che la sua emersione nel panorama mediale in cui siamo immersi è sempre più evidente. Se la letteratura socio-antropologica è generalmente concorde nel ritenere che, rispetto al passato, oggi la morte si manifesti con meno evidenza nella vita comune delle persone, che tendono a rimuovere i segni della contiguità vivendo il lutto in forma privata, essa è però percepita in modo pervasivo perché disseminata nei (e dai) media. L'elaborato, concentrandosi in maniera specifica sulle produzioni audiovisive, e quindi sulla possibilità intrinseca al cinema – e alle sue forme derivate – di registrare un evento in diretta, tenta di mappare alcune dinamiche di produzione e fruizione considerando una particolare manifestazione della morte: quella che viene comunemente indicata come “morte in diretta”. Dopo una prima ricognizione dedicata alla tensione continua tra la spinta a considerare la morte come l'ultimo tabù e le manifestazioni che essa assume all'interno della “necrocultura”, appare chiaro che il paradigma pornografico risulta ormai inefficace a delineare compiutamente le emersioni della morte nei media, soggetta a opacità e interdizioni variabili, e necessita dunque di prospettive analitiche più articolate. Il fulcro dell'analisi è dunque la produzione e il consumo di precisi filoni quali snuff, cannibal e mondo movie e quelle declinazioni del gore che hanno ibridato reale e fittizio: il tentativo è tracciare un percorso che, a partire dal cinema muto, giunga al panorama contemporaneo e alle pratiche di remix rese possibili dai media digitali, toccando episodi controversi come i Video Nasties, le dinamiche di moral panic scatenate dagli snuff film e quelle di contagio derivanti dalla manipolazione e diffusione delle immagini di morte.

Automatic Food Intake Assessment Using Camera Phones

Obesity is becoming an epidemic phenomenon in most developed countries. The fundamental cause of obesity and overweight is an energy imbalance between calories consumed and calories expended. It is essential to monitor everyday food intake for obesity prevention and management. Existing dietary assessment methods usually require manually recording and recall of food types and portions. Accuracy of the results largely relies on many uncertain factors such as user's memory, food knowledge, and portion estimations. As a result, the accuracy is often compromised. Accurate and convenient dietary assessment methods are still blank and needed in both population and research societies. In this thesis, an automatic food intake assessment method using cameras, inertial measurement units (IMUs) on smart phones was developed to help people foster a healthy life style. With this method, users use their smart phones before and after a meal to capture images or videos around the meal. The smart phone will recognize food items and calculate the volume of the food consumed and provide the results to users. The technical objective is to explore the feasibility of image based food recognition and image based volume estimation. This thesis comprises five publications that address four specific goals of this work: (1) to develop a prototype system with existing methods to review the literature methods, find their drawbacks and explore the feasibility to develop novel methods; (2) based on the prototype system, to investigate new food classification methods to improve the recognition accuracy to a field application level; (3) to design indexing methods for large-scale image database to facilitate the development of new food image recognition and retrieval algorithms; (4) to develop novel convenient and accurate food volume estimation methods using only smart phones with cameras and IMUs. A prototype system was implemented to review existing methods. Image feature detector and descriptor were developed and a nearest neighbor classifier were implemented to classify food items. A reedit card marker method was introduced for metric scale 3D reconstruction and volume calculation. To increase recognition accuracy, novel multi-view food recognition algorithms were developed to recognize regular shape food items. To further increase the accuracy and make the algorithm applicable to arbitrary food items, new food features, new classifiers were designed. The efficiency of the algorithm was increased by means of developing novel image indexing method in large-scale image database. Finally, the volume calculation was enhanced through reducing the marker and introducing IMUs. Sensor fusion technique to combine measurements from cameras and IMUs were explored to infer the metric scale of the 3D model as well as reduce noises from these sensors.

Interactive Augmentation of Live Images using a HDR Stereo Camera

Adding virtual objects to real environments plays an important role in todays computer graphics: Typical examples are virtual furniture in a real room and virtual characters in real movies. For a believable appearance, consistent lighting of the virtual objects is required. We present an augmented reality system that displays virtual objects with consistent illumination and shadows in the image of a simple webcam. We use two high dynamic range video cameras with fisheye lenses permanently recording the environment illumination. A sampling algorithm selects a few bright parts in one of the wide angle images and the corresponding points in the second camera image. The 3D position can then be calculated using epipolar geometry. Finally, the selected point lights are used in a multi pass algorithm to draw the virtual object with shadows. To validate our approach, we compare the appearance and shadows of the synthetic objects with real objects.

Gesture-Based, Touch-Free Multi-User Gaming on Wall-Sized, High-Resolution Tiled Displays

Having to carry input devices can be inconvenient when interacting with wall-sized, high-resolution tiled displays. Such displays are typically driven by a cluster of computers. Running existing games on a cluster is non-trivial, and the performance attained using software solutions like Chromium is not good enough. This paper presents a touch-free, multi-user, humancomputer interface for wall-sized displays that enables completely device-free interaction. The interface is built using 16 cameras and a cluster of computers, and is integrated with the games Quake 3 Arena (Q3A) and Homeworld. The two games were parallelized using two different approaches in order to run on a 7x4 tile, 21 megapixel display wall with good performance. The touch-free interface enables interaction with a latency of 116 ms, where 81 ms are due to the camera hardware. The rendering performance of the games is compared to their sequential counterparts running on the display wall using Chromium. Parallel Q3A’s framerate is an order of magnitude higher compared to using Chromium. The parallel version of Homeworld performed on par with the sequential, which did not run at all using Chromium. Informal use of the touch-free interface indicates that it works better for controlling Q3A than Homeworld.

NEW TOOLS FOR MONITORING GAMMA CAMERA UNIFORMITY

Detector uniformity is a fundamental performance characteristic of all modern gamma camera systems, and ensuring a stable, uniform detector response is critical for maintaining clinical images that are free of artifact. For these reasons, the assessment of detector uniformity is one of the most common activities associated with a successful clinical quality assurance program in gamma camera imaging. The evaluation of this parameter, however, is often unclear because it is highly dependent upon acquisition conditions, reviewer expertise, and the application of somewhat arbitrary limits that do not characterize the spatial location of the non-uniformities. Furthermore, as the goal of any robust quality control program is the determination of significant deviations from standard or baseline conditions, clinicians and vendors often neglect the temporal nature of detector degradation (1). This thesis describes the development and testing of new methods for monitoring detector uniformity. These techniques provide more quantitative, sensitive, and specific feedback to the reviewer so that he or she may be better equipped to identify performance degradation prior to its manifestation in clinical images. The methods exploit the temporal nature of detector degradation and spatially segment distinct regions-of-non-uniformity using multi-resolution decomposition. These techniques were tested on synthetic phantom data using different degradation functions, as well as on experimentally acquired time series floods with induced, progressively worsening defects present within the field-of-view. The sensitivity of conventional, global figures-of-merit for detecting changes in uniformity was evaluated and compared to these new image-space techniques. The image-space algorithms provide a reproducible means of detecting regions-of-non-uniformity prior to any single flood image’s having a NEMA uniformity value in excess of 5%. The sensitivity of these image-space algorithms was found to depend on the size and magnitude of the non-uniformities, as well as on the nature of the cause of the non-uniform region. A trend analysis of the conventional figures-of-merit demonstrated their sensitivity to shifts in detector uniformity. The image-space algorithms are computationally efficient. Therefore, the image-space algorithms should be used concomitantly with the trending of the global figures-of-merit in order to provide the reviewer with a richer assessment of gamma camera detector uniformity characteristics.

Anamorphic pixels for multi-channel superresolution

Superresolution from plenoptic cameras or camera arrays is usually treated similarly to superresolution from video streams. However, the transformation between the low-resolution views can be determined precisely from camera geometry and parallax. Furthermore, as each low-resolution image originates from a unique physical camera, its sampling properties can also be unique. We exploit this option with a custom design of either the optics or the sensor pixels. This design makes sure that the sampling matrix of the complete system is always well-formed, enabling robust and high-resolution image reconstruction. We show that simply changing the pixel aspect ratio from square to anamorphic is sufficient to achieve that goal, as long as each camera has a unique aspect ratio. We support this claim with theoretical analysis and image reconstruction of real images. We derive the optimal aspect ratios for sets of 2 or 4 cameras. Finally, we verify our solution with a camera system using an anamorphic lens.

Simultaneous 3D object tracking and camera parameter estimation by Bayesian methods and transdimensional MCMC sampling

Multi-camera 3D tracking systems with overlapping cameras represent a powerful mean for scene analysis, as they potentially allow greater robustness than monocular systems and provide useful 3D information about object location and movement. However, their performance relies on accurately calibrated camera networks, which is not a realistic assumption in real surveillance environments. Here, we introduce a multi-camera system for tracking the 3D position of a varying number of objects and simultaneously refin-ing the calibration of the network of overlapping cameras. Therefore, we introduce a Bayesian framework that combines Particle Filtering for tracking with recursive Bayesian estimation methods by means of adapted transdimensional MCMC sampling. Addi-tionally, the system has been designed to work on simple motion detection masks, making it suitable for camera networks with low transmission capabilities. Tests show that our approach allows a successful performance even when starting from clearly inaccurate camera calibrations, which would ruin conventional approaches.

Aplicaciones de interés forestal de las imágenes obtenidas con el sensor Multi-angle Imaging SpectroRadiometer (MISR)

Las aplicaciones de la teledetección al seguimiento de lo que ocurre en la superficie terrestre se han ido multiplicando y afinando con el lanzamiento de nuevos sensores por parte de las diferentes agencias espaciales. La necesidad de tener información actualizada cada poco tiempo y espacialmente homogénea, ha provocado el desarrollo de nuevos programas como el Earth Observing System (EOS) de la National Aeronautics and Space Administration (NASA). Uno de los sensores que incorpora el buque insignia de ese programa, el satélite TERRA, es el Multi-angle Imaging SpectroRadiometer (MISR), diseñado para capturar información multiangular de la superficie terrestre. Ya desde los años 1970, se conocía que la reflectancia de las diversas ocupaciones y usos del suelo variaba en función del ángulo de observación y de iluminación, es decir, que eran anisotrópicas. Tal variación estaba además relacionada con la estructura tridimensional de tales ocupaciones, por lo que se podía aprovechar tal relación para obtener información de esa estructura, más allá de la que pudiera proporcionar la información meramente espectral. El sensor MISR incorpora 9 cámaras a diferentes ángulos para capturar 9 imágenes casi simultáneas del mismo punto, lo que permite estimar con relativa fiabilidad la respuesta anisotrópica de la superficie terrestre. Varios trabajos han demostrado que se pueden estimar variables relacionadas con la estructura de la vegetación con la información que proporciona MISR. En esta Tesis se ha realizado una primera aplicación a la Península Ibérica, para comprobar su utilidad a la hora de estimar variables de interés forestal. En un primer paso se ha analizado la variabilidad temporal que se produce en los datos, debido a los cambios en la geometría de captación, es decir, debido a la posición relativa de sensores y fuente de iluminación, que en este caso es el Sol. Se ha comprobado cómo la anisotropía es mayor desde finales de otoño hasta principios de primavera debido a que la posición del Sol es más cercana al plano de los sensores. También se ha comprobado que los valores máximo y mínimo se van desplazando temporalmente entre el centro y el extremo angular. En la caracterización multiangular de ocupaciones del suelo de CORINE Land Cover que se ha realizado, se puede observar cómo la forma predominante en las imágenes con el Sol más alto es convexa con un máximo en la cámara más cercana a la fuente de iluminación. Sin embargo, cuando el Sol se encuentra mucho más bajo, ese máximo es muy externo. Por otra parte, los datos obtenidos en verano son mucho más variables para cada ocupación que los de noviembre, posiblemente debido al aumento proporcional de las zonas en sombra. Para comprobar si la información multiangular tiene algún efecto en la obtención de imágenes clasificadas según ocupación y usos del suelo, se han realizado una serie de clasificaciones variando la información utilizada, desde sólo multiespectral, a multiangular y multiespectral. Los resultados muestran que, mientras para las clasificaciones más genéricas la información multiangular proporciona los peores resultados, a medida que se amplían el número de clases a obtener tal información mejora a lo obtenido únicamente con información multiespectral. Por otra parte, se ha realizado una estimación de variables cuantitativas como la fracción de cabida cubierta (Fcc) y la altura de la vegetación a partir de información proporcionada por MISR a diferentes resoluciones. En el valle de Alcudia (Ciudad Real) se ha estimado la fracción de cabida cubierta del arbolado para un píxel de 275 m utilizando redes neuronales. Los resultados muestran que utilizar información multiespectral y multiangular puede mejorar casi un 20% las estimaciones realizadas sólo con datos multiespectrales. Además, las relaciones obtenidas llegan al 0,7 de R con errores inferiores a un 10% en Fcc, siendo éstos mucho mejores que los obtenidos con el producto elaborado a partir de datos multiespectrales del sensor Moderate Resolution Imaging Spectroradiometer (MODIS), también a bordo de Terra, para la misma variable. Por último, se ha estimado la fracción de cabida cubierta y la altura efectiva de la vegetación para 700.000 ha de la provincia de Murcia, con una resolución de 1.100 m. Los resultados muestran la relación existente entre los datos espectrales y los multiangulares, obteniéndose coeficientes de Spearman del orden de 0,8 en el caso de la fracción de cabida cubierta de la vegetación, y de 0,4 en el caso de la altura efectiva. Las estimaciones de ambas variables con redes neuronales y diversas combinaciones de datos, arrojan resultados con R superiores a 0,85 para el caso del grado de cubierta vegetal, y 0,6 para la altura efectiva. Los parámetros multiangulares proporcionados en los productos elaborados con MISR a 1.100 m, no obtienen buenos resultados por sí mismos pero producen cierta mejora al incorporarlos a la información espectral. Los errores cuadráticos medios obtenidos son inferiores a 0,016 para la Fcc de la vegetación en tanto por uno, y 0,7 m para la altura efectiva de la misma. Regresiones geográficamente ponderadas muestran además que localmente se pueden obtener mejores resultados aún mejores, especialmente cuando hay una mayor variabilidad espacial de las variables estimadas. En resumen, la utilización de los datos proporcionados por MISR ofrece una prometedora vía de mejora de resultados en la media-baja resolución, tanto para la clasificación de imágenes como para la obtención de variables cuantitativas de la estructura de la vegetación. ABSTRACT Applications of remote sensing for monitoring what is happening on the land surface have been multiplied and refined with the launch of new sensors by different Space Agencies. The need of having up to date and spatially homogeneous data, has led to the development of new programs such as the Earth Observing System (EOS) of the National Aeronautics and Space Administration (NASA). One of the sensors incorporating the flagship of that program, the TERRA satellite, is Multi-angle Imaging Spectroradiometer (MISR), designed to capture the multi-angle information of the Earth's surface. Since the 1970s, it was known that the reflectance of various land covers and land uses varied depending on the viewing and ilumination angles, so they are anisotropic. Such variation was also related to the three dimensional structure of such covers, so that one could take advantage of such a relationship to obtain information from that structure, beyond which spectral information could provide. The MISR sensor incorporates 9 cameras at different angles to capture 9 almost simultaneous images of the same point, allowing relatively reliable estimates of the anisotropic response of the Earth's surface. Several studies have shown that we can estimate variables related to the vegetation structure with the information provided by this sensor, so this thesis has made an initial application to the Iberian Peninsula, to check their usefulness in estimating forest variables of interest. In a first step we analyzed the temporal variability that occurs in the data, due to the changes in the acquisition geometry, i.e. the relative position of sensor and light source, which in this case is the Sun. It has been found that the anisotropy is greater from late fall through early spring due to the Sun's position closer to the plane of the sensors. It was also found that the maximum and minimum values are displaced temporarily between the center and the ends. In characterizing CORINE Land Covers that has been done, one could see how the predominant form in the images with the highest sun is convex with a maximum in the camera closer to the light source. However, when the sun is much lower, the maximum is external. Moreover, the data obtained for each land cover are much more variable in summer that in November, possibly due to the proportional increase in shadow areas. To check whether the information has any effect on multi-angle imaging classification of land cover and land use, a series of classifications have been produced changing the data used, from only multispectrally, to multi-angle and multispectral. The results show that while for the most generic classifications multi-angle information is the worst, as there are extended the number of classes to obtain such information it improves the results. On the other hand, an estimate was made of quantitative variables such as canopy cover and vegetation height using information provided by MISR at different resolutions. In the valley of Alcudia (Ciudad Real), we estimated the canopy cover of trees for a pixel of 275 m by using neural networks. The results showed that using multispectral and multiangle information can improve by almost 20% the estimates that only used multispectral data. Furthermore, the relationships obtained reached an R coefficient of 0.7 with errors below 10% in canopy cover, which is much better result than the one obtained using data from the Moderate Resolution Imaging Spectroradiometer (MODIS), also onboard Terra, for the same variable. Finally we estimated the canopy cover and the effective height of the vegetation for 700,000 hectares in the province of Murcia, with a spatial resolution of 1,100 m. The results show a relationship between the spectral and the multi-angle data, and provide estimates of the canopy cover with a Spearman’s coefficient of 0.8 in the case of the vegetation canopy cover, and 0.4 in the case of the effective height. The estimates of both variables using neural networks and various combinations of data, yield results with an R coefficient greater than 0.85 for the case of the canopy cover, and 0.6 for the effective height. Multi-angle parameters provided in the products made from MISR at 1,100 m pixel size, did not produce good results from themselves but improved the results when included to the spectral information. The mean square errors were less than 0.016 for the canopy cover, and 0.7 m for the effective height. Geographically weighted regressions also showed that locally we can have even better results, especially when there is high spatial variability of estimated variables. In summary, the use of the data provided by MISR offers a promising way of improving remote sensing performance in the low-medium spatial resolution, both for image classification and for the estimation of quantitative variables of the vegetation structure.

A cooperative multi-agent robotics system: design and modelling

This paper presents the development of the robotic multi-agent system SMART. In this system, the agent concept is applied to both hardware and software entities. Hardware agents are robots, with three and four legs, and an IP-camera that takes images of the scene where the cooperative task is carried out. Hardware agents strongly cooperate with software agents. These latter agents can be classified into image processing, communications, task management and decision making, planning and trajectory generation agents. To model, control and evaluate the performance of cooperative tasks among agents, a kind of PetriNet, called Work-Flow Petri Net, is used. Experimental results shows the good performance of the system.

Seamless, Static Multi-Texturing of 3D Meshes

In the context of 3D reconstruction, we present a static multi-texturing system yielding a seamless texture atlas calculated by combining the colour information from several photos from the same subject covering most of its surface. These pictures can be provided by shooting just one camera several times when reconstructing a static object, or a set of synchronized cameras, when dealing with a human or any other moving object. We suppress the colour seams due to image misalignments and irregular lighting conditions that multi-texturing approaches typically suffer from, while minimizing the blurring effect introduced by colour blending techniques. Our system is robust enough to compensate for the almost inevitable inaccuracies of 3D meshes obtained with visual hull–based techniques: errors in silhouette segmentation, inherently bad handling of concavities, etc.

Data reduction pipeline for emir, the near-IR multi-object spectrograph for GTC

EMIR (Balcells et al. 2000) is a near-infrared wide-field camera and multi-object spectrograph being built for the GTC. The Data Reduction Pipeline (DRP) will be optimized for handling and reducing near-infrared data acquired with EMIR.

Self-calibrated, multi-spectral photometric stereo for 3D face capture

This paper addresses the problem of obtaining 3d detailed reconstructions of human faces in real-time and with inexpensive hardware. We present an algorithm based on a monocular multi-spectral photometric-stereo setup. This system is known to capture high-detailed deforming 3d surfaces at high frame rates and without having to use any expensive hardware or synchronized light stage. However, the main challenge of such a setup is the calibration stage, which depends on the lights setup and how they interact with the specific material being captured, in this case, human faces. For this purpose we develop a self-calibration technique where the person being captured is asked to perform a rigid motion in front of the camera, maintaining a neutral expression. Rigidity constrains are then used to compute the head's motion with a structure-from-motion algorithm. Once the motion is obtained, a multi-view stereo algorithm reconstructs a coarse 3d model of the face. This coarse model is then used to estimate the lighting parameters with a stratified approach: In the first step we use a RANSAC search to identify purely diffuse points on the face and to simultaneously estimate this diffuse reflectance model. In the second step we apply non-linear optimization to fit a non-Lambertian reflectance model to the outliers of the previous step. The calibration procedure is validated with synthetic and real data.