Plenoptic camera : theory and experimental results

The purpose of this research was to develop a working physical model of the focused plenoptic camera and develop software that can process the measured image intensity, reconstruct this into a full resolution image, and to develop a depth map from its corresponding rendered image. The plenoptic camera is a specialized imaging system designed to acquire spatial, angular, and depth information in a single intensity measurement. This camera can also computationally refocus an image by adjusting the patch size used to reconstruct the image. The published methods have been vague and conflicting, so the motivation behind this research is to decipher the work that has been done in order to develop a working proof-of-concept model. This thesis outlines the theory behind the plenoptic camera operation and shows how the measured intensity from the image sensor can be turned into a full resolution rendered image with its corresponding depth map. The depth map can be created by a cross-correlation of adjacent sub-images created by the microlenslet array (MLA.) The full resolution image reconstruction can be done by taking a patch from each MLA sub-image and piecing them together like a puzzle. The patch size determines what object plane will be in-focus. This thesis also goes through a very rigorous explanation of the design constraints involved with building a plenoptic camera. Plenoptic camera data from Adobe © was used to help with the development of the algorithms written to create a rendered image and its depth map. Finally, using the algorithms developed from these tests and the knowledge for developing the plenoptic camera, a working experimental system was built, which successfully generated a rendered image and its corresponding depth map.

Reconocimiento de objetos a partir de imágenes termográficas

La termografía es un método de inspección y diagnóstico basado en la radiación infrarroja que emiten los cuerpos. Permite medir dicha radiación a distancia y sin contacto, obteniendo un termograma o imagen termográfica, objeto de estudio de este proyecto. Todos los cuerpos que se encuentren a una cierta temperatura emiten radiación infrarroja. Sin embargo, para hacer una inspección termográfica hay que tener en cuenta la emisividad de los cuerpos, capacidad que tienen de emitir radiación, ya que ésta no sólo depende de la temperatura del cuerpo, sino también de sus características superficiales. Las herramientas necesarias para conseguir un termograma son principalmente una cámara termográfica y un software que permita su análisis. La cámara percibe la emisión infrarroja de un objeto y lo convierte en una imagen visible, originalmente monocromática. Sin embargo, después es coloreada por la propia cámara o por un software para una interpretación más fácil del termograma. Para obtener estas imágenes termográficas existen varias técnicas, que se diferencian en cómo la energía calorífica se transfiere al cuerpo. Estas técnicas se clasifican en termografía pasiva, activa y vibrotermografía. El método que se utiliza en cada caso depende de las características térmicas del cuerpo, del tipo de defecto a localizar o la resolución espacial de las imágenes, entre otros factores. Para analizar las imágenes y así obtener diagnósticos y detectar defectos, es importante la precisión. Por ello existe un procesado de las imágenes, para minimizar los efectos provocados por causas externas, mejorar la calidad de la imagen y extraer información de las inspecciones realizadas. La termografía es un método de ensayo no destructivo muy flexible y que ofrece muchas ventajas. Por esta razón el campo de aplicación es muy amplio, abarcando desde aplicaciones industriales hasta investigación y desarrollo. Vigilancia y seguridad, ahorro energético, medicina o medio ambiente, son algunos de los campos donde la termografía aportaimportantes beneficios. Este proyecto es un estudio teórico de la termografía, donde se describen detalladamente cada uno de los aspectos mencionados. Concluye con una aplicación práctica, creando una cámara infrarroja a partir de una webcam, y realizando un análisis de las imágenes obtenidas con ella. Con esto se demuestran algunas de las teorías explicadas, así como la posibilidad de reconocer objetos mediante la termografía. Thermography is a method of testing and diagnosis based on the infrared radiation emitted by bodies. It allows to measure this radiation from a distance and with no contact, getting a thermogram or thermal image, object of study of this project. All bodies that are at a certain temperature emit infrared radiation. However, making a thermographic inspection must take into account the emissivity of the body, capability of emitting radiation. This not only depends on the temperature of the body, but also on its surface characteristics. The tools needed to get a thermogram are mainly a thermal imaging camera and software that allows analysis. The camera sees the infrared emission of an object and converts it into a visible image, originally monochrome. However, after it is colored by the camera or software for easier interpretation of thermogram. To obtain these thermal images it exists various techniques, which differ in how heat energy is transferred to the body. These techniques are classified into passive thermography, active and vibrotermografy. The method used in each case depends on the thermal characteristics of the body, the type of defect to locate or spatial resolution of images, among other factors. To analyze the images and obtain diagnoses and defects, accuracy is important. Thus there is a image processing to minimize the effects caused by external causes, improving image quality and extract information from inspections. Thermography is a non-­‐destructive test method very flexible and offers many advantages. So the scope is very wide, ranging from industrial applications to research and development.Surveillance and security, energy saving, environmental or medicine are some of the areas where thermography provides significant benefits. This project is a theoretical study of thermography, which describes in detail each of these aspects. It concludes with a practical application, creating an infrared camera from a webcam, and making an analysis of the images obtained with it. This will demonstrate some of the theories explained as well as the ability to recognize objects by thermography.

VLSI Watermark Implementations and Applications

This paper presents an up to date review of digital watermarking (WM) from a VLSI designer point of view. The reader is introduced to basic principles and terms in the field of image watermarking. It goes through a brief survey on WM theory, laying out common classification criterions and discussing important design considerations and trade-offs. Elementary WM properties such as robustness, computational complexity and their influence on image quality are discussed. Common attacks and testing benchmarks are also briefly mentioned. It is shown that WM design must take the intended application into account. The difference between software and hardware implementations is explained through the introduction of a general scheme of a WM system and two examples from previous works. A versatile methodology to aid in a reliable and modular design process is suggested. Relating to mixed-signal VLSI design and testing, the proposed methodology allows an efficient development of a CMOS image sensor with WM capabilities.

Multi-sensor tracking using a scalable condensation filter

Surveillance and tracking systems typically use a single colour modality for their input. These systems work well in controlled conditions but often fail with low lighting, shadowing, smoke, dust, unstable backgrounds or when the foreground object is of similar colouring to the background. With advances in technology and manufacturing techniques, sensors that allow us to see into the thermal infrared spectrum are becoming more affordable. By using modalities from both the visible and thermal infrared spectra, we are able to obtain more information from a scene and overcome the problems associated with using visible light only for surveillance and tracking. Thermal images are not affected by lighting or shadowing and are not overtly affected by smoke, dust or unstable backgrounds. We propose and evaluate three approaches for fusing visual and thermal images for person tracking. We also propose a modified condensation filter to track and aid in the fusion of the modalities. We compare the proposed fusion schemes with using the visual and thermal domains on their own, and demonstrate that significant improvements can be achieved by using multiple modalities.

Distributed low-power image processing in wireless sensor networks for intelligent video surveillance applications

Distributed Wireless Smart Camera (DWSC) network is a special type of Wireless Sensor Network (WSN) that processes captured images in a distributed manner. While image processing on DWSCs sees a great potential for growth, with its applications possessing a vast practical application domain such as security surveillance and health care, it suffers from tremendous constraints. In addition to the limitations of conventional WSNs, image processing on DWSCs requires more computational power, bandwidth and energy that presents significant challenges for large scale deployments. This dissertation has developed a number of algorithms that are highly scalable, portable, energy efficient and performance efficient, with considerations of practical constraints imposed by the hardware and the nature of WSN. More specifically, these algorithms tackle the problems of multi-object tracking and localisation in distributed wireless smart camera net- works and optimal camera configuration determination. Addressing the first problem of multi-object tracking and localisation requires solving a large array of sub-problems. The sub-problems that are discussed in this dissertation are calibration of internal parameters, multi-camera calibration for localisation and object handover for tracking. These topics have been covered extensively in computer vision literatures, however new algorithms must be invented to accommodate the various constraints introduced and required by the DWSC platform. A technique has been developed for the automatic calibration of low-cost cameras which are assumed to be restricted in their freedom of movement to either pan or tilt movements. Camera internal parameters, including focal length, principal point, lens distortion parameter and the angle and axis of rotation, can be recovered from a minimum set of two images of the camera, provided that the axis of rotation between the two images goes through the camera's optical centre and is parallel to either the vertical (panning) or horizontal (tilting) axis of the image. For object localisation, a novel approach has been developed for the calibration of a network of non-overlapping DWSCs in terms of their ground plane homographies, which can then be used for localising objects. In the proposed approach, a robot travels through the camera network while updating its position in a global coordinate frame, which it broadcasts to the cameras. The cameras use this, along with the image plane location of the robot, to compute a mapping from their image planes to the global coordinate frame. This is combined with an occupancy map generated by the robot during the mapping process to localised objects moving within the network. In addition, to deal with the problem of object handover between DWSCs of non-overlapping fields of view, a highly-scalable, distributed protocol has been designed. Cameras that follow the proposed protocol transmit object descriptions to a selected set of neighbours that are determined using a predictive forwarding strategy. The received descriptions are then matched at the subsequent camera on the object's path using a probability maximisation process with locally generated descriptions. The second problem of camera placement emerges naturally when these pervasive devices are put into real use. The locations, orientations, lens types etc. of the cameras must be chosen in a way that the utility of the network is maximised (e.g. maximum coverage) while user requirements are met. To deal with this, a statistical formulation of the problem of determining optimal camera configurations has been introduced and a Trans-Dimensional Simulated Annealing (TDSA) algorithm has been proposed to effectively solve the problem.

Combining multiple sensor modalities for a localisation robust to smoke

This paper proposes an approach to obtain a localisation that is robust to smoke by exploiting multiple sensing modalities: visual and infrared (IR) cameras. This localisation is based on a state-of-the-art visual SLAM algorithm. First, we show that a reasonably accurate localisation can be obtained in the presence of smoke by using only an IR camera, a sensor that is hardly affected by smoke, contrary to a visual camera (operating in the visible spectrum). Second, we demonstrate that improved results can be obtained by combining the information from the two sensor modalities (visual and IR cameras). Third, we show that by detecting the impact of smoke on the visual images using a data quality metric, we can anticipate and mitigate the degradation in performance of the localisation by discarding the most affected data. The experimental validation presents multiple trajectories estimated by the various methods considered, all thoroughly compared to an accurate dGPS/INS reference.

Comparison of 10 Multi-Sensor Image Fusion Paradigms for IKONOS Images

Fusion of multi-sensor imaging data enables a synergetic interpretation of complementary information obtained by sensors of different spectral ranges. Multi-sensor data of diverse spectral, spatial and temporal resolutions require advanced numerical techniques for analysis and interpretation. This paper reviews ten advanced pixel based image fusion techniques – Component substitution (COS), Local mean and variance matching, Modified IHS (Intensity Hue Saturation), Fast Fourier Transformed-enhanced IHS, Laplacian Pyramid, Local regression, Smoothing filter (SF), Sparkle, SVHC and Synthetic Variable Ratio. The above techniques were tested on IKONOS data (Panchromatic band at 1 m spatial resolution and Multispectral 4 bands at 4 m spatial resolution). Evaluation of the fused results through various accuracy measures, revealed that SF and COS methods produce images closest to corresponding multi-sensor would observe at the highest resolution level (1 m).

MULTI-OBJECTIVE GENETIC ALGORITHM FOR EFFICIENT POINT MATCHING IN MULTI-SENSOR SATELLITE IMAGE

This paper investigates a new approach for point matching in multi-sensor satellite images. The feature points are matched using multi-objective optimization (angle criterion and distance condition) based on Genetic Algorithm (GA). This optimization process is more efficient as it considers both the angle criterion and distance condition to incorporate multi-objective switching in the fitness function. This optimization process helps in matching three corresponding corner points detected in the reference and sensed image and thereby using the affine transformation, the sensed image is aligned with the reference image. From the results obtained, the performance of the image registration is evaluated and it is concluded that the proposed approach is efficient.

Anvers, Rudd [non visible sur l'image] gagne [la] finale du 400 m [des Jeux olympiques, seuls apparaissent les coureurs Ainsworth, Shea et Dafel, 20/8/20] : [photographie de presse] / [Agence Rol]

Référence bibliographique : Rol, 60650

Optimization of the Mechanical and Optical Properties of Tunable Optical Sensor Arrays (TOSA) for a Nanospectrometer in the Visible and Near Infrared Spectral Range

Tunable Optical Sensor Arrays (TOSA) based on Fabry-Pérot (FP) filters, for high quality spectroscopic applications in the visible and near infrared spectral range are investigated within this work. The optical performance of the FP filters is improved by using ion beam sputtered niobium pentoxide (Nb2O5) and silicon dioxide (SiO2) Distributed Bragg Reflectors (DBRs) as mirrors. Due to their high refractive index contrast, only a few alternating pairs of Nb2O5 and SiO2 films can achieve DBRs with high reflectivity in a wide spectral range, while ion beam sputter deposition (IBSD) is utilized due to its ability to produce films with high optical purity. However, IBSD films are highly stressed; resulting in stress induced mirror curvature and suspension bending in the free standing filter suspensions of the MEMS (Micro-Electro-Mechanical Systems) FP filters. Stress induced mirror curvature results in filter transmission line degradation, while suspension bending results in high required filter tuning voltages. Moreover, stress induced suspension bending results in higher order mode filter operation which in turn degrades the optical resolution of the filter. Therefore, the deposition process is optimized to achieve both near zero absorption and low residual stress. High energy ion bombardment during film deposition is utilized to reduce the film density, and hence the film compressive stress. Utilizing this technique, the compressive stress of Nb2O5 is reduced by ~43%, while that for SiO2 is reduced by ~40%. Filters fabricated with stress reduced films show curvatures as low as 100 nm for 70 μm mirrors. To reduce the stress induced bending in the free standing filter suspensions, a stress optimized multi-layer suspension design is presented; with a tensile stressed metal sandwiched between two compressively stressed films. The stress in Physical Vapor Deposited (PVD) metals is therefore characterized for use as filter top-electrode and stress compensating layer. Surface micromachining is used to fabricate tunable FP filters in the visible spectral range using the above mentioned design. The upward bending of the suspensions is reduced from several micrometers to less than 100 nm and 250 nm for two different suspension layer combinations. Mechanical tuning of up to 188 nm is obtained by applying 40 V of actuation voltage. Alternatively, a filter line with transmission of 65.5%, Full Width at Half Maximum (FWHM) of 10.5 nm and a stopband of 170 nm (at an output wavelength of 594 nm) is achieved. Numerical model simulations are also performed to study the validity of the stress optimized suspension design for the near infrared spectral range, wherein membrane displacement and suspension deformation due to material residual stress is studied. Two bandpass filter designs based on quarter-wave and non-quarter-wave layers are presented as integral components of the TOSA. With a filter passband of 135 nm and a broad stopband of over 650 nm, high average filter transmission of 88% is achieved inside the passband, while maximum filter transmission of less than 1.6% outside the passband is achieved.

Visible-near infrared spectroscopy sensor for predicting curd and whey composition during cheese processing

The potential of visible-near infrared spectra, obtained using a light backscatter sensor, in conjunction with chemometrics, to predict curd moisture and whey fat content in a cheese vat was examined. A three-factor (renneting temperature, calcium chloride, cutting time), central composite design was carried out in triplicate. Spectra (300–1,100 nm) of the product in the cheese vat were captured during syneresis using a prototype light backscatter sensor. Stirring followed upon cutting the gel, and samples of curd and whey were removed at 10 min intervals and analyzed for curd moisture and whey fat content. Spectral data were used to develop models for predicting curd moisture and whey fat contents using partial least squares regression. Subjecting the spectral data set to Jack-knifing improved the accuracy of the models. The whey fat models (R = 0.91, 0.95) and curd moisture model (R = 0.86, 0.89) provided good and approximate predictions, respectively. Visible-near infrared spectroscopy was found to have potential for the prediction of important syneresis indices in stirred cheese vats.

Land cover classification of Lago Grande de Curuai floodplain (Amazon, Brazil) using multi-sensor and image fusion techniques

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Robot navigation in sensor space

This thesis investigates the problem of robot navigation using only landmark bearings. The proposed system allows a robot to move to a ground target location specified by the sensor values observed at this ground target posi- tion. The control actions are computed based on the difference between the current landmark bearings and the target landmark bearings. No Cartesian coordinates with respect to the ground are computed by the control system. The robot navigates using solely information from the bearing sensor space. Most existing robot navigation systems require a ground frame (2D Cartesian coordinate system) in order to navigate from a ground point A to a ground point B. The commonly used sensors such as laser range scanner, sonar, infrared, and vision do not directly provide the 2D ground coordi- nates of the robot. The existing systems use the sensor measurements to localise the robot with respect to a map, a set of 2D coordinates of the objects of interest. It is more natural to navigate between the points in the sensor space corresponding to A and B without requiring the Cartesian map and the localisation process. Research on animals has revealed how insects are able to exploit very limited computational and memory resources to successfully navigate to a desired destination without computing Cartesian positions. For example, a honeybee balances the left and right optical flows to navigate in a nar- row corridor. Unlike many other ants, Cataglyphis bicolor does not secrete pheromone trails in order to find its way home but instead uses the sun as a compass to keep track of its home direction vector. The home vector can be inaccurate, so the ant also uses landmark recognition. More precisely, it takes snapshots and compass headings of some landmarks. To return home, the ant tries to line up the landmarks exactly as they were before it started wandering. This thesis introduces a navigation method based on reflex actions in sensor space. The sensor vector is made of the bearings of some landmarks, and the reflex action is a gradient descent with respect to the distance in sensor space between the current sensor vector and the target sensor vec- tor. Our theoretical analysis shows that except for some fully characterized pathological cases, any point is reachable from any other point by reflex action in the bearing sensor space provided the environment contains three landmarks and is free of obstacles. The trajectories of a robot using reflex navigation, like other image- based visual control strategies, do not correspond necessarily to the shortest paths on the ground, because the sensor error is minimized, not the moving distance on the ground. However, we show that the use of a sequence of waypoints in sensor space can address this problem. In order to identify relevant waypoints, we train a Self Organising Map (SOM) from a set of observations uniformly distributed with respect to the ground. This SOM provides a sense of location to the robot, and allows a form of path planning in sensor space. The navigation proposed system is analysed theoretically, and evaluated both in simulation and with experiments on a real robot.

Improved detection and tracking of objects in surveillance video

Surveillance networks are typically monitored by a few people, viewing several monitors displaying the camera feeds. It is then very difficult for a human operator to effectively detect events as they happen. Recently, computer vision research has begun to address ways to automatically process some of this data, to assist human operators. Object tracking, event recognition, crowd analysis and human identification at a distance are being pursued as a means to aid human operators and improve the security of areas such as transport hubs. The task of object tracking is key to the effective use of more advanced technologies. To recognize an event people and objects must be tracked. Tracking also enhances the performance of tasks such as crowd analysis or human identification. Before an object can be tracked, it must be detected. Motion segmentation techniques, widely employed in tracking systems, produce a binary image in which objects can be located. However, these techniques are prone to errors caused by shadows and lighting changes. Detection routines often fail, either due to erroneous motion caused by noise and lighting effects, or due to the detection routines being unable to split occluded regions into their component objects. Particle filters can be used as a self contained tracking system, and make it unnecessary for the task of detection to be carried out separately except for an initial (often manual) detection to initialise the filter. Particle filters use one or more extracted features to evaluate the likelihood of an object existing at a given point each frame. Such systems however do not easily allow for multiple objects to be tracked robustly, and do not explicitly maintain the identity of tracked objects. This dissertation investigates improvements to the performance of object tracking algorithms through improved motion segmentation and the use of a particle filter. A novel hybrid motion segmentation / optical flow algorithm, capable of simultaneously extracting multiple layers of foreground and optical flow in surveillance video frames is proposed. The algorithm is shown to perform well in the presence of adverse lighting conditions, and the optical flow is capable of extracting a moving object. The proposed algorithm is integrated within a tracking system and evaluated using the ETISEO (Evaluation du Traitement et de lInterpretation de Sequences vidEO - Evaluation for video understanding) database, and significant improvement in detection and tracking performance is demonstrated when compared to a baseline system. A Scalable Condensation Filter (SCF), a particle filter designed to work within an existing tracking system, is also developed. The creation and deletion of modes and maintenance of identity is handled by the underlying tracking system; and the tracking system is able to benefit from the improved performance in uncertain conditions arising from occlusion and noise provided by a particle filter. The system is evaluated using the ETISEO database. The dissertation then investigates fusion schemes for multi-spectral tracking systems. Four fusion schemes for combining a thermal and visual colour modality are evaluated using the OTCBVS (Object Tracking and Classification in and Beyond the Visible Spectrum) database. It is shown that a middle fusion scheme yields the best results and demonstrates a significant improvement in performance when compared to a system using either mode individually. Findings from the thesis contribute to improve the performance of semi-automated video processing and therefore improve security in areas under surveillance.

Design and evaluation of an image analysis platform for low-power, low-bandwidth camera networks

We describe the design and evaluation of a platform for networks of cameras in low-bandwidth, low-power sensor networks. In our work to date we have investigated two different DSP hardware/software platforms for undertaking the tasks of compression and object detection and tracking. We compare the relative merits of each of the hardware and software platforms in terms of both performance and energy consumption. Finally we discuss what we believe are the ongoing research questions for image processing in WSNs.