Super-resolution of remotely sensed images with variable-pixel linear reconstruction

This paper describes the development and applications of a super-resolution method, known as Super-Resolution Variable-Pixel Linear Reconstruction. The algorithm works combining different lower resolution images in order to obtain, as a result, a higher resolution image. We show that it can make significant spatial resolution improvements to satellite images of the Earth¿s surface allowing recognition of objects with size approaching the limiting spatial resolution of the lower resolution images. The algorithm is based on the Variable-Pixel Linear Reconstruction algorithm developed by Fruchter and Hook, a well-known method in astronomy but never used for Earth remote sensing purposes. The algorithm preserves photometry, can weight input images according to the statistical significance of each pixel, and removes the effect of geometric distortion on both image shape and photometry. In this paper, we describe its development for remote sensing purposes, show the usefulness of the algorithm working with images as different to the astronomical images as the remote sensing ones, and show applications to: 1) a set of simulated multispectral images obtained from a real Quickbird image; and 2) a set of multispectral real Landsat Enhanced Thematic Mapper Plus (ETM+) images. These examples show that the algorithm provides a substantial improvement in limiting spatial resolution for both simulated and real data sets without significantly altering the multispectral content of the input low-resolution images, without amplifying the noise, and with very few artifacts.

Role of color in face recognition

One of the key challenges in face perception lies in determining the contribution of different cues to face identification. In this study, we focus on the role of color cues. Although color appears to be a salient attribute of faces, past research has suggested that it confers little recognition advantage for identifying people. Here we report experimental results suggesting that color cues do play a role in face recognition and their contribution becomes evident when shape cues are degraded. Under such conditions, recognition performance with color images is significantly better than that with grayscale images. Our experimental results also indicate that the contribution of color may lie not so much in providing diagnostic cues to identity as in aiding low-level image-analysis processes such as segmentation.

Relative Contributions of Internal and External Features to Face Recognition

The central challenge in face recognition lies in understanding the role different facial features play in our judgments of identity. Notable in this regard are the relative contributions of the internal (eyes, nose and mouth) and external (hair and jaw-line) features. Past studies that have investigated this issue have typically used high-resolution images or good-quality line drawings as facial stimuli. The results obtained are therefore most relevant for understanding the identification of faces at close range. However, given that real-world viewing conditions are rarely optimal, it is also important to know how image degradations, such as loss of resolution caused by large viewing distances, influence our ability to use internal and external features. Here, we report experiments designed to address this issue. Our data characterize how the relative contributions of internal and external features change as a function of image resolution. While we replicated results of previous studies that have shown internal features of familiar faces to be more useful for recognition than external features at high resolution, we found that the two feature sets reverse in importance as resolution decreases. These results suggest that the visual system uses a highly non-linear cue-fusion strategy in combining internal and external features along the dimension of image resolution and that the configural cues that relate the two feature sets play an important role in judgments of facial identity.

Depth-based face recognition using local quantized patterns adapted for range data

A depth-based face recognition algorithm specially adapted to high range resolution data acquired by the new Microsoft Kinect 2 sensor is presented. A novel descriptor called Depth Local Quantized Pattern descriptor has been designed to make use of the extended range resolution of the new sensor. This descriptor is a substantial modification of the popular Local Binary Pattern algorithm. One of the main contributions is the introduction of a quantification step, increasing its capacity to distinguish different depth patterns. The proposed descriptor has been used to train and test a Support Vector Machine classifier, which has proven to be able to accurately recognize different people faces from a wide range of poses. In addition, a new depth-based face database acquired by the new Kinect 2 sensor have been created and made public to evaluate the proposed face recognition system.

A novel polar-based human face recognition computational model

Motivated by a recently proposed biologically inspired face recognition approach, we investigated the relation between human behavior and a computational model based on Fourier-Bessel (FB) spatial patterns. We measured human recognition performance of FB filtered face images using an 8-alternative forced-choice method. Test stimuli were generated by converting the images from the spatial to the FB domain, filtering the resulting coefficients with a band-pass filter, and finally taking the inverse FB transformation of the filtered coefficients. The performance of the computational models was tested using a simulation of the psychophysical experiment. In the FB model, face images were first filtered by simulated V1- type neurons and later analyzed globally for their content of FB components. In general, there was a higher human contrast sensitivity to radially than to angularly filtered images, but both functions peaked at the 11.3-16 frequency interval. The FB-based model presented similar behavior with regard to peak position and relative sensitivity, but had a wider frequency band width and a narrower response range. The response pattern of two alternative models, based on local FB analysis and on raw luminance, strongly diverged from the human behavior patterns. These results suggest that human performance can be constrained by the type of information conveyed by polar patterns, and consequently that humans might use FB-like spatial patterns in face processing.

High throughput 3D super-resolution microscopy reveals Caulobacter crescentus in vivo Z-ring organization.

We created a high-throughput modality of photoactivated localization microscopy (PALM) that enables automated 3D PALM imaging of hundreds of synchronized bacteria during all stages of the cell cycle. We used high-throughput PALM to investigate the nanoscale organization of the bacterial cell division protein FtsZ in live Caulobacter crescentus. We observed that FtsZ predominantly localizes as a patchy midcell band, and only rarely as a continuous ring, supporting a model of "Z-ring" organization whereby FtsZ protofilaments are randomly distributed within the band and interact only weakly. We found evidence for a previously unidentified period of rapid ring contraction in the final stages of the cell cycle. We also found that DNA damage resulted in production of high-density continuous Z-rings, which may obstruct cytokinesis. Our results provide a detailed quantitative picture of in vivo Z-ring organization.

Prediction of difficult intubation with advanced face recognition techniques: a preliminary study

Introduction: Difficult tracheal intubation remains a constant and significant source of morbidity and mortality in anaesthetic practice. Insufficient airway assessment in the preoperative period continues to be a major cause of unanticipated difficult intubation. Although many risk factors have already been identified, preoperative airway evaluation is not always regarded as a standard procedure and the respective weight of each risk factor remains unclear. Moreover the predictive scores available are not sensitive, moderately specific and often operator-dependant. In order to improve the preoperative detection of patients at risk for difficult intubation, we developed a system for automated and objective evaluation of morphologic criteria of the face and neck using video recordings and advanced techniques borrowed from face recognition. Method and results: Frontal video sequences were recorded in 5 healthy volunteers. During the video recording, subjects were requested to perform maximal flexion-extension of the neck and to open wide the mouth with tongue pulled out. A robust and real-time face tracking system was then applied, allowing to automatically identify and map a grid of 55 control points on the face, which were tracked during head motion. These points located important features of the face, such as the eyebrows, the nose, the contours of the eyes and mouth, and the external contours, including the chin. Moreover, based on this face tracking, the orientation of the head could also be estimated at each frame of the video sequence. Thus, we could infer for each frame the pitch angle of the head pose (related to the vertical rotation of the head) and obtain the degree of head extension. Morphological criteria used in the most frequent cited predictive scores were also extracted, such as mouth opening, degree of visibility of the uvula or thyreo-mental distance. Discussion and conclusion: Preliminary results suggest the high feasibility of the technique. The next step will be the application of the same automated and objective evaluation to patients who will undergo tracheal intubation. The difficulties related to intubation will be then correlated to the biometric characteristics of the patients. The objective in mind is to analyze the biometrics data with artificial intelligence algorithms to build a highly sensitive and specific predictive test.

Empirical mode decomposition-based face recognition system

In this work we explore the multivariate empirical mode decomposition combined with a Neural Network classifier as technique for face recognition tasks. Images are simultaneously decomposed by means of EMD and then the distance between the modes of the image and the modes of the representative image of each class is calculated using three different distance measures. Then, a neural network is trained using 10- fold cross validation in order to derive a classifier. Preliminary results (over 98 % of classification rate) are satisfactory and will justify a deep investigation on how to apply mEMD for face recognition.

An efficient total variation algorithm for super-resolution in fetal brain MRI with adaptive regularization.

Although fetal anatomy can be adequately viewed in new multi-slice MR images, many critical limitations remain for quantitative data analysis. To this end, several research groups have recently developed advanced image processing methods, often denoted by super-resolution (SR) techniques, to reconstruct from a set of clinical low-resolution (LR) images, a high-resolution (HR) motion-free volume. It is usually modeled as an inverse problem where the regularization term plays a central role in the reconstruction quality. Literature has been quite attracted by Total Variation energies because of their ability in edge preserving but only standard explicit steepest gradient techniques have been applied for optimization. In a preliminary work, it has been shown that novel fast convex optimization techniques could be successfully applied to design an efficient Total Variation optimization algorithm for the super-resolution problem. In this work, two major contributions are presented. Firstly, we will briefly review the Bayesian and Variational dual formulations of current state-of-the-art methods dedicated to fetal MRI reconstruction. Secondly, we present an extensive quantitative evaluation of our SR algorithm previously introduced on both simulated fetal and real clinical data (with both normal and pathological subjects). Specifically, we study the robustness of regularization terms in front of residual registration errors and we also present a novel strategy for automatically select the weight of the regularization as regards the data fidelity term. Our results show that our TV implementation is highly robust in front of motion artifacts and that it offers the best trade-off between speed and accuracy for fetal MRI recovery as in comparison with state-of-the art methods.

Shared feature extraction for nearest neighbor face recognition

In this paper, we propose a new supervised linearfeature extraction technique for multiclass classification problemsthat is specially suited to the nearest neighbor classifier (NN).The problem of finding the optimal linear projection matrix isdefined as a classification problem and the Adaboost algorithmis used to compute it in an iterative way. This strategy allowsthe introduction of a multitask learning (MTL) criterion in themethod and results in a solution that makes no assumptions aboutthe data distribution and that is specially appropriated to solvethe small sample size problem. The performance of the methodis illustrated by an application to the face recognition problem.The experiments show that the representation obtained followingthe multitask approach improves the classic feature extractionalgorithms when using the NN classifier, especially when we havea few examples from each class

Combining Near-Field and Far-Field Microscopy: A new method for nanoscale super-resolution imaging

Lähikenttä- ja kaukokenttämikroskopian yhdistäminen: Uusi korkearesoluutioinen menetelmä nanokuvantamiseen. Osteoporoosi on sairaus, jossa luun uudistumisprosessi ei ole enää tasapainossa. Uuden luun muodostuminen on hitaampaa johtuen osteoblastien laskeneesta aktiivisuudesta. Yksi keino estää osteoporoosin syntyä on estää osteoklastien sitoutuminen luun pinnalle, jolloin ne eivät aloita luun syömisprosessia. Tämän Pro gradu -tutkielman tarkoituksena on luoda uusi työkalu osteoklastien sitoutumisen tutkimiseen samanaikaisesti fluoresenssi- ja atomivoimamikroskoopilla. Tätä tarkoitusta varten yhdistettiin atomivoimamikroskooppi sekä STED mikroskooppi. Kirjallisuuskatsauksessa käydään läpi yksityiskohtaisesti molempien mikroskooppitekniikoiden teoriat. Kokeellisessa osiossa esitetään käytetyt metodit ja alustavat tulokset uudella systeemillä. Lisäksi keskustellaan lyhyesti kuvan analysoinnista ImageJohjelmalla. Konfokaalisen fluoresenssimikroskoopin ja atomivoimamikroskoopin yhdistelmä on keksitty jo aikaisemmin, mutta tavallisen konfokaalimikroskoopin erottelukyvyn raja on noin 200 nanometriä johtuen valon diffraktioluonteesta. Yksityiskohdat eivät erotu, jos ne ovat pienempiä kuin puolet käytettävästä aallonpituudesta. STED mikroskooppi mahdollistaa fluoresenssikuvien taltioimisen solunsisäisistä prosesseista 50 nanometrin lateraalisella erotuskyvyllä ja atomivoimamikroskooppi antaa topografista tietoa näytteestä nanometrien erotuskyvyllä. Biologisia näytteitä kuvannettaessa atomivoimamikroskoopin erotuskyky kuitenkin huononee ja yleensä saavutetaan 30-50 nanometrin erotuskyky. Kuvien kerrostaminen vaatii vertauspisteitä ja tätä varten käytettiin atomivoimamikroskoopin kärjen tunnistamista ja referenssipartikkeleita. Kuva-analysointi suoritettiin ImageJ-kuvankäsittelyohjelmalla. Tuloksista nähdään, että referenssipartikkelit ovat hyviä, mutta niiden sijoittaminen tarkasti tietylle kohdealueelle on hankalaa nanoskaalassa. Tästä johtuen kärjen havaitseminen fluoresenssikuvassa on parempi metodi. Atomivoimamikroskoopin kärki voidaan päällystää fluoresoivalla aineella, mutta tämä lisää kärjen aiheuttamaa konvoluutiota mittausdataan. Myös valon takaisinsirontaa kärjestä voidaan tutkia, jolloin konvoluutio ei lisäänny. Ensimmäisten kuvien kerrostamisessa käytettiin hyväksi fluoresoivalla aineella päällystettyä kärkeä ja lopputuloksessa oli vain 50 nanometrin yhteensopimattomuus fluoresenssi- ja topografiakuvan kanssa. STED mikroskoopin avulla nähdään leimattujen proteiinien tarkat sijainnit tiettynä ajankohtana elävän solun sisällä. Samaan aikaan pystytään kuvantamaan solun fyysisiä muotoja tai mitata adheesiovoimia atomivoimamikroskoopilla. Lisäksi voidaan käyttää funktinalisoitua kärkeä, jolla voidaan laukaista signalointitapahtumia solun ja soluväliaineen välillä. Sitoutuminen soluväliaineeseen voidaan rekisteröidä samoin kuin adheesiomediaattorien sijainnit sitoutumisalueella. Nämä dynaamiset havainnot tuottavat uutta informaatiota solun signaloinnista, kun osteoklasti kiinnittyy luun pintaan. Tämä teknologia tarjoaa uuden näkökulman monimutkaisiin signalointiprosesseihin nanoskaalassa ja tulee ratkaisemaan lukemattoman määrän biologisia ongelmia.

A novel polar-based human face recognition computational model

Motivated by a recently proposed biologically inspired face recognition approach, we investigated the relation between human behavior and a computational model based on Fourier-Bessel (FB) spatial patterns. We measured human recognition performance of FB filtered face images using an 8-alternative forced-choice method. Test stimuli were generated by converting the images from the spatial to the FB domain, filtering the resulting coefficients with a band-pass filter, and finally taking the inverse FB transformation of the filtered coefficients. The performance of the computational models was tested using a simulation of the psychophysical experiment. In the FB model, face images were first filtered by simulated V1- type neurons and later analyzed globally for their content of FB components. In general, there was a higher human contrast sensitivity to radially than to angularly filtered images, but both functions peaked at the 11.3-16 frequency interval. The FB-based model presented similar behavior with regard to peak position and relative sensitivity, but had a wider frequency band width and a narrower response range. The response pattern of two alternative models, based on local FB analysis and on raw luminance, strongly diverged from the human behavior patterns. These results suggest that human performance can be constrained by the type of information conveyed by polar patterns, and consequently that humans might use FB-like spatial patterns in face processing.