Vision-based vehicle detection and tracking with a mobile camera using a statistical framework

En esta tesis se aborda la detección y el seguimiento automático de vehículos mediante técnicas de visión artificial con una cámara monocular embarcada. Este problema ha suscitado un gran interés por parte de la industria automovilística y de la comunidad científica ya que supone el primer paso en aras de la ayuda a la conducción, la prevención de accidentes y, en última instancia, la conducción automática. A pesar de que se le ha dedicado mucho esfuerzo en los últimos años, de momento no se ha encontrado ninguna solución completamente satisfactoria y por lo tanto continúa siendo un tema de investigación abierto. Los principales problemas que plantean la detección y seguimiento mediante visión artificial son la gran variabilidad entre vehículos, un fondo que cambia dinámicamente debido al movimiento de la cámara, y la necesidad de operar en tiempo real. En este contexto, esta tesis propone un marco unificado para la detección y seguimiento de vehículos que afronta los problemas descritos mediante un enfoque estadístico. El marco se compone de tres grandes bloques, i.e., generación de hipótesis, verificación de hipótesis, y seguimiento de vehículos, que se llevan a cabo de manera secuencial. No obstante, se potencia el intercambio de información entre los diferentes bloques con objeto de obtener el máximo grado posible de adaptación a cambios en el entorno y de reducir el coste computacional. Para abordar la primera tarea de generación de hipótesis, se proponen dos métodos complementarios basados respectivamente en el análisis de la apariencia y la geometría de la escena. Para ello resulta especialmente interesante el uso de un dominio transformado en el que se elimina la perspectiva de la imagen original, puesto que este dominio permite una búsqueda rápida dentro de la imagen y por tanto una generación eficiente de hipótesis de localización de los vehículos. Los candidatos finales se obtienen por medio de un marco colaborativo entre el dominio original y el dominio transformado. Para la verificación de hipótesis se adopta un método de aprendizaje supervisado. Así, se evalúan algunos de los métodos de extracción de características más populares y se proponen nuevos descriptores con arreglo al conocimiento de la apariencia de los vehículos. Para evaluar la efectividad en la tarea de clasificación de estos descriptores, y dado que no existen bases de datos públicas que se adapten al problema descrito, se ha generado una nueva base de datos sobre la que se han realizado pruebas masivas. Finalmente, se presenta una metodología para la fusión de los diferentes clasificadores y se plantea una discusión sobre las combinaciones que ofrecen los mejores resultados. El núcleo del marco propuesto está constituido por un método Bayesiano de seguimiento basado en filtros de partículas. Se plantean contribuciones en los tres elementos fundamentales de estos filtros: el algoritmo de inferencia, el modelo dinámico y el modelo de observación. En concreto, se propone el uso de un método de muestreo basado en MCMC que evita el elevado coste computacional de los filtros de partículas tradicionales y por consiguiente permite que el modelado conjunto de múltiples vehículos sea computacionalmente viable. Por otra parte, el dominio transformado mencionado anteriormente permite la definición de un modelo dinámico de velocidad constante ya que se preserva el movimiento suave de los vehículos en autopistas. Por último, se propone un modelo de observación que integra diferentes características. En particular, además de la apariencia de los vehículos, el modelo tiene en cuenta también toda la información recibida de los bloques de procesamiento previos. El método propuesto se ejecuta en tiempo real en un ordenador de propósito general y da unos resultados sobresalientes en comparación con los métodos tradicionales. ABSTRACT This thesis addresses on-road vehicle detection and tracking with a monocular vision system. This problem has attracted the attention of the automotive industry and the research community as it is the first step for driver assistance and collision avoidance systems and for eventual autonomous driving. Although many effort has been devoted to address it in recent years, no satisfactory solution has yet been devised and thus it is an active research issue. The main challenges for vision-based vehicle detection and tracking are the high variability among vehicles, the dynamically changing background due to camera motion and the real-time processing requirement. In this thesis, a unified approach using statistical methods is presented for vehicle detection and tracking that tackles these issues. The approach is divided into three primary tasks, i.e., vehicle hypothesis generation, hypothesis verification, and vehicle tracking, which are performed sequentially. Nevertheless, the exchange of information between processing blocks is fostered so that the maximum degree of adaptation to changes in the environment can be achieved and the computational cost is alleviated. Two complementary strategies are proposed to address the first task, i.e., hypothesis generation, based respectively on appearance and geometry analysis. To this end, the use of a rectified domain in which the perspective is removed from the original image is especially interesting, as it allows for fast image scanning and coarse hypothesis generation. The final vehicle candidates are produced using a collaborative framework between the original and the rectified domains. A supervised classification strategy is adopted for the verification of the hypothesized vehicle locations. In particular, state-of-the-art methods for feature extraction are evaluated and new descriptors are proposed by exploiting the knowledge on vehicle appearance. Due to the lack of appropriate public databases, a new database is generated and the classification performance of the descriptors is extensively tested on it. Finally, a methodology for the fusion of the different classifiers is presented and the best combinations are discussed. The core of the proposed approach is a Bayesian tracking framework using particle filters. Contributions are made on its three key elements: the inference algorithm, the dynamic model and the observation model. In particular, the use of a Markov chain Monte Carlo method is proposed for sampling, which circumvents the exponential complexity increase of traditional particle filters thus making joint multiple vehicle tracking affordable. On the other hand, the aforementioned rectified domain allows for the definition of a constant-velocity dynamic model since it preserves the smooth motion of vehicles in highways. Finally, a multiple-cue observation model is proposed that not only accounts for vehicle appearance but also integrates the available information from the analysis in the previous blocks. The proposed approach is proven to run near real-time in a general purpose PC and to deliver outstanding results compared to traditional methods.

Anomaly Detection Using Remote Sensing for the Archaeological Heritage Registration

The aim of this work is an approach using multisensor remote sensing techniques to recognize the potential remains and recreate the original landscape of three archaeological sites. We investigate the spectral characteristics of the reflectance parameter and emissivity in the pattern recognition of archaeological materials in several hyperspectral scenes of the prehispanic site in Palmar Sur (Costa Rica), the Jarama Valley site and the celtiberian city of Segeda in Spain. Spectral ranges of the visible-near infrared (VNIR), shortwave infrared (SWIR) and thermal infrared (TIR) from hyperspectral data cubes of HyMAP, AHS, MASTER and ATM have been used. Several experiments on natural scenarios of Costa Rica and Spain of different complexity, have been designed. Spectral patterns and thermal anomalies have been calculated as evidences of buried remains and change detection. First results, land cover change analyses and their consequences in the digital heritage registration are discussed.

Detection of residue contacts in a protein folding intermediate

Protein folding can be described in terms of the development of specific contacts between residues as a highly disordered polypeptide chain converts into the native state. Here we describe an NMR based strategy designed to detect such contacts by observation of nuclear Overhauser effects (NOEs). Experiments with α-lactalbumin reveal the existence of extensive NOEs between aromatic and aliphatic protons in the archetypal molten globule formed by this protein at low pH. Analysis of their time development provides direct evidence for near-native compactness of this state. Through a rapid refolding procedure the NOE intensity can be transferred efficiently into the resolved and assigned spectrum of the native state. This demonstrates the viability of using this approach to map out time-averaged interactions between residues in a partially folded protein.

High Resolution Detection of Mechanical Forces Exerted by Locomoting Fibroblasts on the Substrate

We have developed a new approach to detect mechanical forces exerted by locomoting fibroblasts on the substrate. Cells were cultured on elastic, collagen-coated polyacrylamide sheets embedded with 0.2-μm fluorescent beads. Forces exerted by the cell cause deformation of the substrate and displacement of the beads. By recording the position of beads during cell locomotion and after cell removal, we discovered that most forces were radially distributed, switching direction in the anterior region. Deformations near the leading edge were strong, transient, and variable in magnitude, consistent with active local contractions, whereas those in the posterior region were weaker, more stable, and more uniform, consistent with passive resistance. Treatment of cells with cytochalasin D or myosin II inhibitors caused relaxation of the forces, suggesting that they are generated primarily via actin–myosin II interactions; treatment with nocodazole caused no immediate effect on forces. Immunofluorescence indicated that the frontal region of strong deformation contained many vinculin plaques but no apparent concentration of actin or myosin II filaments. Strong mechanical forces in the anterior region, generated by locally activated myosin II and transmitted through vinculin-rich structures, likely play a major role in cell locomotion and in mechanical signaling with the surrounding environment.

Detection of a Yb3+ binding site in regenerated bacteriorhodopsin that is coordinated with the protein and phospholipid head groups

Near infrared Yb3+ vibronic sideband spectroscopy was used to characterize specific lanthanide binding sites in bacteriorhodopsin (bR) and retinal free bacteriorhodopsin (bO). The VSB spectra for deionized bO regenerated with a ratio of 1:1 and 2:1 ion to bO are identical. Application of a two-dimensional anti-correlation technique suggests that only a single Yb3+ site is observed. The Yb3+ binding site in bO is observed to consist of PO2− groups and carboxylic acid groups, both of which are bound in a bidentate manner. An additional contribution most likely arising from a phenolic group is also observed. This implies that the ligands for the observed single binding site are the lipid head groups and amino acid residues. The vibronic sidebands of Yb3+ in deionized bR regenerated at a ratio of 2:1 ion to bR are essentially identical to those in bO. The other high-affinity binding site is thus either not evident or its fluorescence is quenched. A discussion is given on the difference in binding of Ca2+ (or Mg2+) and lanthanides in phospholipid membrane proteins.

Is blindsight like normal, near-threshold vision?

Blindsight is the rare and paradoxical ability of some human subjects with occipital lobe brain damage to discriminate unseen stimuli in their clinically blind field defects when forced-choice procedures are used, implying that lesions of striate cortex produce a sharp dissociation between visual performance and visual awareness. Skeptics have argued that this is no different from the behavior of normal subjects at the lower limits of conscious vision, at which such dissociations could arise trivially by using different response criteria during clinical and forced-choice tests. We tested this claim explicitly by measuring the sensitivity of a hemianopic patient independently of his response criterion in yes-no and forced-choice detection tasks with the same stimulus and found that, unlike normal controls, his sensitivity was significantly higher during the forced-choice task. Thus, the dissociation by which blindsight is defined is not simply due to a difference in the patients’ response bias between the two paradigms. This result implies that blindsight is unlike normal, near-threshold vision and that information about the stimulus is processed in blindsighted patients in an unusual way.

Detection of Ca2+ entry through mechanosensitive channels localizes the site of mechanoelectrical transduction in hair cells.

A hair cell, the sensory receptor of the internal ear, transduces mechanical stimuli into electrical responses. Transduction results from displacement of the hair bundle, a cluster of rod-shaped stereocilia extending from the cell's apical surface. Biophysical experiments indicate that, by producing shear between abutting stereocilia, a bundle displacement directly opens cation-selective transduction channels. Specific models of gating depend on the location of these channels, which has been controversial: although some physiological and immunocytochemical experiments have situated the transduction channels at the hair bundle's top, monitoring of fluorescence signals from the Ca2+ indicator fura-2 has instead suggested that Ca2+ traverses channels at the bundle's base. To examine the site of Ca2+ entry through transduction channels, we used laser-scanning confocal microscopy, with a spatial resolution of < 1 micron and a temporal resolution of < 2 ms, to observe hair cells filled with the indicator fluo-3. An unstimulated hair cell showed a "tip blush" of enhanced fluorescence at the hair bundle's top, which we attribute to Ca2+ permeation through transduction channels open at rest. Upon mechanical stimulation, individual stereocilia displayed increased fluorescence that originated near their tips, then spread toward their bases. Our results confirm that mechanoelectrical transduction occurs near stereociliary tips.

Biogeochemical parameters of sea water and bottom sediments near the Curieuse Island (Seychelles Islands)

Biogeochemical reef studies carried out in 1981 and 1984 found low concentration of total natural and anthropogenic hydrocarbons in inshore waters. Detection of lignin in marine and bottom sediments indicates that the land has major effect on makeup of organic matter there. Comparison of compositions of organic matter in sea water, suspended matter and bottom sediments indicated that it was altered rapidly by the reef community. Thus, in the inshore zone of the island, runoff from the land is important in supplying nutrients to the reef ecosystem alongside with transport of nutrients by deep waters. Concentrations of nutri¬ents (N, P) in the inshore zone are higher than in waters of the tropical part of the ocean. Nitrogen is the limiting element in development of phytoplankton in the inshore zone.

Improved detection in CDMA for biased sources

We consider the detection of biased information sources in the ubiquitous code-division multiple-access (CDMA) scheme. We propose a simple modification to both the popular single-user matched-filter detector and a recently introduced near-optimal message-passing-based multiuser detector. This modification allows for detecting modulated biased sources directly with no need for source coding. Analytical results and simulations with excellent agreement are provided, demonstrating substantial improvement in bit error rate in comparison with the unmodified detectors and the alternative of source compression. The robustness of error-performance improvement is shown under practical model settings, including bias estimation mismatch and finite-length spreading codes. © 2007 IOP Publishing Ltd.

Low level feature detection in human vision

Influential models of edge detection have generally supposed that an edge is detected at peaks in the 1st derivative of the luminance profile, or at zero-crossings in the 2nd derivative. However, when presented with blurred triangle-wave images, observers consistently marked edges not at these locations, but at peaks in the 3rd derivative. This new phenomenon, termed ‘Mach edges’ persisted when a luminance ramp was added to the blurred triangle-wave. Modelling of these Mach edge detection data required the addition of a physiologically plausible filter, prior to the 3rd derivative computation. A viable alternative model was examined, on the basis of data obtained with short-duration, high spatial-frequency stimuli. Detection and feature-making methods were used to examine the perception of Mach bands in an image set that spanned a range of Mach band detectabilities. A scale-space model that computed edge and bar features in parallel provided a better fit to the data than 4 competing models that combined information across scale in a different manner, or computed edge or bar features at a single scale. The perception of luminance bars was examined in 2 experiments. Data for one image-set suggested a simple rule for perception of a small Gaussian bar on a larger inverted Gaussian bar background. In previous research, discriminability (d’) has typically been reported to be a power function of contrast, where the exponent (p) is 2 to 3. However, using bar, grating, and Gaussian edge stimuli, with several methodologies, values of p were obtained that ranged from 1 to 1.7 across 6 experiments. This novel finding was explained by appealing to low stimulus uncertainty, or a near-linear transducer.

Refractive index sensitivity characteristics near the dispersion turning point of the multimode microfiber-based Mach–Zehnder interferometer

The turning point of the refractive index (RI) sensitivity based on the multimode microfiber (MMMF) in-line Mach–Zehnder interferometer (MZI) is observed. By tracking the resonant wavelength shift of the MZI generated between the HE11 and HE12 modes in the MMMF, the surrounding RI (SRI) could be detected. Theoretical analysis demonstrates that the RI sensitivity will reach ±∞ on either side of the turning point due to the group effective RI difference (퐺) approaching zero. Significantly, the positive sensitivity exists in a very wide fiber diameter range, while the negative sensitivity can be achieved in a narrow diameter range of only 0.3 μm. Meanwhile, the experimental sensitivities and variation trend at different diameters exhibit high consistency with the theoretical results. High RI sensitivity of 10777.8 nm/RIU (RI unit) at the fiber diameter of 4.6 μm and the RI around 1.3334 is realized. The discovery of the sensitivity turning points has great significance on trace detection due to the possibility of ultrahigh RI sensitivity.

Sensitivity and Bias within the Binary Signal Detection Theory, BSDT

Similar to classic Signal Detection Theory (SDT), recent optimal Binary Signal Detection Theory (BSDT) and based on it Neural Network Assembly Memory Model (NNAMM) can successfully reproduce Receiver Operating Characteristic (ROC) curves although BSDT/NNAMM parameters (intensity of cue and neuron threshold) and classic SDT parameters (perception distance and response bias) are essentially different. In present work BSDT/NNAMM optimal likelihood and posterior probabilities are analytically analyzed and used to generate ROCs and modified (posterior) mROCs, optimal overall likelihood and posterior. It is shown that for the description of basic discrimination experiments in psychophysics within the BSDT a ‘neural space’ can be introduced where sensory stimuli as neural codes are represented and decision processes are defined, the BSDT’s isobias curves can simultaneously be interpreted as universal psychometric functions satisfying the Neyman-Pearson objective, the just noticeable difference (jnd) can be defined and interpreted as an atom of experience, and near-neutral values of biases are observers’ natural choice. The uniformity or no-priming hypotheses, concerning the ‘in-mind’ distribution of false-alarm probabilities during ROC or overall probability estimations, is introduced. The BSDT’s and classic SDT’s sensitivity, bias, their ROC and decision spaces are compared.

A sensing mechanism for the detection of carbon nanotubes using selective photoluminescent probes based on ionic complexes with organic dyes

The multifunctional properties of carbon nanotubes (CNTs) make them a powerful platform for unprecedented innovations in a variety of practical applications. As a result of the surging growth of nanotechnology, nanotubes present a potential problem as an environmental pollutant, and as such, an efficient method for their rapid detection must be established. Here, we propose a novel type of ionic sensor complex for detecting CNTs – an organic dye that responds sensitively and selectively to CNTs with a photoluminescent signal. The complexes are formed through Coulomb attractions between dye molecules with uncompensated charges and CNTs covered with an ionic surfactant in water. We demonstrate that the photoluminescent excitation of the dye can be transferred to the nanotubes, resulting in selective and strong amplification (up to a factor of 6) of the light emission from the excitonic levels of CNTs in the near-infrared spectral range, as experimentally observed via excitation-emission photoluminescence (PL) mapping. The chirality of the nanotubes and the type of ionic surfactant used to disperse the nanotubes both strongly affect the amplification; thus, the complexation provides sensing selectivity towards specific CNTs. Additionally, neither similar uncharged dyes nor CNTs covered with neutral surfactant form such complexes. As model organic molecules, we use a family of polymethine dyes with an easily tailorable molecular structure and, consequently, tunable absorbance and PL characteristics. This provides us with a versatile tool for the controllable photonic and electronic engineering of an efficient probe for CNT detection.

System-on-a-Chip (SOC) based environmental monitoring platform for the detection of hazardous materials

The purpose of this research is design considerations for environmental monitoring platforms for the detection of hazardous materials using System-on-a-Chip (SoC) design. Design considerations focus on improving key areas such as: (1) sampling methodology; (2) context awareness; and (3) sensor placement. These design considerations for environmental monitoring platforms using wireless sensor networks (WSN) is applied to the detection of methylmercury (MeHg) and environmental parameters affecting its formation (methylation) and deformation (demethylation). ^ The sampling methodology investigates a proof-of-concept for the monitoring of MeHg using three primary components: (1) chemical derivatization; (2) preconcentration using the purge-and-trap (P&T) method; and (3) sensing using Quartz Crystal Microbalance (QCM) sensors. This study focuses on the measurement of inorganic mercury (Hg) (e.g., Hg2+) and applies lessons learned to organic Hg (e.g., MeHg) detection. ^ Context awareness of a WSN and sampling strategies is enhanced by using spatial analysis techniques, namely geostatistical analysis (i.e., classical variography and ordinary point kriging), to help predict the phenomena of interest in unmonitored locations (i.e., locations without sensors). This aids in making more informed decisions on control of the WSN (e.g., communications strategy, power management, resource allocation, sampling rate and strategy, etc.). This methodology improves the precision of controllability by adding potentially significant information of unmonitored locations.^ There are two types of sensors that are investigated in this study for near-optimal placement in a WSN: (1) environmental (e.g., humidity, moisture, temperature, etc.) and (2) visual (e.g., camera) sensors. The near-optimal placement of environmental sensors is found utilizing a strategy which minimizes the variance of spatial analysis based on randomly chosen points representing the sensor locations. Spatial analysis is employed using geostatistical analysis and optimization occurs with Monte Carlo analysis. Visual sensor placement is accomplished for omnidirectional cameras operating in a WSN using an optimal placement metric (OPM) which is calculated for each grid point based on line-of-site (LOS) in a defined number of directions where known obstacles are taken into consideration. Optimal areas of camera placement are determined based on areas generating the largest OPMs. Statistical analysis is examined by using Monte Carlo analysis with varying number of obstacles and cameras in a defined space. ^

Functionalized carbon micro/nanostructures for biomolecular detection

Advancements in the micro-and nano-scale fabrication techniques have opened up new avenues for the development of portable, scalable and easier-to-use biosensors. Over the last few years, electrodes made of carbon have been widely used as sensing units in biosensors due to their attractive physiochemical properties. The aim of this research is to investigate different strategies to develop functionalized high surface carbon micro/nano-structures for electrochemical and biosensing devices. High aspect ratio three-dimensional carbon microarrays were fabricated via carbon microelectromechanical systems (C-MEMS) technique, which is based on pyrolyzing pre-patterned organic photoresist polymers. To further increase the surface area of the carbon microstructures, surface porosity was introduced by two strategies, i.e. (i) using F127 as porogen and (ii) oxygen reactive ion etch (RIE) treatment. Electrochemical characterization showed that porous carbon thin film electrodes prepared by using F127 as porogen had an effective surface area (Aeff 185%) compared to the conventional carbon electrode. To achieve enhanced electrochemical sensitivity for C-MEMS based functional devices, graphene was conformally coated onto high aspect ratio three-dimensional (3D) carbon micropillar arrays using electrostatic spray deposition (ESD) technique. The amperometric response of graphene/carbon micropillar electrode arrays exhibited higher electrochemical activity, improved charge transfer and a linear response towards H2O2 detection between 250&mgr;M to 5.5mM. Furthermore, carbon structures with dimensions from 50 nano-to micrometer level have been fabricated by pyrolyzing photo-nanoimprint lithography patterned organic resist polymer. Microstructure, elemental composition and resistivity characterization of the carbon nanostructures produced by this process were very similar to conventional photoresist derived carbon. Surface functionalization of the carbon nanostructures was performed using direct amination technique. Considering the need for requisite functional groups to covalently attach bioreceptors on the carbon surface for biomolecule detection, different oxidation techniques were compared to study the types of carbon-oxygen groups formed on the surface and their percentages with respect to different oxidation pretreatment times. Finally, a label-free detection strategy using signaling aptamer/protein binding complex for platelet-derived growth factor oncoprotein detection on functionalized three-dimensional carbon microarrays platform was demonstrated. The sensor showed near linear relationship between the relative fluorescence difference and protein concentration even in the sub-nanomolar range with an excellent detection limit of 5 pmol.