Modelling stereoscopic vision systems for robotic applications

Aquesta tesi s'emmarca dins del projecte CICYT TAP 1999-0443-C05-01. L'objectiu d'aquest projecte és el disseny, implementació i avaluació de robots mòbils, amb un sistema de control distribuït, sistemes de sensorització i xarxa de comunicacions per realitzar tasques de vigilància. Els robots han de poder-se moure per un entorn reconeixent la posició i orientació dels diferents objectes que l'envolten. Aquesta informació ha de permetre al robot localitzar-se dins de l'entorn on es troba per poder-se moure evitant els possibles obstacles i dur a terme la tasca encomanada. El robot ha de generar un mapa dinàmic de l'entorn que serà utilitzat per localitzar la seva posició. L'objectiu principal d'aquest projecte és aconseguir que un robot explori i construeixi un mapa de l'entorn sense la necessitat de modificar el propi entorn. Aquesta tesi està enfocada en l'estudi de la geometria dels sistemes de visió estereoscòpics formats per dues càmeres amb l'objectiu d'obtenir informació geomètrica 3D de l'entorn d'un vehicle. Aquest objectiu tracta de l'estudi del modelatge i la calibració de càmeres i en la comprensió de la geometria epipolar. Aquesta geometria està continguda en el que s'anomena emph{matriu fonamental}. Cal realitzar un estudi del càlcul de la matriu fonamental d'un sistema estereoscòpic amb la finalitat de reduir el problema de la correspondència entre dos plans imatge. Un altre objectiu és estudiar els mètodes d'estimació del moviment basats en la geometria epipolar diferencial per tal de percebre el moviment del robot i obtenir-ne la posició. Els estudis de la geometria que envolta els sistemes de visió estereoscòpics ens permeten presentar un sistema de visió per computador muntat en un robot mòbil que navega en un entorn desconegut. El sistema fa que el robot sigui capaç de generar un mapa dinàmic de l'entorn a mesura que es desplaça i determinar quin ha estat el moviment del robot per tal de emph{localitzar-se} dins del mapa. La tesi presenta un estudi comparatiu dels mètodes de calibració de càmeres més utilitzats en les últimes dècades. Aquestes tècniques cobreixen un gran ventall dels mètodes de calibració clàssics. Aquest mètodes permeten estimar els paràmetres de la càmera a partir d'un conjunt de punts 3D i de les seves corresponents projeccions 2D en una imatge. Per tant, aquest estudi descriu un total de cinc tècniques de calibració diferents que inclouen la calibració implicita respecte l'explicita i calibració lineal respecte no lineal. Cal remarcar que s'ha fet un gran esforç en utilitzar la mateixa nomenclatura i s'ha estandaritzat la notació en totes les tècniques presentades. Aquesta és una de les dificultats principals a l'hora de poder comparar les tècniques de calibració ja què cada autor defineix diferents sistemes de coordenades i diferents conjunts de paràmetres. El lector és introduït a la calibració de càmeres amb la tècnica lineal i implícita proposada per Hall i amb la tècnica lineal i explicita proposada per Faugeras-Toscani. A continuació es passa a descriure el mètode a de Faugeras incloent el modelatge de la distorsió de les lents de forma radial. Seguidament es descriu el conegut mètode proposat per Tsai, i finalment es realitza una descripció detallada del mètode de calibració proposat per Weng. Tots els mètodes són comparats tant des del punt de vista de model de càmera utilitzat com de la precisió de la calibració. S'han implementat tots aquests mètodes i s'ha analitzat la precisió presentant resultats obtinguts tant utilitzant dades sintètiques com càmeres reals. Calibrant cada una de les càmeres del sistema estereoscòpic es poden establir un conjunt de restriccions geomètri ques entre les dues imatges. Aquestes relacions són el que s'anomena geometria epipolar i estan contingudes en la matriu fonamental. Coneixent la geometria epipolar es pot: simplificar el problema de la correspondència reduint l'espai de cerca a llarg d'una línia epipolar; estimar el moviment d'una càmera quan aquesta està muntada sobre un robot mòbil per realitzar tasques de seguiment o de navegació; reconstruir una escena per aplicacions d'inspecció, propotipatge o generació de motlles. La matriu fonamental s'estima a partir d'un conjunt de punts en una imatges i les seves correspondències en una segona imatge. La tesi presenta un estat de l'art de les tècniques d'estimació de la matriu fonamental. Comença pels mètode lineals com el dels set punts o el mètode dels vuit punts, passa pels mètodes iteratius com el mètode basat en el gradient o el CFNS, fins arribar las mètodes robustos com el M-Estimators, el LMedS o el RANSAC. En aquest treball es descriuen fins a 15 mètodes amb 19 implementacions diferents. Aquestes tècniques són comparades tant des del punt de vista algorísmic com des del punt de vista de la precisió que obtenen. Es presenten el resultats obtinguts tant amb imatges reals com amb imatges sintètiques amb diferents nivells de soroll i amb diferent quantitat de falses correspondències. Tradicionalment, l'estimació del moviment d'una càmera està basada en l'aplicació de la geometria epipolar entre cada dues imatges consecutives. No obstant el cas tradicional de la geometria epipolar té algunes limitacions en el cas d'una càmera situada en un robot mòbil. Les diferencies entre dues imatges consecutives són molt petites cosa que provoca inexactituds en el càlcul de matriu fonamental. A més cal resoldre el problema de la correspondència, aquest procés és molt costós en quant a temps de computació i no és gaire efectiu per aplicacions de temps real. En aquestes circumstàncies les tècniques d'estimació del moviment d'una càmera solen basar-se en el flux òptic i en la geometria epipolar diferencial. En la tesi es realitza un recull de totes aquestes tècniques degudament classificades. Aquests mètodes són descrits unificant la notació emprada i es remarquen les semblances i les diferencies entre el cas discret i el cas diferencial de la geometria epipolar. Per tal de poder aplicar aquests mètodes a l'estimació de moviment d'un robot mòbil, aquest mètodes generals que estimen el moviment d'una càmera amb sis graus de llibertat, han estat adaptats al cas d'un robot mòbil que es desplaça en una superfície plana. Es presenten els resultats obtinguts tant amb el mètodes generals de sis graus de llibertat com amb els adaptats a un robot mòbil utilitzant dades sintètiques i seqüències d'imatges reals. Aquest tesi finalitza amb una proposta de sistema de localització i de construcció d'un mapa fent servir un sistema estereoscòpic situat en un robot mòbil. Diverses aplicacions de robòtica mòbil requereixen d'un sistema de localització amb l'objectiu de facilitar la navegació del vehicle i l'execució del les trajectòries planificades. La localització es sempre relativa al mapa de l'entorn on el robot s'està movent. La construcció de mapes en un entorn desconegut és una tasca important a realitzar per les futures generacions de robots mòbils. El sistema que es presenta realitza la localització i construeix el mapa de l'entorn de forma simultània. A la tesi es descriu el robot mòbil GRILL, que ha estat la plataforma de treball emprada per aquesta aplicació, amb el sistema de visió estereoscòpic que s'ha dissenyat i s'ha muntat en el robot. També es descriu tots el processos que intervenen en el sistema de localització i construcció del mapa. La implementació d'aquest processos ha estat possible gràcies als estudis realitzats i presentats prèviament (calibració de càmeres, estimació de la matriu fonamental, i estimació del moviment) sense els quals no s'hauria pogut plantejar aquest sistema. Finalment es presenten els mapes en diverses trajectòries realitzades pel robot GRILL en el laboratori. Les principals contribucions d'aquest treball són: ·Un estat de l'art sobre mètodes de calibració de càmeres. El mètodes són comparats tan des del punt de vista del model de càmera utilitzat com de la precisió dels mètodes. ·Un estudi dels mètodes d'estimació de la matriu fonamental. Totes les tècniques estudiades són classificades i descrites des d'un punt de vista algorísmic. ·Un recull de les tècniques d'estimació del moviment d'una càmera centrat en el mètodes basat en la geometria epipolar diferencial. Aquestes tècniques han estat adaptades per tal d'estimar el moviment d'un robot mòbil. ·Una aplicació de robòtica mòbil per tal de construir un mapa dinàmic de l'entorn i localitzar-se per mitja d'un sistema estereoscòpic. L'aplicació presentada es descriu tant des del punt de vista del maquinari com del programari que s'ha dissenyat i implementat.

Pose Refinement of Active Models using Forces in 3D

A new algorithm is described for refining the pose of a model of a rigid object, to conform more accurately to the image structure. Elemental 3D forces are considered to act on the model. These are derived from directional derivatives of the image local to the projected model features. The convergence properties of the algorithm is investigated and compared to a previous technique. Its use in a video sequence of a cluttered outdoor traffic scene is also illustrated and assessed.

A Newton method for pose refinement of 3D models

Different optimization methods can be employed to optimize a numerical estimate for the match between an instantiated object model and an image. In order to take advantage of gradient-based optimization methods, perspective inversion must be used in this context. We show that convergence can be very fast by extrapolating to maximum goodness-of-fit with Newton's method. This approach is related to methods which either maximize a similar goodness-of-fit measure without use of gradient information, or else minimize distances between projected model lines and image features. Newton's method combines the accuracy of the former approach with the speed of convergence of the latter.

Variational data assimilation for parameter estimation: application to a simple morphodynamic model

Data assimilation is a sophisticated mathematical technique for combining observational data with model predictions to produce state and parameter estimates that most accurately approximate the current and future states of the true system. The technique is commonly used in atmospheric and oceanic modelling, combining empirical observations with model predictions to produce more accurate and well-calibrated forecasts. Here, we consider a novel application within a coastal environment and describe how the method can also be used to deliver improved estimates of uncertain morphodynamic model parameters. This is achieved using a technique known as state augmentation. Earlier applications of state augmentation have typically employed the 4D-Var, Kalman filter or ensemble Kalman filter assimilation schemes. Our new method is based on a computationally inexpensive 3D-Var scheme, where the specification of the error covariance matrices is crucial for success. A simple 1D model of bed-form propagation is used to demonstrate the method. The scheme is capable of recovering near-perfect parameter values and, therefore, improves the capability of our model to predict future bathymetry. Such positive results suggest the potential for application to more complex morphodynamic models.

QUAGMIRE v1.3: a quasi-geostrophic model for investigating rotating fluids experiments

QUAGMIRE is a quasi-geostrophic numerical model for performing fast, high-resolution simulations of multi-layer rotating annulus laboratory experiments on a desktop personal computer. The model uses a hybrid finite-difference/spectral approach to numerically integrate the coupled nonlinear partial differential equations of motion in cylindrical geometry in each layer. Version 1.3 implements the special case of two fluid layers of equal resting depths. The flow is forced either by a differentially rotating lid, or by relaxation to specified streamfunction or potential vorticity fields, or both. Dissipation is achieved through Ekman layer pumping and suction at the horizontal boundaries, including the internal interface. The effects of weak interfacial tension are included, as well as the linear topographic beta-effect and the quadratic centripetal beta-effect. Stochastic forcing may optionally be activated, to represent approximately the effects of random unresolved features. A leapfrog time stepping scheme is used, with a Robert filter. Flows simulated by the model agree well with those observed in the corresponding laboratory experiments.

Cue combination for 3D location judgements

Cue combination rules have often been applied to the perception of surface shape but not to judgements of object location. Here, we used immersive virtual reality to explore the relationship between different cues to distance. Participants viewed a virtual scene and judged the change in distance of an object presented in two intervals, where the scene changed in size between intervals (by a factor of between 0.25 and 4). We measured thresholds for detecting a change in object distance when there were only 'physical' (stereo and motion parallax) or 'texture-based' cues (independent of the scale of the scene) and used these to predict biases in a distance matching task. Under a range of conditions, in which the viewing distance and position of the tarte relative to other objects was varied, the ration of 'physical' to 'texture-based' thresholds was a good predictor of biases in the distance matching task. The cue combination approach, which successfully accounts for our data, relies on quite different principles from those underlying geometric reconstruction.

Rapid model quality assessment for protein structure predictions using the comparison of multiple models without structural alignments

MOTIVATION: The accurate prediction of the quality of 3D models is a key component of successful protein tertiary structure prediction methods. Currently, clustering or consensus based Model Quality Assessment Programs (MQAPs) are the most accurate methods for predicting 3D model quality; however they are often CPU intensive as they carry out multiple structural alignments in order to compare numerous models. In this study, we describe ModFOLDclustQ - a novel MQAP that compares 3D models of proteins without the need for CPU intensive structural alignments by utilising the Q measure for model comparisons. The ModFOLDclustQ method is benchmarked against the top established methods in terms of both accuracy and speed. In addition, the ModFOLDclustQ scores are combined with those from our older ModFOLDclust method to form a new method, ModFOLDclust2, that aims to provide increased prediction accuracy with negligible computational overhead. RESULTS: The ModFOLDclustQ method is competitive with leading clustering based MQAPs for the prediction of global model quality, yet it is up to 150 times faster than the previous version of the ModFOLDclust method at comparing models of small proteins (<60 residues) and over 5 times faster at comparing models of large proteins (>800 residues). Furthermore, a significant improvement in accuracy can be gained over the previous clustering based MQAPs by combining the scores from ModFOLDclustQ and ModFOLDclust to form the new ModFOLDclust2 method, with little impact on the overall time taken for each prediction. AVAILABILITY: The ModFOLDclustQ and ModFOLDclust2 methods are available to download from: http://www.reading.ac.uk/bioinf/downloads/ CONTACT: l.j.mcguffin@reading.ac.uk.

Prediction of global and local model quality in CASP8 using the ModFOLD server

The development of effective methods for predicting the quality of three-dimensional (3D) models is fundamentally important for the success of tertiary structure (TS) prediction strategies. Since CASP7, the Quality Assessment (QA) category has existed to gauge the ability of various model quality assessment programs (MQAPs) at predicting the relative quality of individual 3D models. For the CASP8 experiment, automated predictions were submitted in the QA category using two methods from the ModFOLD server-ModFOLD version 1.1 and ModFOLDclust. ModFOLD version 1.1 is a single-model machine learning based method, which was used for automated predictions of global model quality (QMODE1). ModFOLDclust is a simple clustering based method, which was used for automated predictions of both global and local quality (QMODE2). In addition, manual predictions of model quality were made using ModFOLD version 2.0-an experimental method that combines the scores from ModFOLDclust and ModFOLD v1.1. Predictions from the ModFOLDclust method were the most successful of the three in terms of the global model quality, whilst the ModFOLD v1.1 method was comparable in performance to other single-model based methods. In addition, the ModFOLDclust method performed well at predicting the per-residue, or local, model quality scores. Predictions of the per-residue errors in our own 3D models, selected using the ModFOLD v2.0 method, were also the most accurate compared with those from other methods. All of the MQAPs described are publicly accessible via the ModFOLD server at: http://www.reading.ac.uk/bioinf/ModFOLD/. The methods are also freely available to download from: http://www.reading.ac.uk/bioinf/downloads/.

From a discrete to a continuum model of cell dynamics in one dimension

Multiscale modeling is emerging as one of the key challenges in mathematical biology. However, the recent rapid increase in the number of modeling methodologies being used to describe cell populations has raised a number of interesting questions. For example, at the cellular scale, how can the appropriate discrete cell-level model be identified in a given context? Additionally, how can the many phenomenological assumptions used in the derivation of models at the continuum scale be related to individual cell behavior? In order to begin to address such questions, we consider a discrete one-dimensional cell-based model in which cells are assumed to interact via linear springs. From the discrete equations of motion, the continuous Rouse [P. E. Rouse, J. Chem. Phys. 21, 1272 (1953)] model is obtained. This formalism readily allows the definition of a cell number density for which a nonlinear "fast" diffusion equation is derived. Excellent agreement is demonstrated between the continuum and discrete models. Subsequently, via the incorporation of cell division, we demonstrate that the derived nonlinear diffusion model is robust to the inclusion of more realistic biological detail. In the limit of stiff springs, where cells can be considered to be incompressible, we show that cell velocity can be directly related to cell production. This assumption is frequently made in the literature but our derivation places limits on its validity. Finally, the model is compared with a model of a similar form recently derived for a different discrete cell-based model and it is shown how the different diffusion coefficients can be understood in terms of the underlying assumptions about cell behavior in the respective discrete models.

Benchmarking consensus model quality assessment for protein fold recognition

Background: Selecting the highest quality 3D model of a protein structure from a number of alternatives remains an important challenge in the field of structural bioinformatics. Many Model Quality Assessment Programs (MQAPs) have been developed which adopt various strategies in order to tackle this problem, ranging from the so called "true" MQAPs capable of producing a single energy score based on a single model, to methods which rely on structural comparisons of multiple models or additional information from meta-servers. However, it is clear that no current method can separate the highest accuracy models from the lowest consistently. In this paper, a number of the top performing MQAP methods are benchmarked in the context of the potential value that they add to protein fold recognition. Two novel methods are also described: ModSSEA, which based on the alignment of predicted secondary structure elements and ModFOLD which combines several true MQAP methods using an artificial neural network. Results: The ModSSEA method is found to be an effective model quality assessment program for ranking multiple models from many servers, however further accuracy can be gained by using the consensus approach of ModFOLD. The ModFOLD method is shown to significantly outperform the true MQAPs tested and is competitive with methods which make use of clustering or additional information from multiple servers. Several of the true MQAPs are also shown to add value to most individual fold recognition servers by improving model selection, when applied as a post filter in order to re-rank models. Conclusion: MQAPs should be benchmarked appropriately for the practical context in which they are intended to be used. Clustering based methods are the top performing MQAPs where many models are available from many servers; however, they often do not add value to individual fold recognition servers when limited models are available. Conversely, the true MQAP methods tested can often be used as effective post filters for re-ranking few models from individual fold recognition servers and further improvements can be achieved using a consensus of these methods.

Syntheses of two new 1D and 3D networks of Cu(II) and Co(II) using malonate and urotropine as bridging ligands: Crystal structures and magnetic studies

Two new metal-organic based polymeric complexes, [Cu-4(O2CCH2CO2)(4)(L)].7H(2)O (1) and [CO2(O2CCH2CO2)(2)(L)].2H(2)O (2) [L = hexamethylenetetramine (urotropine)], have been synthesized and characterized by X-ray crystal structure determination and magnetic studies. Complex 1 is a 1D coordination polymer comprising a carboxylato, bridged Cu-4 moiety linked by a tetradentate bridging urotropine. Complex 2 is a 3D coordination polymer made of pseudo-two-dimensional layers of Co(II) ions linked by malonate anions in syn-anticonformation which are bridged by bidentate urotropine in trans fashion, Complex 1 crystallizes in the orthothombic system, space group Pmmn, with a = 14,80(2) Angstrom, b = 14.54(2) Angstrom, c = 7.325(10) Angstrom, beta = 90degrees, and Z = 4. Complex 2 crystallizes in the orthorhombic system, space group Imm2, a = 7.584(11) Angstrom, b = 15.80(2) Angstrom, c = 6.939(13) Angstrom, beta = 90.10degrees(1), and Z = 4. Variable temperature (300-2 K) magnetic behavior reveals the existence of ferro- and antiferromagnetic interactions in 1 and only antiferromagnetic interactions in 2. The best fitted parameters for complex 1 are J = 13.5 cm(-1), J = -18.1 cm(-1), and g = 2.14 considering only intra-Cu-4 interactions through carboxylate and urotropine pathways. In case of complex 2, the fit of the magnetic data considering intralayer interaction through carboxylate pathway as well as interlayer interaction via urotropine pathway gave no satisfactory result at this moment using any model known due to considerable orbital contribution of Co(II) ions to the magnetic moment and its complicated structure. Assuming isolated Co(II) ions (without any coupling, J = 0) the shape of the chi(M)T curve fits well with experimental data except at very low temperatures.

Can misalignments in typical infants be used as a model for infantile esotropia?

PURPOSE. To investigate the nature of early ocular misalignments in human infants to determine whether they can provide insight into the etiology of esotropia and, in particular, to examine the correlates of misalignments. METHODS. A remote haploscopic photorefraction system was used to measure accommodation and vergence in 146 infants between 0 and 12 months of age. Infants underwent photorefraction immediately after watching a target moving between two of five viewing distances (25, 33, 50, 100, and 200 cm). In some instances, infants were tested in two conditions: both eyes open and one eye occluded. The resultant data were screened for instances of large misalignments. Data were assessed to determine whether accommodative, retinal disparity, or other cues were associated with the occurrence of misalignments. RESULTS. The results showed that there was no correlation between accommodative behavior and misalignments. Infants were more likely to show misalignments when retinal disparity cues were removed through occlusion. They were also more likely to show misalignments immediately after the target moved from a near to a far position in comparison to far-to-near target movement. DISCUSSION. The data suggest that the prevalence of misalignments in infants of 2 to 3 months of age is decreased by the addition of retinal disparity cues to the stimulus. In addition, target movement away from the infant increases the prevalence of misalignments. These data are compatible with the notion that misalignment are caused by poor sensitivity to targets moving away from the infant and support the theory that some forms of strabismus could be related to failure in a system that is sensitive to the direction of motion.

Humans ignore motion and stereo cues in favor of a fictional stable world

As we move through the world, our eyes acquire a sequence of images. The information from this sequence is sufficient to determine the structure of a three-dimensional scene, up to a scale factor determined by the distance that the eyes have moved [1, 2]. Previous evidence shows that the human visual system accounts for the distance the observer has walked [3,4] and the separation of the eyes [5-8] when judging the scale, shape, and distance of objects. However, in an immersive virtual-reality environment, observers failed to notice when a scene expanded or contracted, despite having consistent information about scale from both distance walked and binocular vision. This failure led to large errors in judging the size of objects. The pattern of errors cannot be explained by assuming a visual reconstruction of the scene with an incorrect estimate of interocular separation or distance walked. Instead, it is consistent with a Bayesian model of cue integration in which the efficacy of motion and disparity cues is greater at near viewing distances. Our results imply that observers are more willing to adjust their estimate of interocular separation or distance walked than to accept that the scene has changed in size.

Application of constrained predictive control on a 3D crane system

This paper describes the SIMULINK implementation of a constrained predictive control algorithm based on quadratic programming and linear state space models, and its application to a laboratory-scale 3D crane system. The algorithm is compatible with Real Time. Windows Target and, in the case of the crane system, it can be executed with a sampling period of 0.01 s and a prediction horizon of up to 300 samples, using a linear state space model with 3 inputs, 5 outputs and 13 states.

MIMO Wiener model identification for large scale fading of wireless mobile communications links

The large scale fading of wireless mobile communications links is modelled assuming the mobile receiver motion is described by a dynamic linear system in state-space. The geometric relations involved in the attenuation and multi-path propagation of the electric field are described by a static non-linear mapping. A Wiener system subspace identification algorithm in conjunction with polynomial regression is used to identify a model from time-domain estimates of the field intensity assuming a multitude of emitters and an antenna array at the receiver end.