Resection-intersection bundle adjustment revisited

Bundle adjustment is one of the essential components of the computer vision toolbox. This paper revisits the resection-intersection approach, which has previously been shown to have inferior convergence properties. Modifications are proposed that greatly improve the performance of this method, resulting in a fast and accurate approach. Firstly, a linear triangulation step is added to the intersection stage, yielding higher accuracy and improved convergence rate. Secondly, the effect of parameter updates is tracked in order to reduce wasteful computation; only variables coupled to significantly changing variables are updated. This leads to significant improvements in computation time, at the cost of a small, controllable increase in error. Loop closures are handled effectively without the need for additional network modelling. The proposed approach is shown experimentally to yield comparable accuracy to a full sparse bundle adjustment (20% error increase) while computation time scales much better with the number of variables. Experiments on a progressive reconstruction system show the proposed method to be more efficient by a factor of 65 to 177, and 4.5 times more accurate (increasing over time) than a localised sparse bundle adjustment approach.

Understanding the abnormal brain activity in epilepsy as a potential predictor of the onset of an epileptic seizure

The Brain Research Institute (BRI) uses various types of indirect measurements, including EEG and fMRI, to understand and assess brain activity and function. As well as the recovery of generic information about brain function, research also focuses on the utilisation of such data and understanding to study the initiation, dynamics, spread and suppression of epileptic seizures. To assist with the future focussing of this aspect of their research, the BRI asked the MISG 2010 participants to examine how the available EEG and fMRI data and current knowledge about epilepsy should be analysed and interpreted to yield an enhanced understanding about brain activity occurring before, at commencement of, during, and after a seizure. Though the deliberations of the study group were wide ranging in terms of the related matters considered and discussed, considerable progress was made with the following three aspects. (1) The science behind brain activity investigations depends crucially on the quality of the analysis and interpretation of, as well as the recovery of information from, EEG and fMRI measurements. A number of specific methodologies were discussed and formalised, including independent component analysis, principal component analysis, profile monitoring and change point analysis (hidden Markov modelling, time series analysis, discontinuity identification). (2) Even though EEG measurements accurately and very sensitively record the onset of an epileptic event or seizure, they are, from the perspective of understanding the internal initiation and localisation, of limited utility. They only record neuronal activity in the cortical (surface layer) neurons of the brain, which is a direct reflection of the type of electrical activity they have been designed to record. Because fMRI records, through the monitoring of blood flow activity, the location of localised brain activity within the brain, the possibility of combining fMRI measurements with EEG, as a joint inversion activity, was discussed and examined in detail. (3) A major goal for the BRI is to improve understanding about ``when'' (at what time) an epileptic seizure actually commenced before it is identified on an eeg recording, ``where'' the source of this initiation is located in the brain, and ``what'' is the initiator. Because of the general agreement in the literature that, in one way or another, epileptic events and seizures represent abnormal synchronisations of localised and/or global brain activity the modelling of synchronisations was examined in some detail. References C. M. Michel, G. Thut, S. Morand, A. Khateb, A. J. Pegna, R. Grave de Peralta, S. Gonzalez, M. Seeck and T. Landis, Electric source imaging of human brain functions, Brain Res. Rev. , 36 (2--3), 2001, 108--118. doi:10.1016/S0165-0173(01)00086-8 S. Ogawa, R. S. Menon, S. G. Kim and K. Ugurbil, On the characteristics of functional magnetic resonance imaging of the brain, Annu. Rev. Bioph. Biom. , 27 , 1998, 447--474. doi:10.1146/annurev.biophys.27.1.447 C. D. Binnie and H. Stefan, Modern electroencephalography: its role in epilepsy management, Clin. Neurophysiol. , 110 (10), 1999, 1671--1697. doi:10.1016/S1388-2457(99)00125-X J. X. Tao, A. Ray, S. Hawes-Ebersole and J. S. Ebersole, Intracranial eeg substrates of scalp eeg interictal spikes, Epilepsia , 46 (5), 2005, 669--76. doi:10.1111/j.1528-1167.2005.11404.x S. Ogawa, D. W. Tank, R. Menon, J. M. Ellermann, S. G. Kim, H. Merkle and K. Ugurbil, Intrinsic signal changes accompanying sensory stimulation: Functional brain mapping with magnetic resonance imaging, P. Natl. Acad. Sci. USA , 89 (13), 1992, 5951--5955. doi:10.1073/pnas.89.13.5951 J. Engel Jr., Report of the ilae classification core group, Epilepsia , 47 (9), 2006, 1558--1568. doi:10.1111/j.1528-1167.2006.00215.x L. Lemieux, A. Salek-Haddadi, O. Josephs, P. Allen, N. Toms, C. Scott, K. Krakow, R. Turner and D. R. Fish, Event-related fmri with simultaneous and continuous eeg: description of the method and initial case r port, NeuroImage , 14 (3), 2001, 780--7. doi:10.1006/nimg.2001.0853 P. Federico, D. F. Abbott, R. S. Briellmann, A. S. Harvey and G. D. Jackson, Functional mri of the pre-ictal state, Brain , 128 (8), 2005, 1811-7. doi:10.1093/brain/awh533 C. S. Hawco, A. P. Bagshaw, Y. Lu, F. Dubeau and J. Gotman, bold changes occur prior to epileptic spikes seen on scalp eeg, NeuroImage , 35 (4), 2007, 1450--1458. doi:10.1016/j.neuroimage.2006.12.042 F. Moeller, H. R. Siebner, S. Wolff, H. Muhle, R. Boor, O. Granert, O. Jansen, U. Stephani and M. Siniatchkin, Changes in activity of striato-thalamo-cortical network precede generalized spike wave discharges, NeuroImage , 39 (4), 2008, 1839--1849. doi:10.1016/j.neuroimage.2007.10.058 V. Osharina, E. Ponchel, A. Aarabi, R. Grebe and F. Wallois, Local haemodynamic changes preceding interictal spikes: A simultaneous electrocorticography (ecog) and near-infrared spectroscopy (nirs) analysis in rats, NeuroImage , 50 (2), 2010, 600--607. doi:10.1016/j.neuroimage.2010.01.009 R. S. Fisher, W. Boas, W. Blume, C. Elger, P. Genton, P. Lee and J. Engel, Epileptic seizures and epilepsy: Definitions proposed by the international league against epilepsy (ilae) and the international bureau for epilepsy (ibe), Epilepsia , 46 (4), 2005, 470--472. doi:10.1111/j.0013-9580.2005.66104.x H. Berger, Electroencephalogram in humans, Arch. Psychiat. Nerven. , 87 , 1929, 527--570. C. M. Michel, M. M. Murray, G. Lantz, S. Gonzalez, L. Spinelli and R. G. de Peralta, eeg source imaging, Clin. Neurophysiol. , 115 (10), 2004, 2195--2222. doi:10.1016/j.clinph.2004.06.001 P. L. Nunez and R. B. Silberstein, On the relationship of synaptic activity to macroscopic measurements: Does co-registration of eeg with fmri make sense?, Brain Topogr. , 13 (2), 2000, 79--96. doi:10.1023/A:1026683200895 S. Ogawa, T. M. Lee, A. R. Kay and D. W. Tank, Brain magnetic resonance imaging with contrast dependent on blood oxygenation, P. Natl. Acad. Sci. USA , 87 (24), 1990, 9868--9872. doi:10.1073/pnas.87.24.9868 J. S. Gati, R. S. Menon, K. Ugurbil and B. K. Rutt, Experimental determination of the bold field strength dependence in vessels and tissue, Magn. Reson. Med. , 38 (2), 1997, 296--302. doi:10.1002/mrm.1910380220 P. A. Bandettini, E. C. Wong, R. S. Hinks, R. S. Tikofsky and J. S. Hyde, Time course EPI of human brain function during task activation, Magn. Reson. Med. , 25 (2), 1992, 390--397. K. K. Kwong, J. W. Belliveau, D. A. Chesler, I. E. Goldberg, R. M. Weisskoff, B. P. Poncelet, D. N. Kennedy, B. E. Hoppelm, M. S. Cohen and R. Turner, Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation, P. Natl. Acad. Sci. USA , 89 (12), 1992, 5675--5679. doi:10.1073/pnas.89.12.5675 J. Frahm, K. D. Merboldt and W. Hnicke, Functional mri of human brain activation at high spatial resolution, Magn. Reson. Med. , 29 (1), 1993, 139--144. P. A. Bandettini, A. Jesmanowicz, E. C. Wong and J. S. Hyde, Processing strategies for time-course data sets in functional MRI of the human brain, Magn. Reson. Med. , 30 (2), 1993, 161--173. K. J. Friston, P. Jezzard and R. Turner, Analysis of functional MRI time-series, Hum. Brain Mapp. , 1 (2), 1994, 153--171. B. Biswal, F. Z. Yetkin, V. M. Haughton and J. S. Hyde, Functional connectivity in the motor cortex of resting human brain using echo-planar mri, Mag. Reson. Med. , 34 (4), 1995, 537--541. doi:10.1002/mrm.1910340409 K. J. Friston, J. Ashburner, C. D. Frith, J. Poline, J. D. Heather and R. S. J. Frackowiak, Spatial registration and normalization of images, Hum. Brain Mapp. , 3 (3), 1995, 165--189. K. J. Friston, S. Williams, R. Howard, R. S. Frackowiak and R. Turner, Movement-related effects in fmri time-series, Magn. Reson. Med. , 35 (3), 1996, 346--355. G. H. Glover, T. Q. Li and D. Ress, Image-based method for retrospective correction of physiological motion effects in fmri: Retroicor, Magn. Reson. Med. , 44 (1), 2000, 162--167. doi:10.1002/1522-2594(200007)44:13.0.CO;2-E K. J. Friston, O. Josephs, G. Rees and R. Turner, Nonlinear event-related responses in fmri, Magn. Reson. Med. , 39 (1), 1998, 41--52. doi:10.1002/mrm.1910390109 K. Ugurbil, L. Toth and D. Kim, How accurate is magnetic resonance imaging of brain function?, Trends Neurosci. , 26 (2), 2003, 108--114. doi:10.1016/S0166-2236(02)00039-5 D. S. Kim, I. Ronen, C. Olman, S. G. Kim, K. Ugurbil and L. J. Toth, Spatial relationship between neuronal activity and bold functional mri, NeuroImage , 21 (3), 2004, 876--885. doi:10.1016/j.neuroimage.2003.10.018 A. Connelly, G. D. Jackson, R. S. Frackowiak, J. W. Belliveau, F. Vargha-Khadem and D. G. Gadian, Functional mapping of activated human primary cortex with a clinical mr imaging system, Radiology , 188 (1), 1993, 125--130. L. Allison, Hidden Markov Models, Technical Report , School of Computer and Software Engineering, Monash University, 2000. R. J. Elliott, L. Aggoun and J.B. Moore, Hidden Markov Models: Estimation and Control, Appl. Math.-Czech. , 2004. B. Bhavnagri, Discontinuities of plane functions projected from a surface with methods for finding these , Technical Report, 2009. B. Bhavnagri, Computer Vision using Shape Spaces , Technical Report,1996, University of Adelaide. B. Bhavnagri, A method for representing shape based on an equivalence relation on polygons, Pattern Recogn. , 27 (2), 1994, 247--260. doi:10.1016/0031-3203(94)90057-4 D. F. Abbott, A. B. Waites, A. S. Harvey and G. D. Jackson, Exploring epileptic seizure onset with fmri, NeuroImage , 36(S1) (344TH-PM), 2007. M. C. Mackey and L. Glass, Oscillation and chaos in physiological control systems, Science , 197 , 1977, 287--289. S. H. Strogatz, SYNC - The Emerging Science of Spontaneous Order , Theia, New York, 2003. J. W. Kim, J. A. Roberts and P. A. Robinson, Dynamics of epileptic seizures: Evolution, spreading, and suppression, J. Theor. Biol. , 257 (4), 2009, 527--532. doi:10.1016/j.jtbi.2008.12.009 Y. Kuramoto, T. Aoyagi, I. Nishikawa, T. Chawanya T and K. Okuda, Neural network model carrying phase information with application to collective dynamics, J. Theor. Phys. , 87 (5), 1992, 1119--1126. V. B. Mountcastle, The columnar organization of the neocortex, Brain , 120 (4), 1997, 701. doi:10.1093/brain/120.4.701 F. L. Silva, W. Blanes, S. N. Kalitzin, J. Parra, P. Suffczynski and D. N. Velis, Epilepsies as dynamical diseases of brain systems: Basic models of the transition between normal and epileptic activity, Epilepsia , 44 (12), 2003, 72--83. F. H. Lopes da Silva, W. Blanes, S. N. Kalitzin, J. Parra, P. Suffczynski and D. N. Velis, Dynamical diseases of brain systems: different routes to epileptic seizures, ieee T. Bio-Med. Eng. , 50 (5), 2003, 540. L.D. Iasemidis, Epileptic seizure prediction and control, ieee T. Bio-Med. Eng. , 50 (5), 2003, 549--558. L. D. Iasemidis, D. S. Shiau, W. Chaovalitwongse, J. C. Sackellares, P. M. Pardalos, J. C. Principe, P. R. Carney, A. Prasad, B. Veeramani, and K. Tsakalis, Adaptive epileptic seizure prediction system, ieee T. Bio-Med. Eng. , 50 (5), 2003, 616--627. K. Lehnertz, F. Mormann, T. Kreuz, R.G. Andrzejak, C. Rieke, P. David and C. E. Elger, Seizure prediction by nonlinear eeg analysis, ieee Eng. Med. Biol. , 22 (1), 2003, 57--63. doi:10.1109/MEMB.2003.1191451 K. Lehnertz, R. G. Andrzejak, J. Arnhold, T. Kreuz, F. Mormann, C. Rieke, G. Widman and C. E. Elger, Nonlinear eeg analysis in epilepsy: Its possible use for interictal focus localization, seizure anticipation, and prevention, J. Clin. Neurophysiol. , 18 (3), 2001, 209. B. Litt and K. Lehnertz, Seizure prediction and the preseizure period, Curr. Opin. Neurol. , 15 (2), 2002, 173. doi:10.1097/00019052-200204000-00008 B. Litt and J. Echauz, Prediction of epileptic seizures, Lancet Neurol. , 1 (1), 2002, 22--30. doi:10.1016/S1474-4422(02)00003-0 M. M{a}kiranta, J. Ruohonen, K Suominen, J. Niinim{a}ki, E. Sonkaj{a}rvi, V. Kiviniemi, T. Sepp{a}nen, S. Alahuhta, V. J{a}ntti and O. Tervonen, {bold} signal increase preceeds eeg spike activity--a dynamic penicillin induced focal epilepsy in deep anesthesia, NeuroImage , 27 (4), 2005, 715--724. doi:10.1016/j.neuroimage.2005.05.025 K. Lehnertz, F. Mormann, H. Osterhage, A. M{u}ller, J. Prusseit, A. Chernihovskyi, M. Staniek, D. Krug, S. Bialonski and C. E. Elger, State-of-the-art of seizure prediction, J. Clin. Neurophysiol. , 24 (2), 2007, 147. doi:10.1097/WNP.0b013e3180336f16 F. Mormann, T. Kreuz, C. Rieke, R. G. Andrzejak, A. Kraskov, P. David, C. E. Elger and K. Lehnertz, On the predictability of epileptic seizures, Clin. Neurophysiol. , 116 (3), 2005, 569--587. doi:10.1016/j.clinph.2004.08.025 F. Mormann, R. G. Andrzejak, C. E. Elger and K. Lehnertz, Seizure prediction: the long and winding road, Brain , 130 (2), 2007, 314--333. doi:10.1093/brain/awl241 Z. Rogowski, I. Gath and E. Bental, On the prediction of epileptic seizures, Biol. Cybern. , 42 (1), 1981, 9--15. Y. Salant, I. Gath, O. Henriksen, Prediction of epileptic seizures from two-channel eeg, Med. Biol. Eng. Comput. , 36 (5), 1998, 549--556. doi:10.1007/BF02524422 J. Gotman and D.J. Koffler, Interictal spiking increases after seizures but does not after decrease in medication, Evoked Potential , 72 (1), 1989, 7--15. J. Gotman and M. G. Marciani, Electroencephalographic spiking activity, drug levels, and seizure occurence in epileptic patients, Ann. Neurol. , 17 (6), 1985, 59--603. A. Katz, D. A. Marks, G. McCarthy and S. S. Spencer, Does interictal spiking change prior to seizures?, Electroen. Clin. Neuro. , 79 (2), 1991, 153--156. A. Granada, R. M. Hennig, B. Ronacher, A. Kramer and H. Herzel, Phase Response Curves: Elucidating the dynamics of couples oscillators, Method Enzymol. , 454 (A), 2009, 1--27. doi:10.1016/S0076-6879(08)03801-9 doi:10.1016/S0076-6879(08)03801-9 H. Kantz and T. Schreiber, Nonlinear time series analysis , 2004, Cambridge Univ Press. M. V. L. Bennett and R. S Zukin, Electrical coupling and neuronal synchronization in the mammalian brain, Neuron , 41 (4), 2004, 495 --511. doi:10.1016/S0896-6273(04)00043-1 L.D. Iasemidis, J. Chris Sackellares, H. P. Zaveri and W. J. Williams, Phase space topography and the Lyapunov exponent of electrocorticograms in partial seizures, Brain Topogr. , 2 (3), 1990, 187--201. doi:10.1007/BF01140588 M. Le Van Quyen, J. Martinerie, V. Navarro, M. Baulac and F. J. Varela, Characterizing neurodynamic changes before seizures, J. Clin. Neurophysiol. , 18 (3), 2001, 191. J. Martinerie, C. Adam, M. Le Van Quyen, M. Baulac, S. Clemenceau, B. Renault and F. J. Varela, Epileptic seizures can be anticipated by non-linear analysis, Nat. Med. , 4 (10), 1998, 1173--1176. doi:10.1038/2667 A. Pikovsky, M. Rosenblum, J. Kurths and R. C. Hilborn, Synchronization: A universal concept in nonlinear science, Amer. J. Phys. , 70 , 2002, 655. H. R. Wilson and J. D. Cowan, Excitatory and inhibitory interactions in localized populations of model neurons, Biophys. J. , 12 (1), 1972, 1--24. D. Cumin and C. P. Unsworth, Generalising the Kuramoto model for the study of neuronal synchronisation in the brain, Physica D , 226 (2), 2007, 181--196. doi:10.1016/j.physd.2006.12.004 F. K. Skinner, H. Bazzazi and S. A. Campbell, Two-cell to N-cell heterogeneous, inhibitory networks: Precise linking of multistable and coherent properties, J. Comput. Neurosci. , 18 (3), 2005, 343--352. doi:10.1007/s10827-005-0331-1 W. W. Lytton, Computer modelling of epilepsy, Nat. Rev. Neurosci. , 9 (8), 2008, 626--637. doi:10.1038/nrn2416 R. D. Traub, A. Bibbig, F. E. N. LeBeau, E. H. Buhl and M. A. Whittington, Cellular mechanisms of neuronal population oscillations in the hippocampus in vitro, Ann. Rev. , 2004. R. D. Traub, A. Draguhn, M. A. Whittington, T. Baldeweg, A. Bibbig, E. H. Buhl and D. Schmitz, Axonal gap junc ions between principal neurons: A novel source of network oscillations, and perhaps epileptogenesis., Rev. Neuroscience , 13 (1), 2002, 1. doi:10.1146/annurev.neuro.27.070203.144303 M. Scheffer, J. Bascompte, W. A. Brock, V. Brovkin, S. R. Carpenter, V. Dakos, H. Held, E. H. van Nes, M. Rietkerk and G. Sugihara, Early-warning signals for critical transitions, Nature , 461 (7260), 2009, 53--59. doi:10.1038/nature08227 K. Murphy, A Brief Introduction to Graphical Models and Bayesian Networks , 2008, http://www.cs.ubc.ca/murphyk/Bayes/bnintro.html . R. C. Bradley, An elementary

Random Gabor based templates for facial expression recognition in images with facial occlusion

Robust facial expression recognition (FER) under occluded face conditions is challenging. It requires robust algorithms of feature extraction and investigations into the effects of different types of occlusion on the recognition performance to gain insight. Previous FER studies in this area have been limited. They have spanned recovery strategies for loss of local texture information and testing limited to only a few types of occlusion and predominantly a matched train-test strategy. This paper proposes a robust approach that employs a Monte Carlo algorithm to extract a set of Gabor based part-face templates from gallery images and converts these templates into template match distance features. The resulting feature vectors are robust to occlusion because occluded parts are covered by some but not all of the random templates. The method is evaluated using facial images with occluded regions around the eyes and the mouth, randomly placed occlusion patches of different sizes, and near-realistic occlusion of eyes with clear and solid glasses. Both matched and mis-matched train and test strategies are adopted to analyze the effects of such occlusion. Overall recognition performance and the performance for each facial expression are investigated. Experimental results on the Cohn-Kanade and JAFFE databases demonstrate the high robustness and fast processing speed of our approach, and provide useful insight into the effects of occlusion on FER. The results on the parameter sensitivity demonstrate a certain level of robustness of the approach to changes in the orientation and scale of Gabor filters, the size of templates, and occlusions ratios. Performance comparisons with previous approaches show that the proposed method is more robust to occlusion with lower reductions in accuracy from occlusion of eyes or mouth.

Object tracking via non-Euclidean geometry : a Grassmann approach

A robust visual tracking system requires an object appearance model that is able to handle occlusion, pose, and illumination variations in the video stream. This can be difficult to accomplish when the model is trained using only a single image. In this paper, we first propose a tracking approach based on affine subspaces (constructed from several images) which are able to accommodate the abovementioned variations. We use affine subspaces not only to represent the object, but also the candidate areas that the object may occupy. We furthermore propose a novel approach to measure affine subspace-to-subspace distance via the use of non-Euclidean geometry of Grassmann manifolds. The tracking problem is then considered as an inference task in a Markov Chain Monte Carlo framework via particle filtering. Quantitative evaluation on challenging video sequences indicates that the proposed approach obtains considerably better performance than several recent state-of-the-art methods such as Tracking-Learning-Detection and MILtrack.

Random projections on manifolds of symmetric positive definite matrices for image classification

Recent advances suggest that encoding images through Symmetric Positive Definite (SPD) matrices and then interpreting such matrices as points on Riemannian manifolds can lead to increased classification performance. Taking into account manifold geometry is typically done via (1) embedding the manifolds in tangent spaces, or (2) embedding into Reproducing Kernel Hilbert Spaces (RKHS). While embedding into tangent spaces allows the use of existing Euclidean-based learning algorithms, manifold shape is only approximated which can cause loss of discriminatory information. The RKHS approach retains more of the manifold structure, but may require non-trivial effort to kernelise Euclidean-based learning algorithms. In contrast to the above approaches, in this paper we offer a novel solution that allows SPD matrices to be used with unmodified Euclidean-based learning algorithms, with the true manifold shape well-preserved. Specifically, we propose to project SPD matrices using a set of random projection hyperplanes over RKHS into a random projection space, which leads to representing each matrix as a vector of projection coefficients. Experiments on face recognition, person re-identification and texture classification show that the proposed approach outperforms several recent methods, such as Tensor Sparse Coding, Histogram Plus Epitome, Riemannian Locality Preserving Projection and Relational Divergence Classification.

Summarisation of short-term and long-term videos using texture and colour

We present a novel approach to video summarisation that makes use of a Bag-of-visual-Textures (BoT) approach. Two systems are proposed, one based solely on the BoT approach and another which exploits both colour information and BoT features. On 50 short-term videos from the Open Video Project we show that our BoT and fusion systems both achieve state-of-the-art performance, obtaining an average F-measure of 0.83 and 0.86 respectively, a relative improvement of 9% and 13% when compared to the previous state-of-the-art. When applied to a new underwater surveillance dataset containing 33 long-term videos, the proposed system reduces the amount of footage by a factor of 27, with only minor degradation in the information content. This order of magnitude reduction in video data represents significant savings in terms of time and potential labour cost when manually reviewing such footage.

Matching image sets via adaptive multi convex hull

Traditional nearest points methods use all the samples in an image set to construct a single convex or affine hull model for classification. However, strong artificial features and noisy data may be generated from combinations of training samples when significant intra-class variations and/or noise occur in the image set. Existing multi-model approaches extract local models by clustering each image set individually only once, with fixed clusters used for matching with various image sets. This may not be optimal for discrimination, as undesirable environmental conditions (eg. illumination and pose variations) may result in the two closest clusters representing different characteristics of an object (eg. frontal face being compared to non-frontal face). To address the above problem, we propose a novel approach to enhance nearest points based methods by integrating affine/convex hull classification with an adapted multi-model approach. We first extract multiple local convex hulls from a query image set via maximum margin clustering to diminish the artificial variations and constrain the noise in local convex hulls. We then propose adaptive reference clustering (ARC) to constrain the clustering of each gallery image set by forcing the clusters to have resemblance to the clusters in the query image set. By applying ARC, noisy clusters in the query set can be discarded. Experiments on Honda, MoBo and ETH-80 datasets show that the proposed method outperforms single model approaches and other recent techniques, such as Sparse Approximated Nearest Points, Mutual Subspace Method and Manifold Discriminant Analysis.

Automatic classification of human epithelial type 2 cell indirect immunofluorescence images using cell pyramid matching

This paper describes a novel system for automatic classification of images obtained from Anti-Nuclear Antibody (ANA) pathology tests on Human Epithelial type 2 (HEp-2) cells using the Indirect Immunofluorescence (IIF) protocol. The IIF protocol on HEp-2 cells has been the hallmark method to identify the presence of ANAs, due to its high sensitivity and the large range of antigens that can be detected. However, it suffers from numerous shortcomings, such as being subjective as well as time and labour intensive. Computer Aided Diagnostic (CAD) systems have been developed to address these problems, which automatically classify a HEp-2 cell image into one of its known patterns (eg. speckled, homogeneous). Most of the existing CAD systems use handpicked features to represent a HEp-2 cell image, which may only work in limited scenarios. We propose a novel automatic cell image classification method termed Cell Pyramid Matching (CPM), which is comprised of regional histograms of visual words coupled with the Multiple Kernel Learning framework. We present a study of several variations of generating histograms and show the efficacy of the system on two publicly available datasets: the ICPR HEp-2 cell classification contest dataset and the SNPHEp-2 dataset.

Dictionary learning and sparse coding on Grassmann manifolds : an extrinsic solution

Recent advances in computer vision and machine learning suggest that a wide range of problems can be addressed more appropriately by considering non-Euclidean geometry. In this paper we explore sparse dictionary learning over the space of linear subspaces, which form Riemannian structures known as Grassmann manifolds. To this end, we propose to embed Grassmann manifolds into the space of symmetric matrices by an isometric mapping, which enables us to devise a closed-form solution for updating a Grassmann dictionary, atom by atom. Furthermore, to handle non-linearity in data, we propose a kernelised version of the dictionary learning algorithm. Experiments on several classification tasks (face recognition, action recognition, dynamic texture classification) show that the proposed approach achieves considerable improvements in discrimination accuracy, in comparison to state-of-the-art methods such as kernelised Affine Hull Method and graph-embedding Grassmann discriminant analysis.

Multi-shot person re-identification via relational stein divergence

Person re-identification is particularly challenging due to significant appearance changes across separate camera views. In order to re-identify people, a representative human signature should effectively handle differences in illumination, pose and camera parameters. While general appearance-based methods are modelled in Euclidean spaces, it has been argued that some applications in image and video analysis are better modelled via non-Euclidean manifold geometry. To this end, recent approaches represent images as covariance matrices, and interpret such matrices as points on Riemannian manifolds. As direct classification on such manifolds can be difficult, in this paper we propose to represent each manifold point as a vector of similarities to class representers, via a recently introduced form of Bregman matrix divergence known as the Stein divergence. This is followed by using a discriminative mapping of similarity vectors for final classification. The use of similarity vectors is in contrast to the traditional approach of embedding manifolds into tangent spaces, which can suffer from representing the manifold structure inaccurately. Comparative evaluations on benchmark ETHZ and iLIDS datasets for the person re-identification task show that the proposed approach obtains better performance than recent techniques such as Histogram Plus Epitome, Partial Least Squares, and Symmetry-Driven Accumulation of Local Features.

Improved image set classification via joint sparse approximated nearest subspaces

Existing multi-model approaches for image set classification extract local models by clustering each image set individually only once, with fixed clusters used for matching with other image sets. However, this may result in the two closest clusters to represent different characteristics of an object, due to different undesirable environmental conditions (such as variations in illumination and pose). To address this problem, we propose to constrain the clustering of each query image set by forcing the clusters to have resemblance to the clusters in the gallery image sets. We first define a Frobenius norm distance between subspaces over Grassmann manifolds based on reconstruction error. We then extract local linear subspaces from a gallery image set via sparse representation. For each local linear subspace, we adaptively construct the corresponding closest subspace from the samples of a probe image set by joint sparse representation. We show that by minimising the sparse representation reconstruction error, we approach the nearest point on a Grassmann manifold. Experiments on Honda, ETH-80 and Cambridge-Gesture datasets show that the proposed method consistently outperforms several other recent techniques, such as Affine Hull based Image Set Distance (AHISD), Sparse Approximated Nearest Points (SANP) and Manifold Discriminant Analysis (MDA).

Joint 2D and 3D cues for image segmentation towards robotic applications

This thesis investigates the fusion of 3D visual information with 2D image cues to provide 3D semantic maps of large-scale environments in which a robot traverses for robotic applications. A major theme of this thesis was to exploit the availability of 3D information acquired from robot sensors to improve upon 2D object classification alone. The proposed methods have been evaluated on several indoor and outdoor datasets collected from mobile robotic platforms including a quadcopter and ground vehicle covering several kilometres of urban roads.

Towards the development of 1-to-n human machine interfaces for unmanned aerial vehicles

Multi-touch interfaces across a wide range of hardware platforms are becoming pervasive. This is due to the adoption of smart phones and tablets in both the consumer and corporate market place. This paper proposes a human-machine interface to interact with unmanned aerial systems based on the philosophy of multi-touch hardware-independent high-level interaction with multiple systems simultaneously. Our approach incorporates emerging development methods for multi-touch interfaces on mobile platforms. A framework is defined for supporting multiple protocols. An open source solution is presented that demonstrates: architecture supporting different communications hardware; an extensible approach for supporting multiple protocols; and the ability to monitor and interact with multiple UAVs from multiple clients simultaneously. Validation tests were conducted to assess the performance, scalability and impact on packet latency under different client configurations.

Classifying natural aerial scenery for autonomous aircraft emergency landing

In this paper, we present an approach for image-based surface classification using multi-class Support Vector Machine (SVM). Classifying surfaces in aerial images is an important step towards an increased aircraft autonomy in emergency landing situations. We design a one-vs-all SVM classifier and conduct experiments on five data sets. Results demonstrate consistent overall performance figures over 88% and approximately 8% more accurate to those published on multi-class SVM on the KTH TIPS data set. We also show per-class performance values by using normalised confusion matrices. Our approach is designed to be executed online using a minimum set of feature attributes representing a feasible and ready-to-deploy system for onboard execution.

Advanced in-vehicle applications to mitigate driver aggression

Aggressive behavior at the steering wheel has been indicated as a contributing factor in a majority of crashes and anger has been compared to alcohol impairment in terms of probability to cause a crash. It has been shown that being in a state of anger or excitement while driving can decrease the drivers’ performances. . This paper reports the evaluation of 6 novel design alternatives of In-Vehicle Information Systems (IVIS) aimed at mitigating driver aggression. Each application presented was designed to tackle the following contributing factors to driver aggression: competitiveness, anonymity, territoriality, stress as well as social and emotional isolation. The 6 applications were simulated using computer vision algorithm to automatically overlay the real traffic conditions with ‘Head-Up Display’ visualizations. Two applications emerged over the others from participant’s evaluation: shared music combined the known calming effect of music with the sense of sympathy and intimacy caused by hearing other drivers’ music. The Shared Snapshot application provided an immediate gratification and was evaluated as a potential prevention of roadside quarrels. The paper presents Theoretical foundation, participant’s evaluations, implications and limitations of the study.