A brain inspired approach for multi-view patterns identification

Biologically human brain processes information in both uniimodal and multimodal approaches. In fact, information is progressively abstracted and seamlessly fused. Subsequently, the fusion of multimodal inputs allows a holistic understanding of a problem. The proliferation of technology has exponentially produced various sources of data, which could be likened to being the state of multimodality in human brain. Therefore, this is an inspiration to develop a methodology for exploring multimodal data and further identifying multi-view patterns. Specifically, we propose a brain inspired conceptual model that allows exploration and identification of patterns at different levels of granularity, different types of hierarchies and different types of modalities. A structurally adaptive neural network is deployed to implement the proposed model. Furthermore, the acquisition of multi-view patterns with the proposed model is
demonstrated and discussed with some experimental results.

Clusters driven implementation of a brain inspired model for multi-view pattern identifications

The human brain processes information in both unimodal and multimodal fashion where information is progressively captured, accumulated, abstracted and seamlessly fused. Subsequently, the fusion of multimodal inputs allows a holistic understanding of a problem. The proliferation of technology has produced various sources of electronic data and continues to do so exponentially. Finding patterns from such multi-source and multimodal data could be compared to the multimodal and multidimensional information processing in the human brain. Therefore, such brain functionality could be taken as an inspiration to develop a methodology for exploring multimodal and multi-source electronic data and further identifying multi-view patterns. In this paper, we first propose a brain inspired conceptual model that allows exploration and identification of patterns at different levels of granularity, different types of hierarchies and different types of modalities. Secondly, we present a cluster driven approach for the implementation of the proposed brain inspired model. Particularly, the Growing Self Organising Maps (GSOM) based cross-clustering approach is discussed. Furthermore, the acquisition of multi-view patterns with clusters driven implementation is demonstrated with experimental results.

Identifying multi-view patterns with hierarchy and granularity based multimodal (HGM) cognitive model

Humans perceive entities such as objects, patterns, events, etc. as concepts, which are the basic units in human intelligence and communications. In addition, perceptions of these entities could be abstracted and generalised at multiple levels of granularity. In particular, such granulation allows the formation and usage of concepts in human intelligence. Such natural granularity in human intelligence could inspire and motivate the design and development of pattern identification approach in Data Mining. In our opinion, a pattern could be perceived at multiple levels of granularity and thus we advocate for the co-existence of hierarchy and granularity. In addition, granular patterns exist across different sources of data (multimodality). In this paper, we present a cognitive model that incorporates the characteristics of Hierarchy, Granularity and Multimodality for multi-view patterns identification in crime domain. Such framework is implemented with Growing Self Organising Maps (GSOM) and some experimental results are presented and discussed.

Application of a brain inspired model for profiling multi-view crime patterns

With the massive amount of crime data generated daily, this has put law enforcement under intensive stress. This means that law enforcement has to compete against the time to solve crime. In addition, the focus of crime investigation has been expanded from the ability to catch the criminals towards the ability to act before a crime happens (i.e pre-crime). Given such situation, creation of crime profiles is very important to law enforcement, especially in understanding the behaviours of criminals and identifying the characteristics of similar crimes. In fact, crime profiles could be used to solve similar crimes and thus pre-crime action could be conducted. In this paper, a brain inspired conceptual model is proposed and a structurally adaptive neural network is deployed for its implementation. Subsequently, the proposed model is applied for the identification and presentation of multi-view crime patterns. Such multi-view crime patterns could be useful for the construction of crime profiles. Moreover, the suitability of the proposed model in crime profiling is discussed and demonstrated through some experimental results.

Towards designing an email classification system using multi-view based semi-supervised learning

The goal of email classification is to classify user emails into spam and legitimate ones. Many supervised learning algorithms have been invented in this domain to accomplish the task, and these algorithms require a large number of labeled training data. However, data labeling is a labor intensive task and requires in-depth domain knowledge. Thus, only a very small proportion of the data can be labeled in practice. This bottleneck greatly degrades the effectiveness of supervised email classification systems. In order to address this problem, in this work, we first identify some critical issues regarding supervised machine learning-based email classification. Then we propose an effective classification model based on multi-view disagreement-based semi-supervised learning. The motivation behind the attempt of using multi-view and semi-supervised learning is that multi-view can provide richer information for classification, which is often ignored by literature, and semi-supervised learning supplies with the capability of coping with labeled and unlabeled data. In the evaluation, we demonstrate that the multi-view data can improve the email classification than using a single view data, and that the proposed model working with our algorithm can achieve better performance as compared to the existing similar algorithms.

Mixed-norm sparse representation for multi view face recognition

Face recognition with multiple views is a challenging research problem. Most of the existing works have focused on extracting shared information among multiple views to improve recognition. However, when the pose variation is too large or missing, 'shared information' may not be properly extracted, leading to poor recognition results. In this paper, we propose a novel method for face recognition with multiple view images to overcome the large pose variation and missing pose issue. By introducing a novel mixed norm, the proposed method automatically selects candidates from the gallery to best represent a group of highly correlated face images in a query set to improve classification accuracy. This mixed norm combines the advantages of both sparse representation based classification (SRC) and joint sparse representation based classification (JSRC). A trade off between the ℓ1-norm from SRC and ℓ2,1-norm from JSRC is introduced to achieve this goal. Due to this property, the proposed method decreases the influence when a face image is unseen and has large pose variation in the recognition process. And when some face images with a certain degree of unseen pose variation appear, this mixed norm will find an optimal representation for these query images based on the shared information induced from multiple views. Moreover, we also address an open problem in robust sparse representation and classification which is using ℓ1-norm on the loss function to achieve a robust solution. To solve this formulation, we derive a simple, yet provably convergent algorithm based on the powerful alternative directions method of multipliers (ADMM) framework. We provide extensive comparisons which demonstrate that our method outperforms other state-of-the-arts algorithms on CMU-PIE, Yale B and Multi-PIE databases for multi-view face recognition.

Multi-view subspace clustering for face images

In many real-world computer vision applications, such as multi-camera surveillance, the objects of interest are captured by visual sensors concurrently, resulting in multi-view data. These views usually provide complementary information to each other. One recent and powerful computer vision method for clustering is sparse subspace clustering (SSC); however, it was not designed for multi-view data, which break down its linear separability assumption. To integrate complementary information between views, multi-view clustering algorithms are required to improve the clustering performance. In this paper, we propose a novel multi-view subspace clustering by searching for an unified latent structure as a global affinity matrix in subspace clustering. Due to the integration of affinity matrices for each view, this global affinity matrix can best represent the relationship between clusters. This could help us achieve better performance on face clustering. We derive a provably convergent algorithm based on the alternating direction method of multipliers (ADMM) framework, which is computationally efficient, to solve the formulation. We demonstrate that this formulation outperforms other alternatives based on state-of-The-Arts on challenging multi-view face datasets.

A multi-view framework for generating mobile apps

This paper demonstrates a multi-view framework for Rapid APPlication Tool (RAPPT). RAPPT enables rapid development of mobile applications. It employs a multilevel approach to mobile application development: a Domain Specific Visual Language to define the high level structure of mobile apps, a Domain Specific Textual Language to define behavioural concepts, and concrete source code for fine grained improvements.

Super-resolution of a 3-dimensional scene from novel viewpoints

Super-resolution is a method of post-processing image enhancement that increases the spatial resolution of video or images. Existing super-resolution techniques apply only to images captured of a planar scene. This paper aims to extend super-resolution concepts from the 2D domain to the 3D domain, drawing on ideas from both superresolution and multi-view geometry, two fields of research that until now have predominantly been studied in isolation. 2D super-resolution methods are not without their complexities and limitations. However, once multiple views of a scene are considered within a super-resolution framework, a new range of issues arise that must also be resolved. For example, when input images of a scene with variation in depth are considered, it is no longer clear how and where the images should be registered. This paper describes the use of sparse 3D reconstruction in order to ‘register’ the input images, which are then transferred to a novel image plane and combined to increase the perceived detail in the scene. Experimental results using real images captured from generally positioned input cameras are presented.

3D sparse feature model using short baseline stereo and multiple view registration

This paper outlines a methodology to generate a distinctive object representation offline, using short-baseline stereo fundamentals to triangulate highly descriptive object features in multiple pairs of stereo images. A group of sparse 2.5D perspective views are built and the multiple views are then fused into a single sparse 3D model using a common 3D shape registration technique. Having prior knowledge, such as the proposed sparse feature model, is useful when detecting an object and estimating its pose for real-time systems like augmented reality.

Sparse representation for face images.

This thesis address issues for face recognition with multi-view face images. Several effective methods are proposed and compared with current state of the art. A novel framework that generalises existing sparse representation-based methods in order to exploit the sharing information to against pose variations of face images is proposed.

A field model for repairing 3D shapes

This paper proposes a field model for repairing 3D shapes constructed from multi-view RGB data. Specifically, we represent a 3D shape in a Markov random field (MRF) in which the geometric information is encoded by random binary variables and the appearance information is retrieved from a set of RGB images captured at multiple viewpoints. The local priors in the MRF model capture the local structures of object shapes and are learnt from 3D shape templates using a convolutional deep belief network. Repairing a 3D shape is formulated as the maximum a posteriori (MAP) estimation in the corresponding MRF. Variational mean field approximation technique is adopted for the MAP estimation. The proposed method was evaluated on both artificial data and real data obtained from reconstruction of practical scenes. Experimental results have shown the robustness and efficiency of the proposed method in repairing noisy and incomplete 3D shapes.

Teaching 3-D geometry - the multi representational way

Sonja Kalbitzer and Esther Loong provide an excellent range of activities that promote geometric thinking through the exploration of three-dimensional objects. They also provide some discussion on assessing the tasks and providing student feedback.

A computational approach to the reconstruction of surface geometry from early temple superstructures

Recovering the control or implicit geometry underlying temple architecture requires bringing together fragments of evidence from field measurements, relating these to mathematical and geometric descriptions in canonical texts and proposing "best-fit" constructive models. While scholars in the field have traditionally used manual methods, the innovative application of niche computational techniques can help extend the study of artefact geometry. This paper demonstrates the application of a hybrid computational approach to the problem of recovering the surface geometry of early temple superstructures. The approach combines field measurements of temples, close-range architectural photogrammetry, rule-based generation and parametric modelling. The computing of surface geometry comprises a rule-based global model governing the overall form of the superstructure, several local models for individual motifs using photogrammetry and an intermediate geometry model that combines the two. To explain the technique and the different models, the paper examines an illustrative example of surface geometry reconstruction based on studies undertaken on a tenth century stone superstructure from western India. The example demonstrates that a combination of computational methods yields sophisticated models of the constructive geometry underlying temple form and that these digital artefacts can form the basis for in depth comparative analysis of temples, arising out of similar techniques, spread over geography, culture and time.

Content-based video indexing for sports applications using integrated multi-modal approach

To sustain an ongoing rapid growth of video information, there is an emerging demand for a sophisticated content-based video indexing system. However, current video indexing solutions are still immature and lack of any standard. This doctoral consists of a research work based on an integrated multi-modal approach for sports video indexing and retrieval. By combining specific features extractable from multiple audio-visual modalities, generic structure and specific events can be detected and classified. During browsing and retrieval, users will benefit from the integration of high-level semantic and some descriptive mid-level features such as whistle and close-up view of player(s).