Subject independent human action recognition using spatio-depth information and meta-cognitive RBF network

In this paper, we present a machine learning approach for subject independent human action recognition using depth camera, emphasizing the importance of depth in recognition of actions. The proposed approach uses the flow information of all 3 dimensions to classify an action. In our approach, we have obtained the 2-D optical flow and used it along with the depth image to obtain the depth flow (Z motion vectors). The obtained flow captures the dynamics of the actions in space time. Feature vectors are obtained by averaging the 3-D motion over a grid laid over the silhouette in a hierarchical fashion. These hierarchical fine to coarse windows capture the motion dynamics of the object at various scales. The extracted features are used to train a Meta-cognitive Radial Basis Function Network (McRBFN) that uses a Projection Based Learning (PBL) algorithm, referred to as PBL-McRBFN, henceforth. PBL-McRBFN begins with zero hidden neurons and builds the network based on the best human learning strategy, namely, self-regulated learning in a meta-cognitive environment. When a sample is used for learning, PBLMcRBFN uses the sample overlapping conditions, and a projection based learning algorithm to estimate the parameters of the network. The performance of PBL-McRBFN is compared to that of a Support Vector Machine (SVM) and Extreme Learning Machine (ELM) classifiers with representation of every person and action in the training and testing datasets. Performance study shows that PBL-McRBFN outperforms these classifiers in recognizing actions in 3-D. Further, a subject-independent study is conducted by leave-one-subject-out strategy and its generalization performance is tested. It is observed from the subject-independent study that McRBFN is capable of generalizing actions accurately. The performance of the proposed approach is benchmarked with Video Analytics Lab (VAL) dataset and Berkeley Multimodal Human Action Database (MHAD). (C) 2013 Elsevier Ltd. All rights reserved.

Optical microscopy methods for understanding learning and memory

Structural dynamics of dendritic spines is one of the key correlative measures of synaptic plasticity for encoding short-term and long-term memory. Optical studies of structural changes in brain tissue using confocal microscopy face difficulties of scattering. This results in low signal-to-noise ratio and thus limiting the imaging depth to few tens of microns. Multiphoton microscopy (MpM) overcomes this limitation by using low-energy photons to cause localized excitation and achieve high resolution in all three dimensions. Multiple low-energy photons with longer wavelengths minimize scattering and allow access to deeper brain regions at several hundred microns. In this article, we provide a basic understanding of the physical phenomena that give MpM an edge over conventional microscopy. Further, we highlight a few of the key studies in the field of learning and memory which would not have been possible without the advent of MpM.

Raman spectroscopy explores molecular structural signatures of hidden materials in depth: Universal Multiple Angle Raman Spectroscopy

Non-invasive 3D imaging in materials and medical research involves methodologies such as X-ray imaging, MRI, fluorescence and optical coherence tomography, NIR absorption imaging, etc., providing global morphological/density/absorption changes of the hidden components. However, molecular information of such buried materials has been elusive. In this article we demonstrate observation of molecular structural information of materials hidden/buried in depth using Raman scattering. Typically, Raman spectroscopic observations are made at fixed collection angles, such as, 906, 1356, and 1806, except in spatially offset Raman scattering (SORS) (only back scattering based collection of photons) and transmission techniques. Such specific collection angles restrict the observations of Raman signals either from or near the surface of the materials. Universal Multiple Angle Raman Spectroscopy (UMARS) presented here employs the principle of (a) penetration depth of photons and then diffuse propagation through non-absorbing media by multiple scattering and (b) detection of signals from all the observable angles.

Ultrasound-modulated optical tomography: direct recovery of elasticity distribution from experimentally measured intensity autocorrelation

Based on an ultrasound-modulated optical tomography experiment, a direct, quantitative recovery of Young's modulus (E) is achieved from the modulation depth (M) in the intensity autocorrelation. The number of detector locations is limited to two in orthogonal directions, reducing the complexity of the data gathering step whilst ensuring against an impoverishment of the measurement, by employing ultrasound frequency as a parameter to vary during data collection. The M and E are related via two partial differential equations. The first one connects M to the amplitude of vibration of the scattering centers in the focal volume and the other, this amplitude to E. A (composite) sensitivity matrix is arrived at mapping the variation of M with that of E and used in a (barely regularized) Gauss-Newton algorithm to iteratively recover E. The reconstruction results showing the variation of E are presented. (C) 2015 Optical Society of America

An Optical How Feature and McFIS Based Approach for 3-Dimensional Human Action Recognition

We propose to develop a 3-D optical flow features based human action recognition system. Optical flow based features are employed here since they can capture the apparent movement in object, by design. Moreover, they can represent information hierarchically from local pixel level to global object level. In this work, 3-D optical flow based features a re extracted by combining the 2-1) optical flow based features with the depth flow features obtained from depth camera. In order to develop an action recognition system, we employ a Meta-Cognitive Neuro-Fuzzy Inference System (McFIS). The m of McFIS is to find the decision boundary separating different classes based on their respective optical flow based features. McFIS consists of a neuro-fuzzy inference system (cognitive component) and a self-regulatory learning mechanism (meta-cognitive component). During the supervised learning, self-regulatory learning mechanism monitors the knowledge of the current sample with respect to the existing knowledge in the network and controls the learning by deciding on sample deletion, sample learning or sample reserve strategies. The performance of the proposed action recognition system was evaluated on a proprietary data set consisting of eight subjects. The performance evaluation with standard support vector machine classifier and extreme learning machine indicates improved performance of McFIS is recognizing actions based of 3-D optical flow based features.

Region-of-interest diffuse optical tomography system

Diffuse optical tomography (DOT) using near-infrared light is a promising tool for non-invasive imaging of deep tissue. This technique is capable of quantitative reconstruction of absorption (mu(a)) and scattering coefficient (mu(s)) inhomogeneities in the tissue. The rationale for reconstructing the optical property map is that the absorption coefficient variation provides diagnostic information about metabolic and disease states of the tissue. The aim of DOT is to reconstruct the internal tissue cross section with good spatial resolution and contrast from noisy measurements non-invasively. We develop a region-of-interest scanning system based on DOT principles. Modulated light is injected into the phantom/tissue through one of the four light emitting diode sources. The light traversing through the tissue gets partially absorbed and scattered multiple times. The intensity and phase of the exiting light are measured using a set of photodetectors. The light transport through a tissue is diffusive in nature and is modeled using radiative transfer equation. However, a simplified model based on diffusion equation (DE) can be used if the system satisfies following conditions: (a) the optical parameter of the inhomogeneity is close to the optical property of the background, and (b) mu(s) of the medium is much greater than mu(a) (mu(s) >> mu(a)). The light transport through a highly scattering tissue satisfies both of these conditions. A discrete version of DE based on finite element method is used for solving the inverse problem. The depth of probing light inside the tissue depends on the wavelength of light, absorption, and scattering coefficients of the medium and the separation between the source and detector locations. Extensive simulation studies have been carried out and the results are validated using two sets of experimental measurements. The utility of the system can be further improved by using multiple wavelength light sources. In such a scheme, the spectroscopic variation of absorption coefficient in the tissue can be used to arrive at the oxygenation changes in the tissue. (C) 2016 AIP Publishing LLC.