USHA: unified software and hardware architecture for video decoding

Video decoders used in emerging applications need to be flexible to handle a large variety of video formats and deliver scalable performance to handle wide variations in workloads. In this paper we propose a unified software and hardware architecture for video decoding to achieve scalable performance with flexibility. The light weight processor tiles and the reconfigurable hardware tiles in our architecture enable software and hardware implementations to co-exist, while a programmable interconnect enables dynamic interconnection of the tiles. Our process network oriented compilation flow achieves realization agnostic application partitioning and enables seamless migration across uniprocessor, multi-processor, semi hardware and full hardware implementations of a video decoder. An application quality of service aware scheduler monitors and controls the operation of the entire system. We prove the concept through a prototype of the architecture on an off-the-shelf FPGA. The FPGA prototype shows a scaling in performance from QCIF to 1080p resolutions in four discrete steps. We also demonstrate that the reconfiguration time is short enough to allow migration from one configuration to the other without any frame loss.

Total internal reflection (TIR) Raman tribometer: a new tool for in situ study of friction-induced material transfer

The sliding history in friction-induced material transfer of dry 2H-MoS2 particles in a sheared contact was studied. Video images in contact showed fragmentation of lubricant particles and build-up of a transfer film, and were used to measure the speed of fragmented particles in the contact region. Total internal reflection (TIR) Raman spectroscopy was used to follow the build-up of the MoS2 transfer film. A combination of in situ and ex situ analysis of the mating bodies revealed the thickness of the transfer film at steady state to be of the order of 35 nm on the ball surface and 15 nm on the flat substrate. Insights into the mechanism of formation of the transfer film in the early stages of sliding contact are deduced.

Image deconvolution research: its scope and importance in live cell microscopy

Fluorescence microscopy has become an indispensable tool in cell biology research due its exceptional specificity and ability to visualize subcellular structures with high contrast. It has highest impact when applied in 4D mode, i.e. when applied to record 3D image information as a function of time, since it allows the study of dynamic cellular processes in their native environment. The main issue in 4D fluorescence microscopy is that the phototoxic effect of fluorescence excitation gets accumulated during 4D image acquisition to the extent that normal cell functions are altered. Hence to avoid the alteration of normal cell functioning, it is required to minimize the excitation dose used for individual 2D images constituting a 4D image. Consequently, the noise level becomes very high degrading the resolution. According to the current status of technology, there is a minimum required excitation dose to ensure a resolution that is adequate for biological investigations. This minimum is sufficient to damage light-sensitive cells such as yeast if 4D imaging is performed for an extended period of time, for example, imaging for a complete cell cycle. Nevertheless, our recently developed deconvolution method resolves this conflict forming an enabling technology for visualization of dynamical processes of light-sensitive cells for durations longer than ever without perturbing normal cell functioning. The main goal of this article is to emphasize that there are still possibilities for enabling newer kinds of experiment in cell biology research involving even longer 4D imaging, by only improving deconvolution methods without any new optical technologies.

H.264 COMPRESSED VIDEO CLASSIFICATION USING HISTOGRAM OF ORIENTED MOTION VECTORS (HOMV)

In this paper, we have proposed a simple and effective approach to classify H.264 compressed videos, by capturing orientation information from the motion vectors. Our major contribution involves computing Histogram of Oriented Motion Vectors (HOMV) for overlapping hierarchical Space-Time cubes. The Space-Time cubes selected are partially overlapped. HOMV is found to be very effective to define the motion characteristics of these cubes. We then use Bag of Features (B OF) approach to define the video as histogram of HOMV keywords, obtained using k-means clustering. The video feature, thus computed, is found to be very effective in classifying videos. We demonstrate our results with experiments on two large publicly available video database.

RAPID HUMAN ACTION RECOGNITION IN H.264/AVC COMPRESSED DOMAIN FOR VIDEO SURVEILLANCE

This paper discusses a novel high-speed approach for human action recognition in H. 264/AVC compressed domain. The proposed algorithm utilizes cues from quantization parameters and motion vectors extracted from the compressed video sequence for feature extraction and further classification using Support Vector Machines (SVM). The ultimate goal of our work is to portray a much faster algorithm than pixel domain counterparts, with comparable accuracy, utilizing only the sparse information from compressed video. Partial decoding rules out the complexity of full decoding, and minimizes computational load and memory usage, which can effect in reduced hardware utilization and fast recognition results. The proposed approach can handle illumination changes, scale, and appearance variations, and is robust in outdoor as well as indoor testing scenarios. We have tested our method on two benchmark action datasets and achieved more than 85% accuracy. The proposed algorithm classifies actions with speed (>2000 fps) approximately 100 times more than existing state-of-the-art pixel-domain algorithms.

CHEXVIS: a tool for molecular channel extraction and visualization

Background: Understanding channel structures that lead to active sites or traverse the molecule is important in the study of molecular functions such as ion, ligand, and small molecule transport. Efficient methods for extracting, storing, and analyzing protein channels are required to support such studies. Further, there is a need for an integrated framework that supports computation of the channels, interactive exploration of their structure, and detailed visual analysis of their properties. Results: We describe a method for molecular channel extraction based on the alpha complex representation. The method computes geometrically feasible channels, stores both the volume occupied by the channel and its centerline in a unified representation, and reports significant channels. The representation also supports efficient computation of channel profiles that help understand channel properties. We describe methods for effective visualization of the channels and their profiles. These methods and the visual analysis framework are implemented in a software tool, CHEXVIS. We apply the method on a number of known channel containing proteins to extract pore features. Results from these experiments on several proteins show that CHEXVIS performance is comparable to, and in some cases, better than existing channel extraction techniques. Using several case studies, we demonstrate how CHEXVIS can be used to study channels, extract their properties and gain insights into molecular function. Conclusion: CHEXVIS supports the visual exploration of multiple channels together with their geometric and physico-chemical properties thereby enabling the understanding of the basic biology of transport through protein channels. The CHEXVIS web-server is freely available at http://vgl.serc.iisc.ernet.in/chexvis/. The web-server is supported on all modern browsers with latest Java plug-in.

Conceptualization of a Behavior Modification Tool for University Systems and Doctoral Students

This paper describes a university based system relevant to doctoral students who have problems with themselves, their peers and research supervisors. Doctoral students have various challenges to solve and these challenges contribute to delays in their thesis submission. This tool aims at helping them think through their problem in a pre-counseling stage. The tool uses narratives and hypothetical stories to walk a doctoral student through options of responses he or she can make given the situation in the narrative. Narratives were developed after a preliminary survey (n=57) of doctoral students. The survey indicated that problems they experienced were: busy supervisors, negative competition from peers and laziness with self. The narrative scenarios in the tool prompt self-reflection and provide for options to chose from leading to the next scenario that will ensue. The different stages of the stimulus-response cycles are designed based on Thomas-Kilmann conflict resolution techniques (collaboration and avoidance). Each stimulus-response cycle has a score attached that reflects the student's ability to judge a collaborative approach. At the end of all the stages a scorecard is generated indicating either a progressive or regressive outcome of thesis submission.

Compressed domain human action recognition in H.264/AVC video streams

This paper discusses a novel high-speed approach for human action recognition in H.264/AVC compressed domain. The proposed algorithm utilizes cues from quantization parameters and motion vectors extracted from the compressed video sequence for feature extraction and further classification using Support Vector Machines (SVM). The ultimate goal of the proposed work is to portray a much faster algorithm than pixel domain counterparts, with comparable accuracy, utilizing only the sparse information from compressed video. Partial decoding rules out the complexity of full decoding, and minimizes computational load and memory usage, which can result in reduced hardware utilization and faster recognition results. The proposed approach can handle illumination changes, scale, and appearance variations, and is robust to outdoor as well as indoor testing scenarios. We have evaluated the performance of the proposed method on two benchmark action datasets and achieved more than 85 % accuracy. The proposed algorithm classifies actions with speed (> 2,000 fps) approximately 100 times faster than existing state-of-the-art pixel-domain algorithms.

SeamSeg: Video Object Segmentation using Patch Seams

In this paper, we propose a technique for video object segmentation using patch seams across frames. Typically, seams, which are connected paths of low energy, are utilised for retargeting, where the primary aim is to reduce the image size while preserving the salient image contents. Here, we adapt the formulation of seams for temporal label propagation. The energy function associated with the proposed video seams provides temporal linking of patches across frames, to accurately segment the object. The proposed energy function takes into account the similarity of patches along the seam, temporal consistency of motion and spatial coherency of seams. Label propagation is achieved with high fidelity in the critical boundary regions, utilising the proposed patch seams. To achieve this without additional overheads, we curtail the error propagation by formulating boundary regions as rough-sets. The proposed approach out-perform state-of-the-art supervised and unsupervised algorithms, on benchmark datasets.

Anomaly detection in compressed H.264/AVC video

Real time anomaly detection is the need of the hour for any security applications. In this article, we have proposed a real time anomaly detection for H.264 compressed video streams utilizing pre-encoded motion vectors (MVs). The proposed work is principally motivated by the observation that MVs have distinct characteristics during anomaly than usual. Our observation shows that H.264 MV magnitude and orientation contain relevant information which can be used to model the usual behavior (UB) effectively. This is subsequently extended to detect abnormality/anomaly based on the probability of occurrence of a behavior. The performance of the proposed algorithm was evaluated and bench-marked on UMN and Ped anomaly detection video datasets, with a detection rate of 70 frames per sec resulting in 90x and 250x speedup, along with on-par detection accuracy compared to the state-of-the-art algorithms.

A survey on compressed domain video analysis techniques

Image and video analysis requires rich features that can characterize various aspects of visual information. These rich features are typically extracted from the pixel values of the images and videos, which require huge amount of computation and seldom useful for real-time analysis. On the contrary, the compressed domain analysis offers relevant information pertaining to the visual content in the form of transform coefficients, motion vectors, quantization steps, coded block patterns with minimal computational burden. The quantum of work done in compressed domain is relatively much less compared to pixel domain. This paper aims to survey various video analysis efforts published during the last decade across the spectrum of video compression standards. In this survey, we have included only the analysis part, excluding the processing aspect of compressed domain. This analysis spans through various computer vision applications such as moving object segmentation, human action recognition, indexing, retrieval, face detection, video classification and object tracking in compressed videos.

Raman and infra-red microspectroscopy: towards quantitative evaluation for clinical research by ratiometric analysis

Biomolecular structure elucidation is one of the major techniques for studying the basic processes of life. These processes get modulated, hindered or altered due to various causes like diseases, which is why biomolecular analysis and imaging play an important role in diagnosis, treatment prognosis and monitoring. Vibrational spectroscopy (IR and Raman), which is a molecular bond specific technique, can assist the researcher in chemical structure interpretation. Based on the combination with microscopy, vibrational microspectroscopy is currently emerging as an important tool for biomedical research, with a spatial resolution at the cellular and sub-cellular level. These techniques offer various advantages, enabling label-free, biomolecular fingerprinting in the native state. However, the complexity involved in deciphering the required information from a spectrum hampered their entry into the clinic. Today with the advent of automated algorithms, vibrational microspectroscopy excels in the field of spectropathology. However, researchers should be aware of how quantification based on absolute band intensities may be affected by instrumental parameters, sample thickness, water content, substrate backgrounds and other possible artefacts. In this review these practical issues and their effects on the quantification of biomolecules will be discussed in detail. In many cases ratiometric analysis can help to circumvent these problems and enable the quantitative study of biological samples, including ratiometric imaging in 1D, 2D and 3D. We provide an extensive overview from the recent scientific literature on IR and Raman band ratios used for studying biological systems and for disease diagnosis and treatment prognosis.

Large Eddy Simulation of Complex Turbulent Flows: Physical Aspects and Research Trends

In the current paper, we have primarily addressed one powerful simulation tool developed during the last decades-Large Eddy Simulation (LES), which is most suitable for unsteady three-dimensional complex turbulent flows in industry and natural environment. The main point in LES is that the large-scale motion is resolved while the small-scale motion is modeled or, in geophysical terminology, parameterized. With a view to devising a subgrid-scale(SGS) model of high quality, we have highlighted analyzing physical aspects in scale interaction and-energy transfer such as dissipation, backscatter, local and non-local interaction, anisotropy and resolution requirement. They are the factors responsible for where the advantages and disadvantages in existing SGS models come from. A case study on LES of turbulence in vegetative canopy is presented to illustrate that LES model is more based on physical arguments. Then, varieties of challenging complex turbulent flows in both industry and geophysical fields in the near future-are presented. In conclusion; we may say with confidence that new century shall see the flourish in the research of turbulence with the aid of LES combined with other approaches.

Hydroacoustic surveys: a non-destructive approach to monitoring fish distributions at National Marine Sanctuaries

The science of fisheries acoustics and its applicability to resource management have evolved over the past several decades. This document provides a basic description of fisheries acoustics and recommendations on using this technology for research and monitoring of fish distributions and habitats within sanctuaries. It also describes recent efforts aimed at applying fisheries acoustics to Gray’s Reef National Marine Sanctuary (GRNMS) (Figure 1). Historically, methods to assess the underwater environment have included net trawls, diver censuses, hook and line, video, sonar and other techniques deployed in a variety of ways. Fisheries acoustics, using active sonar, relies on the physics of sound traveling through water to quantify the distribution of biota in the water column. By sending a signal of a given frequency through the water column and recording the time of travel and the strength of the reflected signal, it is possible to determine the size and location of fish and estimate biomass from the acoustic backscatter. As a fisheries assessment tool, active hydroacoustics technology is an efficient, non-intrusive method of mapping the water column at a very fine spatial and temporal resolution. It provides a practical alternative to bottom and mid-water trawls, which are not allowed at GRNMS. Passive acoustics, which uses underwater hydrophones to record man-made and natural sounds such as fish spawning calls and sounds produced by marine mammals for communication and echolocation, can provide a useful, complementary survey tool. This report primarily deals with active acoustics, although the integration of active and passive acoustics is addressed as well. (PDF contains 32 pages)