Image vusion : image and video fusion

It is not uncommon in many image acquisition solutions to balance a trade off between obtaining high resolution images at very low frame rates or acquiring a burst of low resolution images at higher frame rates. This paper introduces a novel image fusion framework for producing a high resolution video by augmenting analysed motion in a low resolution video to a single high resolution image. Many application domains such as remote sensing, low radiation medical imaging and battlefield automation will benefit from this novel fusion framework. The results show that a captured high resolution 30 frames per second video can be produced with 95% cost reduction while maintaining 94% structural similarity.

Robustness comparison of clustering - based vs. non-clustering multi-label classifications for image and video annotations

This paper reports robustness comparison of clustering-based multi-label classification methods versus nonclustering counterparts for multi-concept associated image and video annotations. In the experimental setting of this paper, we adopted six popular multi-label classification Algorithms, two different base classifiers for problem transformation based multilabel classifications, and three different clustering algorithms for pre-clustering of the training data. We conducted experimental evaluation on two multi-label benchmark datasets: scene image data and mediamill video data. We also employed two multi-label classification evaluation metrics, namely, micro F1-measure and Hamming-loss to present the predictive performance of the classifications. The results reveal that different base classifiers and clustering methods contribute differently to the performance of the multi-label classifications. Overall, the pre-clustering methods improve the effectiveness of multi-label classifications in certain experimental settings. This provides vital information to users when deciding which multi-label classification method to choose for multiple-concept associated image and video annotations.

Multimodal iron oxide (Fe₃O₄)-saturated lactoferrin nanocapsules as nanotheranostics for real-time imaging and breast cancer therapy of claudin-low, triple-negative (ER-/PR-/HER2-)

AIM: To unravel the multimodal nanotheranostic ability of Fe3O4-saturated bovine lactoferrin nanocapsules (FebLf NCs) in claudin-low, triple-negative breast cancer model. MATERIALS & METHODS: Xenograft study was performed to examine biocompatibility, antitumor efficacy and multimodal nanotheranostic action in combination with near-infrared live mice imaging. RESULTS: FebLf NCs exhibited a size range of 80 nm ± 5 nm with observed superparamagnetism. FebLf NCs successfully internalized into breast cancer cells through receptor-mediated endocytosis and induced apoptosis through the downregulation of inhibitor of apoptosis survivin and livin proteins. Investigations revealed a remarkable biocompatibility, anticancer efficacy of the FebLf NCs. Near-infrared imaging observations confirmed selective localization of multimodal FebLf NCs at the tumor site and lead to time-dependent reduction of tumor growth. CONCLUSION: FebLf NCs can be safe, biocompatible nanotheranostic approach for real-time imaging and monitoring the effect of drugs in real time and have potentials in future clinical trials.

Rapid identification and differentiation of Mycobacterium avium subspecies paratuberculosis types by use of real-time PCR and high-resolution melt analysis of the MAP1506 locus

High-resolution melt (HRM) analysis can identify sequence polymorphisms by comparing the melting curves of amplicons generated by real-time PCR amplification. We describe the application of this technique to identify Mycobacterium avium subspecies paratuberculosis types I, II, and III. The HRM approach was based on type-specific nucleotide sequences in MAP1506, a member of the PPE (proline-proline-glutamic acid) gene family.

Real time feature-based facial tracking using Lie algebras

We have developed a novel human facial tracking system that operates in real time at a video frame rate without needing any special hardware. The approach is based on the use of Lie algebra, and uses three-dimensional feature points on the targeted human face. It is assumed that the roughly estimated facial model (relative coordinates of the three-dimensional feature points) is known. First, the initial feature positions of the face are determined using a model fitting technique. Then, the tracking is operated by the following sequence: (1) capture the new video frame and render feature points to the image plane; (2) search for new positions of the feature points on the image plane; (3) get the Euclidean matrix from the moving vector and the three-dimensional information for the points; and (4) rotate and translate the feature points by using the Euclidean matrix, and render the new points on the image plane. The key algorithm of this tracker is to estimate the Euclidean matrix by using a least square technique based on Lie algebra. The resulting tracker performed very well on the task of tracking a human face.

Real-time GPU color-based segmentation of football players

In this paper, we propose a multi-camera application capable of processing high resolution images and extracting features based on colors patterns over graphic processing units (GPU). The goal is to work in real time under the uncontrolled environment of a sport event like a football match. Since football players are composed for diverse and complex color patterns, a Gaussian Mixture Models (GMM) is applied as segmentation paradigm, in order to analyze sport live images and video. Optimization techniques have also been applied over the C++ implementation using profiling tools focused on high performance. Time consuming tasks were implemented over NVIDIA's CUDA platform, and later restructured and enhanced, speeding up the whole process significantly. Our resulting code is around 4-11 times faster on a low cost GPU than a highly optimized C++ version on a central processing unit (CPU) over the same data. Real time has been obtained processing until 64 frames per second. An important conclusion derived from our study is the scalability of the application to the number of cores on the GPU. © 2011 Springer-Verlag.

Real-Time Automatic Object Classification and Tracking using Genetic Programming and NVIDIA R CUDA TM

Genetic Programming (GP) is a widely used methodology for solving various computational problems. GP's problem solving ability is usually hindered by its long execution times. In this thesis, GP is applied toward real-time computer vision. In particular, object classification and tracking using a parallel GP system is discussed. First, a study of suitable GP languages for object classification is presented. Two main GP approaches for visual pattern classification, namely the block-classifiers and the pixel-classifiers, were studied. Results showed that the pixel-classifiers generally performed better. Using these results, a suitable language was selected for the real-time implementation. Synthetic video data was used in the experiments. The goal of the experiments was to evolve a unique classifier for each texture pattern that existed in the video. The experiments revealed that the system was capable of correctly tracking the textures in the video. The performance of the system was on-par with real-time requirements.

Video anomaly detection in real-time on a Power-Aware Heterogeneous Platform

FPGAs and GPUs are often used when real-time performance in video processing is required. An accelerated processor is chosen based on task-specific priorities (power consumption, processing time and detection accuracy), and this decision is normally made once at design time. All three characteristics are important, particularly in battery-powered systems. Here we propose a method for moving selection of processing platform from a single design-time choice to a continuous run time one.We implement Histogram of Oriented Gradients (HOG) detectors for cars and people and Mixture of Gaussians (MoG) motion detectors running across FPGA, GPU and CPU in a heterogeneous system. We use this to detect illegally parked vehicles in urban scenes. Power, time and accuracy information for each detector is characterised. An anomaly measure is assigned to each detected object based on its trajectory and location, when compared to learned contextual movement patterns. This drives processor and implementation selection, so that scenes with high behavioural anomalies are processed with faster but more power hungry implementations, but routine or static time periods are processed with power-optimised, less accurate, slower versions. Real-time performance is evaluated on video datasets including i-LIDS. Compared to power-optimised static selection, automatic dynamic implementation mapping is 10% more accurate but draws 12W extra power in our testbed desktop system.

Network protocols and predictive control strategies for distributed real-time control applications

A trend in design and implementation of modern industrial automation systems is to integrate computing, communication and control into a unified framework at different levels of machine/factory operations and information processing. These distributed control systems are referred to as networked control systems (NCSs). They are composed of sensors, actuators, and controllers interconnected over communication networks. As most of communication networks are not designed for NCS applications, the communication requirements of NCSs may be not satisfied. For example, traditional control systems require the data to be accurate, timely and lossless. However, because of random transmission delays and packet losses, the control performance of a control system may be badly deteriorated, and the control system rendered unstable. The main challenge of NCS design is to both maintain and improve stable control performance of an NCS. To achieve this, communication and control methodologies have to be designed. In recent decades, Ethernet and 802.11 networks have been introduced in control networks and have even replaced traditional fieldbus productions in some real-time control applications, because of their high bandwidth and good interoperability. As Ethernet and 802.11 networks are not designed for distributed control applications, two aspects of NCS research need to be addressed to make these communication networks suitable for control systems in industrial environments. From the perspective of networking, communication protocols need to be designed to satisfy communication requirements for NCSs such as real-time communication and high-precision clock consistency requirements. From the perspective of control, methods to compensate for network-induced delays and packet losses are important for NCS design. To make Ethernet-based and 802.11 networks suitable for distributed control applications, this thesis develops a high-precision relative clock synchronisation protocol and an analytical model for analysing the real-time performance of 802.11 networks, and designs a new predictive compensation method. Firstly, a hybrid NCS simulation environment based on the NS-2 simulator is designed and implemented. Secondly, a high-precision relative clock synchronization protocol is designed and implemented. Thirdly, transmission delays in 802.11 networks for soft-real-time control applications are modeled by use of a Markov chain model in which real-time Quality-of- Service parameters are analysed under a periodic traffic pattern. By using a Markov chain model, we can accurately model the tradeoff between real-time performance and throughput performance. Furthermore, a cross-layer optimisation scheme, featuring application-layer flow rate adaptation, is designed to achieve the tradeoff between certain real-time and throughput performance characteristics in a typical NCS scenario with wireless local area network. Fourthly, as a co-design approach for both a network and a controller, a new predictive compensation method for variable delay and packet loss in NCSs is designed, where simultaneous end-to-end delays and packet losses during packet transmissions from sensors to actuators is tackled. The effectiveness of the proposed predictive compensation approach is demonstrated using our hybrid NCS simulation environment.

An examination of MS candidate genes identified as differentially regulated in multiple sclerosis plaque tissue, using absolute and comparative real-time Q-PCR analysis

In our laboratory, we have developed methods in real-time detection and quantitative-polymerase chain reaction (Q-PCR) to analyse the relative levels of gene expression in post mortem brain tissues. We have then applied this method to examine differences in gene activity between normal white matter (NWM) and plaque tissue from multiple sclerosis (MS) patients. Genes were selected based on their association with pathology and through identification by previously conducted global gene expression analysis. Plaque tissue was obtained from secondary progressive (SP) patients displaying chronic active, as well as acute pathologies; while NWM from the same location was obtained from age- and sex-matched controls (normal patients). In this study, we used both SYBR Green I supplementation and commercially available mixes to assess both comparative and absolute levels of gene activity. The results of both methods compared favourably for four of the five genes examined (P < 0.05, Pearsons), while differences in gene expression between chronic active and acute pathologies were also identified. For example, a >50-fold increase in osteopontin (Spp1) and inositol 1-4-5 phosphate 3 kinase B (Itpkb) levels in acute plaques contrasted with the 5-fold or less increase in chronic active plaques (P < 0.05, unpaired t test). By contrast, there was no significant difference in the levels of the MS marker and calcium-dependent protease (Calpain, Capns1) in MS plaque tissue. In summary, Q-PCR analysis using SYBR Green I has allowed us to economically obtain what may be clinically significant information from small amounts of the CNS, providing an opportunity for further clinical investigations.

Multiplex real-time PCR for the detection and differentiation of equid herpesvirus 1 (EHV-1) and equid herpesvirus 4 (EHV-4).

A multiplex real-time PCR was designed to detect and differentiate equid herpesvirus 1 (EHV-1) and equid herpesvirus 4 (EHV-4). The PCR targets the glycoprotein B gene of EHV-1 and EHV-4. Primers and probes were specific to each equine herpesvirus type and can be used in monoplex or multiplex PCRs, allowing the differentiation of these two closely related members of the Alphaherpesvirinae. The two probes were minor-groove binding probes (MGB?) labelled with 6-carboxy-fluorescein (FAM?) and VIC® for detection of EHV-1 and EHV-4, respectively. Ten EHV-1 isolates, six EHV-1 positive clinical samples, one EHV-1 reference strain (EHV-1.438/77), three EHV-4 positive clinical samples, two EHV-4 isolates and one EHV-4 reference strain (EHV-4 405/76) were included in this study. EHV-1 isolates, clinical samples and the reference strain reacted in the EHV-1 real-time PCR but not in the EHV-4 real-time PCR and similarly EHV-4 clinical samples, isolates and the reference strain were positive in the EHV-4 real-time PCR but not in the EHV-1 real-time PCR. Other herpesviruses, such as EHV-2, EHV-3 and EHV-5 were all negative when tested using the multiplex real-time PCR. When bacterial pathogens and opportunistic pathogens were tested in the multiplex real-time PCR they did not react with either system. The multiplex PCR was shown to be sensitive and specific and is a useful tool for detection and differentiation of EHV-1 and EHV-4 in a single reaction. A comprehensive equine herpesvirus disease investigation procedure used in our laboratory is also outlined. This procedure describes the combination of alphaherpesvirus multiplex real-time PCR along with existing gel-based PCRs described by other authors.

Real-Time and Data-Driven Operation Optimization and Knowledge Discovery for an Enterprise Information System

An enterprise information system (EIS) is an integrated data-applications platform characterized by diverse, heterogeneous, and distributed data sources. For many enterprises, a number of business processes still depend heavily on static rule-based methods and extensive human expertise. Enterprises are faced with the need for optimizing operation scheduling, improving resource utilization, discovering useful knowledge, and making data-driven decisions.

This thesis research is focused on real-time optimization and knowledge discovery that addresses workflow optimization, resource allocation, as well as data-driven predictions of process-execution times, order fulfillment, and enterprise service-level performance. In contrast to prior work on data analytics techniques for enterprise performance optimization, the emphasis here is on realizing scalable and real-time enterprise intelligence based on a combination of heterogeneous system simulation, combinatorial optimization, machine-learning algorithms, and statistical methods.

On-demand digital-print service is a representative enterprise requiring a powerful EIS.We use real-life data from Reischling Press, Inc. (RPI), a digit-print-service provider (PSP), to evaluate our optimization algorithms.

In order to handle the increase in volume and diversity of demands, we first present a high-performance, scalable, and real-time production scheduling algorithm for production automation based on an incremental genetic algorithm (IGA). The objective of this algorithm is to optimize the order dispatching sequence and balance resource utilization. Compared to prior work, this solution is scalable for a high volume of orders and it provides fast scheduling solutions for orders that require complex fulfillment procedures. Experimental results highlight its potential benefit in reducing production inefficiencies and enhancing the productivity of an enterprise.

We next discuss analysis and prediction of different attributes involved in hierarchical components of an enterprise. We start from a study of the fundamental processes related to real-time prediction. Our process-execution time and process status prediction models integrate statistical methods with machine-learning algorithms. In addition to improved prediction accuracy compared to stand-alone machine-learning algorithms, it also performs a probabilistic estimation of the predicted status. An order generally consists of multiple series and parallel processes. We next introduce an order-fulfillment prediction model that combines advantages of multiple classification models by incorporating flexible decision-integration mechanisms. Experimental results show that adopting due dates recommended by the model can significantly reduce enterprise late-delivery ratio. Finally, we investigate service-level attributes that reflect the overall performance of an enterprise. We analyze and decompose time-series data into different components according to their hierarchical periodic nature, perform correlation analysis,

and develop univariate prediction models for each component as well as multivariate models for correlated components. Predictions for the original time series are aggregated from the predictions of its components. In addition to a significant increase in mid-term prediction accuracy, this distributed modeling strategy also improves short-term time-series prediction accuracy.

In summary, this thesis research has led to a set of characterization, optimization, and prediction tools for an EIS to derive insightful knowledge from data and use them as guidance for production management. It is expected to provide solutions for enterprises to increase reconfigurability, accomplish more automated procedures, and obtain data-driven recommendations or effective decisions.

Image, Video and 3D Data Registration: Medical, Satellite and Video Processing Applications with Quality Metrics

Data registration refers to a series of techniques for matching or bringing similar objects or datasets together into alignment. These techniques enjoy widespread use in a diverse variety of applications, such as video coding, tracking, object and face detection and recognition, surveillance and satellite imaging, medical image analysis and structure from motion. Registration methods are as numerous as their manifold uses, from pixel level and block or feature based methods to Fourier domain methods.

This book is focused on providing algorithms and image and video techniques for registration and quality performance metrics. The authors provide various assessment metrics for measuring registration quality alongside analyses of registration techniques, introducing and explaining both familiar and state-of-the-art registration methodologies used in a variety of targeted applications.

Key features:
- Provides a state-of-the-art review of image and video registration techniques, allowing readers to develop an understanding of how well the techniques perform by using specific quality assessment criteria
- Addresses a range of applications from familiar image and video processing domains to satellite and medical imaging among others, enabling readers to discover novel methodologies with utility in their own research
- Discusses quality evaluation metrics for each application domain with an interdisciplinary approach from different research perspectives

Image, video and 3D data registration: medical, satellite and video processing applications with quality metrics

Data registration refers to a series of techniques for matching or bringing similar objects or datasets together into alignment. These techniques enjoy widespread use in a diverse variety of applications, such as video coding, tracking, object and face detection and recognition, surveillance and satellite imaging, medical image analysis and structure from motion. Registration methods are as numerous as their manifold uses, from pixel level and block or feature based methods to Fourier domain methods. This book is focused on providing algorithms and image and video techniques for registration and quality performance metrics. The authors provide various assessment metrics for measuring registration quality alongside analyses of registration techniques, introducing and explaining both familiar and state–of–the–art registration methodologies used in a variety of targeted applications.