Towards autonomous image fusion

Mobile robots are providing great assistance operating in hazardous environments such as nuclear cores, battlefields, natural disasters, and even at the nano-level of human cells. These robots are usually equipped with a wide variety of sensors in order to collect data and guide their navigation. Whether a single robot operating all sensors or a swarm of cooperating robots operating their special sensors, the captured data can be too large to be transferred across limited resources (e.g. bandwidth, battery, processing, and response time) in hazardous environments. Therefore, local computations have to be carried out on board the swarming robots to assess the worthiness of captured data and the capacity of fused information in a certain spatial dimension as well as selection of proper combination of fusion algorithms and metrics. This paper introduces to the concepts of Type-I and Type-II fusion errors, fusion capacity, and fusion worthiness. These concepts together form the ladder leading to autonomous fusion systems.

Measuring the capacity of image fusion

Multisource image fusion is usually achieved by repeatedly fusing source images in pairs. However, there is no guarantee on the delivered quality considering the amount of information to be squeezed into the same spatial dimension. This paper presents a fusion capacity measure and examines the limits at which fusing more images will not add further information. The fusion capacity index employs Mutual Information (MI) to measure how far the histogram of the examined image is from a uniformly distributed histogram of a saturated image.

Image fusion metrics : evolution in a nutshell

Image fusion process merges two images into a single more informative image. Objective image fusion per- formance metrics rely primarily on measuring the amount of information transferred from each source image into the fused image. Objective image fusion metrics have evolved from image processing dissimilarity metrics. Additionally, researchers have developed many additions to image dissimilarity metrics in order to better value the local fusion worthy features in source images. This paper studies the evolution of objective image fusion performance metrics and their subjective and objective validation. It describes how a fusion performance metric evolves starting with image dissimilarity metrics, its realization into image fusion contexts, its localized weighting factors and the validation process.

Land cover classification of Lago Grande de Curuai floodplain (Amazon, Brazil) using multi-sensor and image fusion techniques

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Seeking tools for image fusion between computed tomography, structural and functional magnetic resonance methods for applications in neurosurgery

OBJECTIVE: To evaluate tools for the fusion of images generated by tomography and structural and functional magnetic resonance imaging. METHODS: Magnetic resonance and functional magnetic resonance imaging were performed while a volunteer who had previously undergone cranial tomography performed motor and somatosensory tasks in a 3-Tesla scanner. Image data were analyzed with different programs, and the results were compared. RESULTS: We constructed a flow chart of computational processes that allowed measurement of the spatial congruence between the methods. There was no single computational tool that contained the entire set of functions necessary to achieve the goal. CONCLUSION: The fusion of the images from the three methods proved to be feasible with the use of four free-access software programs (OsiriX, Register, MRIcro and FSL). Our results may serve as a basis for building software that will be useful as a virtual tool prior to neurosurgery.

Cardiac image fusion from stand-alone SPECT and CT: clinical experience

Myocardial perfusion imaging with SPECT (SPECT-MPI) and 64-slice CT angiography (CTA) are both established techniques for the noninvasive evaluation of coronary artery disease (CAD). Three-dimensional (3D) SPECT/CT image fusion may offer an incremental diagnostic value by integrating both sets of information. We report our first clinical experiences with fused 3D SPECT/CT in CAD patients. METHODS: Thirty-eight consecutive patients with at least 1 perfusion defect on SPECT-MPI (1-d adenosine stress/rest SPECT with (99m)Tc-tetrofosmin) and 64-slice CTA were included. 3D volume-rendered fused SPECT/CT images were generated and compared with the findings from the side-by-side analysis with regard to coronary lesion interpretation by assigning the perfusion defects to their corresponding coronary lesion. RESULTS: The fused SPECT/CT images added information on pathophysiologic lesion severity in 27 coronary stenoses (22%) of 12 patients (29%) (P<0.001). Among 40 equivocal lesions on side-by-side analysis, the fused interpretation confirmed hemodynamic significance in 14 lesions and excluded functional relevance in 10 lesions. In 3 lesions, assignment of perfusion defect and coronary lesion appeared to be reliable on side-by-side analysis but proved to be inaccurate on fused interpretation. Added diagnostic information by SPECT/CT was more commonly found in patients with stenoses of small vessels (P=0.004) and involvement of diagonal branches (P=0.01). CONCLUSION: In addition to being intuitively convincing, 3D SPECT/CT fusion images in CAD may provide added diagnostic information on the functional relevance of coronary artery lesions.

Improved Localization of Implanted Subdural Electrode Contacts on MRI Using an Elastic Image Fusion Algorithm in Invasive EEG Recording

BACKGROUND: Accurate projection of implanted subdural electrode contacts in presurgical evaluation of pharmacoresistant epilepsy cases by invasive EEG is highly relevant. Linear fusion of CT and MRI images may display the contacts in the wrong position due to brain shift effects. OBJECTIVE: A retrospective study in five patients with pharmacoresistant epilepsy was performed to evaluate whether an elastic image fusion algorithm can provide a more accurate projection of the electrode contacts on the pre-implantation MRI as compared to linear fusion. METHODS: An automated elastic image fusion algorithm (AEF), a guided elastic image fusion algorithm (GEF), and a standard linear fusion algorithm (LF) were used on preoperative MRI and post-implantation CT scans. Vertical correction of virtual contact positions, total virtual contact shift, corrections of midline shift and brain shifts due to pneumencephalus were measured. RESULTS: Both AEF and GEF worked well with all 5 cases. An average midline shift of 1.7mm (SD 1.25) was corrected to 0.4mm (SD 0.8) after AEF and to 0.0mm (SD 0) after GEF. Median virtual distances between contacts and cortical surface were corrected by a significant amount, from 2.3mm after LF to 0.0mm after AEF and GEF (p<.001). Mean total relative corrections of 3.1 mm (SD 1.85) after AEF and 3.0mm (SD 1.77) after GEF were achieved. The tested version of GEF did not achieve a satisfying virtual correction of pneumencephalus. CONCLUSION: The technique provided a clear improvement in fusion of pre- and post-implantation scans, although the accuracy is difficult to evaluate.

Evaluation of treatment response after chemoembolisation (TACE) in hepatocellular carcinoma using real time image fusion of contrast-enhanced ultrasound (CEUS) and computed tomography (CT)--preliminary results.

OBJECTIVE To evaluate treatment response of hepatocellular carcinoma (HCC) after transarterial chemoembolization (TACE) with a new real-time imaging fusion technique of contrast-enhanced ultrasound (CEUS) with multi-slice detection computed tomography (CT) in comparison to conventional post-interventional follow-up. MATERIAL AND METHODS 40 patients with HCC (26 male, ages 46-81 years) were evaluated 24 hours after TACE using CEUS with ultrasound volume navigation and image fusion with CT compared to non-enhanced CT and follow-up contrast-enhanced CT after 6-8 weeks. Reduction of tumor vascularization to less than 25% was regarded as "successful" treatment, whereas reduction to levels >25% was considered as "partial" treatment response. Homogenous lipiodol retention was regarded as successful treatment in non-enhanced CT. RESULTS Post-interventional image fusion of CEUS with CT was feasible in all 40 patients. In 24 patients (24/40), post-interventional image fusion with CEUS revealed residual tumor vascularity, that was confirmed by contrast-enhanced CT 6-8 weeks later in 24/24 patients. In 16 patients (16/40), post-interventional image fusion with CEUS demonstrated successful treatment, but follow-up CT detected residual viable tumor (6/16). Non-enhanced CT did not identify any case of treatment failure. Image fusion with CEUS assessed treatment efficacy with a specificity of 100%, sensitivity of 80% and a positive predictive value of 1 (negative predictive value 0.63). CONCLUSIONS Image fusion of CEUS with CT allows a reliable, highly specific post-interventional evaluation of embolization response with good sensitivity without any further radiation exposure. It can detect residual viable tumor at early state, resulting in a close patient monitoring or re-therapy.