Improving the Accuracy of 3-D Reconstruction in Robotic Vision Applications

Thesis (Ph.D.)--University of Washington, 2016-08

Federated architecture for biomedical data integration

The last decades have been characterized by a continuous adoption of IT solutions in the healthcare sector, which resulted in the proliferation of tremendous amounts of data over heterogeneous systems. Distinct data types are currently generated, manipulated, and stored, in the several institutions where patients are treated. The data sharing and an integrated access to this information will allow extracting relevant knowledge that can lead to better diagnostics and treatments. This thesis proposes new integration models for gathering information and extracting knowledge from multiple and heterogeneous biomedical sources. The scenario complexity led us to split the integration problem according to the data type and to the usage specificity. The first contribution is a cloud-based architecture for exchanging medical imaging services. It offers a simplified registration mechanism for providers and services, promotes remote data access, and facilitates the integration of distributed data sources. Moreover, it is compliant with international standards, ensuring the platform interoperability with current medical imaging devices. The second proposal is a sensor-based architecture for integration of electronic health records. It follows a federated integration model and aims to provide a scalable solution to search and retrieve data from multiple information systems. The last contribution is an open architecture for gathering patient-level data from disperse and heterogeneous databases. All the proposed solutions were deployed and validated in real world use cases.

Hybrid materials for biomedical applications

The increased longevity of humans and the demand for a better quality of life have led to a continuous search for new implant materials. Scientific development coupled with a growing multidisciplinarity between materials science and life sciences has given rise to new approaches such as regenerative medicine and tissue engineering. The search for a material with mechanical properties close to those of human bone produced a new family of hybrid materials that take advantage of the synergy between inorganic silica (SiO4) domains, based on sol-gel bioactive glass compositions, and organic polydimethylsiloxane, PDMS ((CH3)2.SiO2)n, domains. Several studies have shown that hybrid materials based on the system PDMS-SiO2 constitute a promising group of biomaterials with several potential applications from bone tissue regeneration to brain tissue recovery, passing by bioactive coatings and drug delivery systems. The objective of the present work was to prepare hybrid materials for biomedical applications based on the PDMS-SiO2 system and to achieve a better understanding of the relationship among the sol-gel processing conditions, the chemical structures, the microstructure and the macroscopic properties. For that, different characterization techniques were used: Fourier transform infrared spectrometry, liquid and solid state nuclear magnetic resonance techniques, X-ray diffraction, small-angle X-ray scattering, smallangle neutron scattering, surface area analysis by Brunauer–Emmett–Teller method, scanning electron microscopy and transmission electron microscopy. Surface roughness and wettability were analyzed by 3D optical profilometry and by contact angle measurements respectively. Bioactivity was evaluated in vitro by immersion of the materials in Kokubos’s simulated body fluid and posterior surface analysis by different techniques as well as supernatant liquid analysis by inductively coupled plasma spectroscopy. Biocompatibility was assessed using MG63 osteoblastic cells. PDMS-SiO2-CaO materials were first prepared using nitrate as a calcium source. To avoid the presence of nitrate residues in the final product due to its potential toxicity, a heat-treatment step (above 400 °C) is required. In order to enhance the thermal stability of the materials subjected to high temperatures titanium was added to the hybrid system, and a material containing calcium, with no traces of nitrate and the preservation of a significant amount of methyl groups was successfully obtained. The difficulty in eliminating all nitrates from bulk PDMS-SiO2-CaO samples obtained by sol-gel synthesis and subsequent heat-treatment created a new goal which was the search for alternative sources of calcium. New calcium sources were evaluated in order to substitute the nitrate and calcium acetate was chosen due to its good solubility in water. Preparation solgel protocols were tested and homogeneous monolithic samples were obtained. Besides their ability to improve the bioactivity, titanium and zirconium influence the structural and microstructural features of the SiO2-TiO2 and SiO2-ZrO2 binary systems, and also of the PDMS-TiO2 and PDMS-ZrO2 systems. Detailed studies with different sol-gel conditions allowed the understanding of the roles of titanium and zirconium as additives in the PDMS-SiO2 system. It was concluded that titanium and zirconium influence the kinetics of the sol-gel process due to their different alkoxide reactivity leading to hybrid xerogels with dissimilar characteristics and morphologies. Titanium isopropoxide, less reactive than zirconium propoxide, was chosen as source of titanium, used as an additive to the system PDMS-SiO2-CaO. Two different sol-gel preparation routes were followed, using the same base composition and calcium acetate as calcium source. Different microstructures with high hydrophobicit were obtained and both proved to be biocompatible after tested with MG63 osteoblastic cells. Finally, the role of strontium (typically known in bioglasses to promote bone formation and reduce bone resorption) was studied in the PDMS-SiO2-CaOTiO2 hybrid system. A biocompatible material, tested with MG63 osteoblastic cells, was obtained with the ability to release strontium within the values reported as suitable for bone tissue regeneration.

Optical micro-tomography “OPenT” allows the study of large toadfish Halobatrachus didactylus embryos and larvae

Batrachoidids, which include midshipman and toadfish are less known among embryologists, but are common in other fields. They are characteristic for their acoustic communication, and develop hearing and sound production while young juveniles. They lay large benthic eggs (>5mm) with a thick chorion and adhesive disk and slow development, which are particularly challenging for studying embryology. Here we took advantage of a classical tissue clearing technique and the OPenT open-source platform for optical tomography imaging, to image a series of embryos and larvae from 3 to 30mm in length, which allowed detailed 3D anatomical reconstructions non-destructively. We documented some of the developmental stages (early and late in development) and the anatomy of the delicate stato-acoustic organs, swimming bladder and associated sonic muscles. Compared to other techniques accessible to developmental biology labs, OPenT provided advantages in terms of image quality, cost of operation and data throughput, allowing identification and quantitative morphometrics of organs in larvae, earlier and with higher accuracy than is possible with other imaging techniques.

In vivo microvascular oximetry using multispectral imaging

This thesis describes the application of multispectral imaging to several novel oximetry applications. Chapter 1 motivates optical microvascular oximetry, outlines oxygen transport in the body, describes the theory of oximetry, and describes the challenges associated with in vivo oximetry, in particular imaging through tissue. Chapter 2 reviews various imaging techniques for quantitative in vivo oximetry of the microvasculature, including multispectral and hyperspectral imaging, photoacoustic imaging, optical coherence tomography, and laser speckle techniques. Chapter 3 describes a two-wavelength oximetry study of two microvascular beds in the anterior segment of the eye: the bulbar conjunctival and episcleral microvasculature. This study reveals previously unseen oxygen diffusion from ambient air into the bulbar conjunctival microvasculature, altering the oxygen saturation of the bulbar conjunctiva. The response of the bulbar conjunctival and episcleral microvascular beds to acute mild hypoxia is quantified and the rate at which oxygen diffuses into bulbar conjunctival vessels is measured. Chapter 4 describes the development and application of a highly novel non-invasive retinal angiography technique: Oximetric Ratio Contrast Angiography (ORCA). ORCA requires only multispectral imaging and a small perturbation of blood oxygen saturation to produce angiographic sequences. A pilot study of ORCA in human subjects was conducted. This study demonstrates that ORCA can produce angiographic sequences with features such as sequential vessel filling and laminar flow. The application and challenges of ORCA are discussed, with emphasis on comparison with other angiography techniques, such as fluorescein angiography. Chapter 5 describes the development of a multispectral microscope for oximetry in the spinal cord dorsal vein of rats. Measurements of blood oxygen saturation are made in the dorsal vein of both healthy rats, and in rats with the Experimental autoimmune encephalomyelitis (EAE) disease model of multiple sclerosis. The venous blood oxygen saturation of EAE disease model rats was found to be significantly lower than that of healthy controls, indicating increased oxygen uptake from blood in the EAE disease model of multiple sclerosis. Chapter 6 describes the development of video-rate red eye oximetry; a technique which could enable stand-off oximetry of the blood-supply of the eye with high temporal resolution. The various challenges associated with video-rate red eye oximetry are investigated and their influence quantified. The eventual aim of this research is to track circulating deoxygenation perturbations as they arrive in both eyes, which could provide a screening method for carotid artery stenosis, which is major risk-factor for stroke. However, due to time constraints, it was not possible to thoroughly investigate if video-rate red eye can detect such perturbations. Directions and recommendations for future research are outlined.

MICROFLUIDIC ASSAY PERFORMANCE ENHANCEMENT USING POROUS VOLUMETRIC DETECTION ELEMENTS FOR IMPEDEMETRIC AND OPTICAL SENSING

Microfluidic technologies have great potential to help create automated, cost-effective, portable devices for rapid point of care (POC) diagnostics in diverse patient settings. Unfortunately commercialization is currently constrained by the materials, reagents, and instrumentation required and detection element performance. While most microfluidic studies utilize planar detection elements, this dissertation demonstrates the utility of porous volumetric detection elements to improve detection sensitivity and reduce assay times. Impedemetric immunoassays were performed utilizing silver enhanced gold nanoparticle immunoconjugates (AuIgGs) and porous polymer monolith or silica bead bed detection elements within a thermoplastic microchannel. For a direct assay with 10 µm spaced electrodes the detection limit was 0.13 fM AuIgG with a 3 log dynamic range. The same assay was performed with electrode spacing of 15, 40, and 100 µm with no significant difference between configurations. For a sandwich assay the detection limit was10 ng/mL with a 4 log dynamic range. While most impedemetric assays rely on expensive high resolution electrodes to enhance planar senor performance, this study demonstrates the employment of porous volumetric detection elements to achieve similar performance using lower resolution electrodes and shorter incubation times. Optical immunoassays were performed using porous volumetric capture elements perfused with refractive index matching solutions to limit light scattering and enhance signal. First, fluorescence signal enhancement was demonstrated with a porous polymer monolith within a silica capillary. Next, transmission enhancement of a direct assay was demonstrated by infusing aqueous sucrose solutions through silica bead beds with captured silver enhanced AuIgGs yielding a detection limit of 0.1 ng/mL and a 5 log dynamic range. Finally, ex situ functionalized porous silica monolith segments were integrated into thermoplastic channels for a reflectance based sandwich assay yielding a detection limit of 1 ng/mL and a 5 log dynamic range. The simple techniques for optical signal enhancement and ex situ element integration enable development of sensitive, multiplexed microfluidic sensors. Collectively the demonstrated experiments validate the use of porous volumetric detection elements to enhance impedemetric and optical microfluidic assays. The techniques rely on commercial reagents, materials compatible with manufacturing, and measurement instrumentation adaptable to POC diagnostics.

Optical Wireless Communication for Mobile Platforms

The past few decades have witnessed the widespread adaptation of wireless devices such as cellular phones and Wifi-connected laptops, and demand for wireless communication is expected to continue to increase. Though radio frequency (RF) communication has traditionally dominated in this application space, recent decades have seen an increasing interest in the use of optical wireless (OW) communication to supplement RF communications. In contrast to RF communication technology, OW systems offer the use of largely unregulated electromagnetic spectrum and large bandwidths for communication. They also offer the potential to be highly secure against jamming and eavesdropping. Interest in OW has become especially keen in light of the maturation of light-emitting diode (LED) technology. This maturation, and the consequent emerging ubiquity of LED technology in lighting systems, has motivated the exploration of LEDs for wireless communication purposes in a wide variety of applications. Recent interest in this field has largely focused on the potential for indoor local area networks (LANs) to be realized with increasingly common LED-based lighting systems. We envision the use of LED-based OW to serve as a supplement to RF technology in communication between mobile platforms, which may include automobiles, robots, or unmanned aerial vehicles (UAVs). OW technology may be especially useful in what are known as RF-denied environments, in which RF communication may be prohibited or undesirable. The use of OW in these settings presents major challenges. In contrast to many RF systems, OWsystems that operate at ranges beyond a few meters typically require relatively precise alignment. For example, some laser-based optical wireless communication systems require alignment precision to within small fractions of a degree. This level of alignment precision can be difficult to maintain between mobile platforms. Additionally, the use of OW systems in outdoor settings presents the challenge of interference from ambient light, which can be much brighter than any LED transmitter. This thesis addresses these challenges to the use of LED-based communication between mobile platforms. We propose and analyze a dual-link LED-based system that uses one link with a wide transmission beam and relaxed alignment constraints to support a more narrow, precisely aligned, higher-data-rate link. The use of an optical link with relaxed alignment constraints to support the alignment of a more precisely aligned link motivates our exploration of a panoramic imaging receiver for estimating the range and bearing of neighboring nodes. The precision of such a system is analyzed and an experimental system is realized. Finally, we present an experimental prototype of a self-aligning LED-based link.

Endoscopic optical coherence tomography : design and application

This thesis presents an investigation on endoscopic optical coherence tomography (OCT). As a noninvasive imaging modality, OCT emerges as an increasingly important diagnostic tool for many clinical applications. Despite of many of its merits, such as high resolution and depth resolvability, a major limitation is the relatively shallow penetration depth in tissue (about 2∼3 mm). This is mainly due to tissue scattering and absorption. To overcome this limitation, people have been developing many different endoscopic OCT systems. By utilizing a minimally invasive endoscope, the OCT probing beam can be brought to the close vicinity of the tissue of interest and bypass the scattering of intervening tissues so that it can collect the reflected light signal from desired depth and provide a clear image representing the physiological structure of the region, which can not be disclosed by traditional OCT. In this thesis, three endoscope designs have been studied. While they rely on vastly different principles, they all converge to solve this long-standing problem.

A hand-held endoscope with manual scanning is first explored. When a user is holding a hand- held endoscope to examine samples, the movement of the device provides a natural scanning. We proposed and implemented an optical tracking system to estimate and record the trajectory of the device. By registering the OCT axial scan with the spatial information obtained from the tracking system, one can use this system to simply ‘paint’ a desired volume and get any arbitrary scanning pattern by manually waving the endoscope over the region of interest. The accuracy of the tracking system was measured to be about 10 microns, which is comparable to the lateral resolution of most OCT system. Targeted phantom sample and biological samples were manually scanned and the reconstructed images verified the method.

Next, we investigated a mechanical way to steer the beam in an OCT endoscope, which is termed as Paired-angle-rotation scanning (PARS). This concept was proposed by my colleague and we further developed this technology by enhancing the longevity of the device, reducing the diameter of the probe, and shrinking down the form factor of the hand-piece. Several families of probes have been designed and fabricated with various optical performances. They have been applied to different applications, including the collector channel examination for glaucoma stent implantation, and vitreous remnant detection during live animal vitrectomy.

Lastly a novel non-moving scanning method has been devised. This approach is based on the EO effect of a KTN crystal. With Ohmic contact of the electrodes, the KTN crystal can exhibit a special mode of EO effect, termed as space-charge-controlled electro-optic effect, where the carrier electron will be injected into the material via the Ohmic contact. By applying a high voltage across the material, a linear phase profile can be built under this mode, which in turn deflects the light beam passing through. We constructed a relay telescope to adapt the KTN deflector into a bench top OCT scanning system. One of major technical challenges for this system is the strong chromatic dispersion of KTN crystal within the wavelength band of OCT system. We investigated its impact on the acquired OCT images and proposed a new approach to estimate and compensate the actual dispersion. Comparing with traditional methods, the new method is more computational efficient and accurate. Some biological samples were scanned by this KTN based system. The acquired images justified the feasibility of the usage of this system into a endoscopy setting. My research above all aims to provide solutions to implement an OCT endoscope. As technology evolves from manual, to mechanical, and to electrical approaches, different solutions are presented. Since all have their own advantages and disadvantages, one has to determine the actual requirements and select the best fit for a specific application.

INTEGRATION OF INSTRUMENTATION AND PROCESSING SOFTWARE OF A LASER SPECKLE CONTRAST IMAGING SYSTEM

Laser speckle contrast imaging (LSCI) has the potential to be a powerful tool in medicine, but more research in the field is required so it can be used properly. To help in the progression of Michigan Tech's research in the field, a graphical user interface (GUI) was designed in Matlab to control the instrumentation of the experiments as well as process the raw speckle images into contrast images while they are being acquired. The design of the system was successful and is currently being used by Michigan Tech's Biomedical Engineering department. This thesis describes the development of the LSCI GUI as well as offering a full introduction into the history, theory and applications of LSCI.

Funtional Near Infrared Spectroscopy Study of Language, Joint Attention and Motor Skills

Near infrared spectroscopy (NIRS) is an emerging non-invasive optical neuro imaging technique that monitors the hemodynamic response to brain activation with ms-scale temporal resolution and sub-cm spatial resolution. The overall goal of my dissertation was to develop and apply NIRS towards investigation of neurological response to language, joint attention and planning and execution of motor skills in healthy adults. Language studies were performed to investigate the hemodynamic response, synchrony and dominance feature of the frontal and fronto-temporal cortex of healthy adults in response to language reception and expression. The mathematical model developed based on granger causality explicated the directional flow of information during the processing of language stimuli by the fronto-temporal cortex. Joint attention and planning/ execution of motor skill studies were performed to investigate the hemodynamic response, synchrony and dominance feature of the frontal cortex of healthy adults and in children (5-8 years old) with autism (for joint attention studies) and individuals with cerebral palsy (for planning/execution of motor skills studies). The joint attention studies on healthy adults showed differences in activation as well as intensity and phase dependent connectivity in the frontal cortex during joint attention in comparison to rest. The joint attention studies on typically developing children showed differences in frontal cortical activation in comparison to that in children with autism. The planning and execution of motor skills studies on healthy adults and individuals with cerebral palsy (CP) showed difference in the frontal cortical dominance, that is, bilateral and ipsilateral dominance, respectively. The planning and execution of motor skills studies also demonstrated the plastic and learning behavior of brain wherein correlation was found between the relative change in total hemoglobin in the frontal cortex and the kinematics of the activity performed by the participants. Thus, during my dissertation the NIRS neuroimaging technique was successfully implemented to investigate the neurological response of language, joint attention and planning and execution of motor skills in healthy adults as well as preliminarily on children with autism and individuals with cerebral palsy. These NIRS studies have long-term potential for the design of early stage interventions in children with autism and customized rehabilitation in individuals with cerebral palsy.

Bionano Electronics: Magneto-Electric Nanoparticles for Drug Delivery, Brain Stimulation and Imaging Applications

Nanoparticles are often considered as efficient drug delivery vehicles for precisely dispensing the therapeutic payloads specifically to the diseased sites in the patient’s body, thereby minimizing the toxic side effects of the payloads on the healthy tissue. However, the fundamental physics that underlies the nanoparticles’ intrinsic interaction with the surrounding cells is inadequately elucidated. The ability of the nanoparticles to precisely control the release of its payloads externally (on-demand) without depending on the physiological conditions of the target sites has the potential to enable patient- and disease-specific nanomedicine, also known as Personalized NanoMedicine (PNM). In this dissertation, magneto-electric nanoparticles (MENs) were utilized for the first time to enable important functions, such as (i) field-controlled high-efficacy dissipation-free targeted drug delivery system and on-demand release at the sub-cellular level, (ii) non-invasive energy-efficient stimulation of deep brain tissue at body temperature, and (iii) a high-sensitivity contrasting agent to map the neuronal activity in the brain non-invasively. First, this dissertation specifically focuses on using MENs as energy-efficient and dissipation-free field-controlled nano-vehicle for targeted delivery and on-demand release of a anti-cancer Paclitaxel (Taxol) drug and a anti-HIV AZT 5’-triphosphate (AZTTP) drug from 30-nm MENs (CoFe2O4-BaTiO3) by applying low-energy DC and low-frequency (below 1000 Hz) AC fields to separate the functions of delivery and release, respectively. Second, this dissertation focuses on the use of MENs to non-invasively stimulate the deep brain neuronal activity via application of a low energy and low frequency external magnetic field to activate intrinsic electric dipoles at the cellular level through numerical simulations. Third, this dissertation describes the use of MENs to track the neuronal activities in the brain (non-invasively) using a magnetic resonance and a magnetic nanoparticle imaging by monitoring the changes in the magnetization of the MENs surrounding the neuronal tissue under different states. The potential therapeutic and diagnostic impact of this innovative and novel study is highly significant not only in HIV-AIDS, Cancer, Parkinson’s and Alzheimer’s disease but also in many CNS and other diseases, where the ability to remotely control targeted drug delivery/release, and diagnostics is the key.

Microstructured polymer optical fibre sensors for opto-acoustic endoscopy

Opto-acoustic imaging is a growing field of research in recent years, providing functional imaging of physiological biomarkers, such as the oxygenation of haemoglobin. Piezo electric transducers are the industry standard detector for ultrasonics, but their limited bandwidth, susceptibility to electromagnetic interference and their inversely proportional sensitivity to size all affect the detector performance. Sensors based on polymer optical fibres (POF) are immune to electromagnetic interference, have lower acoustic impedance and a reduced Young's Modulus compared to silica fibres. Furthermore, POF enables the possibility of a wideband sensor and a size appropriate to endoscopy. Micro-structured POF (mPOF) used in an interferometric detector has been shown to be an order of magnitude more sensitive than silica fibre at 1 MHz and 3 times more sensitive at 10 MHz. We present the first opto-acoustic measurements obtained using a 4.7mm PMMA mPOF Bragg grating with a fibre diameter of 130 μm and present the lateral directivity pattern of a PMMA mPOF FBG ultrasound sensor over a frequency range of 1-50 MHz. We discuss the impact of the pattern with respect to the targeted application and draw conclusions on how to mitigate the problems encountered.

Sub-kiloparsec alma imaging of compact star-forming galaxies at z ~ 2.5: revealing the formation of dense galactic cores in the progenitors of compact quiescent galaxies

We present spatially resolved Atacama Large Millimeter/submillimeter Array (ALMA) 870 μm dust continuum maps of six massive, compact, dusty star-forming galaxies at z ~ 2.5. These galaxies are selected for their small rest-frame optical sizes (r_e,F160W ~ 1.6 kpc) and high stellar mass densities that suggest that they are direct progenitors of compact quiescent galaxies at z ~ 2. The deep observations yield high far-infrared (FIR) luminosities of L_IR = 10^12.3-12.8 L_⨀ and star formation rates (SFRs) of SFR = 200–700 M_⊙ yr^−1, consistent with those of typical star-forming "main sequence" galaxies. The high spatial resolution (FWHM ~ 0 12–0 18) ALMA and Hubble Space Telescope photometry are combined to construct deconvolved, mean radial profiles of their stellar mass and (UV+IR) SFR. We find that the dusty, nuclear IR–SFR overwhelmingly dominates the bolometric SFR up to r ~ 5 kpc, by a factor of over 100× from the unobscured UV–SFR. Furthermore, the effective radius of the mean SFR profile (r_e,SFR ~ 1 kpc) is ~30% smaller than that of the stellar mass profile. The implied structural evolution, if such nuclear starburst last for the estimated gas depletion time of Δt = ±100 Myr, is a 4×increase of the stellar mass density within the central 1 kpc and a 1.6× decrease of the half-mass–radius. This structural evolution fully supports dissipation-driven, formation scenarios in which strong nuclear starbursts transform larger, star-forming progenitors into compact quiescent galaxies.

3D-form metrology of arbitrary optical surfaces by absorption in fluids

We present an imaging technique for the 3D-form metrology of optical surfaces. It is based on the optical absorption in fluids situated between the surface and a reference. An improved setup with a bi-chromatic light source is fundamental to obtain reliable topographic maps. It is able to measure any surface finish (rough or polished), form and slope and independently of scale. We present results focused on flat and spherical optical surfaces, arrays of lenses and with different surface finish (rough-polished). We achieve form accuracies from several nanometers to sub-lambda for sag departures from tens to hundred of microns. Therefore, it seems suitable for the quality control in the production of precision aspheric, freeform lenses and other complex shapes on transparent substrates, independently of the surface finish.