Computational modeling and psychophysics in low- and mid-level vision

This thesis addresses a series of topics related to the question of how people find the foreground objects from complex scenes. With both computer vision modeling, as well as psychophysical analyses, we explore the computational principles for low- and mid-level vision.

We first explore the computational methods of generating saliency maps from images and image sequences. We propose an extremely fast algorithm called Image Signature that detects the locations in the image that attract human eye gazes. With a series of experimental validations based on human behavioral data collected from various psychophysical experiments, we conclude that the Image Signature and its spatial-temporal extension, the Phase Discrepancy, are among the most accurate algorithms for saliency detection under various conditions.

In the second part, we bridge the gap between fixation prediction and salient object segmentation with two efforts. First, we propose a new dataset that contains both fixation and object segmentation information. By simultaneously presenting the two types of human data in the same dataset, we are able to analyze their intrinsic connection, as well as understanding the drawbacks of today’s “standard” but inappropriately labeled salient object segmentation dataset. Second, we also propose an algorithm of salient object segmentation. Based on our novel discoveries on the connections of fixation data and salient object segmentation data, our model significantly outperforms all existing models on all 3 datasets with large margins.

In the third part of the thesis, we discuss topics around the human factors of boundary analysis. Closely related to salient object segmentation, boundary analysis focuses on delimiting the local contours of an object. We identify the potential pitfalls of algorithm evaluation for the problem of boundary detection. Our analysis indicates that today’s popular boundary detection datasets contain significant level of noise, which may severely influence the benchmarking results. To give further insights on the labeling process, we propose a model to characterize the principles of the human factors during the labeling process.

The analyses reported in this thesis offer new perspectives to a series of interrelating issues in low- and mid-level vision. It gives warning signs to some of today’s “standard” procedures, while proposing new directions to encourage future research.

Cell-targeted regulation of gene expression through synthetic RNA devices

The ability to interface with and program cellular function remains a challenging research frontier in biotechnology. Although the emerging field of synthetic biology has recently generated a variety of gene-regulatory strategies based on synthetic RNA molecules, few strategies exist through which to control such regulatory effects in response to specific exogenous or endogenous molecular signals. Here, we present the development of an engineered RNA-based device platform to detect and act on endogenous protein signals, linking these signals to the regulation of genes and thus cellular function.

We describe efforts to develop an RNA-based device framework for regulating endogenous genes in human cells. Previously developed RNA control devices have demonstrated programmable ligand-responsive genetic regulation in diverse cell types, and we attempted to adapt this class of cis-acting control elements to function in trans. We divided the device into two strands that reconstitute activity upon hybridization. Device function was optimized using an in vivo model system, and we found that device sequence is not as flexible as previously reported. After verifying the in vitro activity of our optimized design, we attempted to establish gene regulation in a human cell line using additional elements to direct device stability, structure, and localization. The significant limitations of our platform prevented endogenous gene regulation.

We next describe the development of a protein-responsive RNA-based regulatory platform. Employing various design strategies, we demonstrated functional devices that both up- and downregulate gene expression in response to a heterologous protein in a human cell line. The activity of our platform exceeded that of a similar, small-molecule-responsive platform. We demonstrated the ability of our devices to respond to both cytoplasmic- and nuclear-localized protein, providing insight into the mechanism of action and distinguishing our platform from previously described devices with more restrictive ligand localization requirements. Finally, we demonstrated the versatility of our device platform by developing a regulatory device that responds to an endogenous signaling protein.

The foundational tool we present here possesses unique advantages over previously described RNA-based gene-regulatory platforms. This genetically encoded technology may find future applications in the development of more effective diagnostic tools and targeted molecular therapy strategies.

Dissecting neural circuits for vision in nonhuman primates using fMRI-guided electrophysiology and optogenetics

The visual system is a remarkable platform that evolved to solve difficult computational problems such as detection, recognition, and classification of objects. Of great interest is the face-processing network, a sub-system buried deep in the temporal lobe, dedicated for analyzing specific type of objects (faces). In this thesis, I focus on the problem of face detection by the face-processing network. Insights obtained from years of developing computer-vision algorithms to solve this task have suggested that it may be efficiently and effectively solved by detection and integration of local contrast features. Does the brain use a similar strategy? To answer this question, I embark on a journey that takes me through the development and optimization of dedicated tools for targeting and perturbing deep brain structures. Data collected using MR-guided electrophysiology in early face-processing regions was found to have strong selectivity for contrast features, similar to ones used by artificial systems. While individual cells were tuned for only a small subset of features, the population as a whole encoded the full spectrum of features that are predictive to the presence of a face in an image. Together with additional evidence, my results suggest a possible computational mechanism for face detection in early face processing regions. To move from correlation to causation, I focus on adopting an emergent technology for perturbing brain activity using light: optogenetics. While this technique has the potential to overcome problems associated with the de-facto way of brain stimulation (electrical microstimulation), many open questions remain about its applicability and effectiveness for perturbing the non-human primate (NHP) brain. In a set of experiments, I use viral vectors to deliver genetically encoded optogenetic constructs to the frontal eye field and faceselective regions in NHP and examine their effects side-by-side with electrical microstimulation to assess their effectiveness in perturbing neural activity as well as behavior. Results suggest that cells are robustly and strongly modulated upon light delivery and that such perturbation can modulate and even initiate motor behavior, thus, paving the way for future explorations that may apply these tools to study connectivity and information flow in the face processing network.

New optoelectronic devices using GaAs-GaAlAs epitaxy

Three subjects related to epitaxial GaAs-GaAlAs optoelectronic devices are discussed in this thesis. They are:

1. Embedded Epitaxy

This is a technique of selective multilayer growth of GaAs- Ga_1-xAl_xAs single crystal structures through stripe openings in masking layers on GaAs substrates. This technique results in prismatic layers of GaAs and Ga_1-xAl_xAs "embedded" in each other and leads to controllable uniform structures terminated by crystal faces. The dependence of the growth habit on the orientation of the stripe openings has been studied. Room temperature embedded double heterostructure lasers have been fabricated using this technique. Threshold current densities as low as 1.5 KA/cm² have been achieved.

2. Barrier Controlled PNPN Laser Diode

It is found that the I-V characteristics of a PNPN device can be controlled by using potential barriers in the base regions. Based on this principle, GaAs-GaAlAs heterostructure PNPN laser diodes have been fabricated. GaAlAs potential barriers in the bases control not only the electrical but also the optical properties of the device. PNPN lasers with low threshold currents and high breakover voltage have been achieved. Numerical calculations of this barrier controlled structure are presented in the ranges where the total current is below the holding point and near the lasing threshold.

3. Injection Lasers on Semi-Insulating Substrates

GaAs-GaAlAs heterostructure lasers fabricated on semi-insulating substrates have been studied. Two different laser structures achieved are: (1) Crowding effect lasers, (2) Lateral injection lasers. Experimental results and the working principles underlying the operation of these lasers are presented. The gain induced guiding mechanism is used to explain the lasers' far field radiation patterns. It is found that Zn diffusion in Ga_1-xAl_xAs depends on the Al content x, and that GaAs can be used as the diffusion mask for Zn diffusion in Ga_1-xAl_xAs. Lasers having very low threshold currents and operating in a stable single mode have been achieved. Because these lasers are fabricated on semi-insulating substrates, it is possible to integrate them with other electronic devices on the same substrate. An integrated device, which consists of a crowding effect laser and a Gunn oscillator on a common semi-insulating GaAs substrate, has been achieved.

Acceleration Sensing, Feedback Cooling, and Nonlinear Dynamics with Nanoscale Cavity-Optomechanical Devices

Light has long been used for the precise measurement of moving bodies, but the burgeoning field of optomechanics is concerned with the interaction of light and matter in a regime where the typically weak radiation pressure force of light is able to push back on the moving object. This field began with the realization in the late 1960's that the momentum imparted by a recoiling photon on a mirror would place fundamental limits on the smallest measurable displacement of that mirror. This coupling between the frequency of light and the motion of a mechanical object does much more than simply add noise, however. It has been used to cool objects to their quantum ground state, demonstrate electromagnetically-induced-transparency, and modify the damping and spring constant of the resonator. Amazingly, these radiation pressure effects have now been demonstrated in systems ranging 18 orders of magnitude in mass (kg to fg).

In this work we will focus on three diverse experiments in three different optomechanical devices which span the fields of inertial sensors, closed-loop feedback, and nonlinear dynamics. The mechanical elements presented cover 6 orders of magnitude in mass (ng to fg), but they all employ nano-scale photonic crystals to trap light and resonantly enhance the light-matter interaction. In the first experiment we take advantage of the sub-femtometer displacement resolution of our photonic crystals to demonstrate a sensitive chip-scale optical accelerometer with a kHz-frequency mechanical resonator. This sensor has a noise density of approximately 10 micro-g/rt-Hz over a useable bandwidth of approximately 20 kHz and we demonstrate at least 50 dB of linear dynamic sensor range. We also discuss methods to further improve performance of this device by a factor of 10.

In the second experiment, we used a closed-loop measurement and feedback system to damp and cool a room-temperature MHz-frequency mechanical oscillator from a phonon occupation of 6.5 million down to just 66. At the time of the experiment, this represented a world-record result for the laser cooling of a macroscopic mechanical element without the aid of cryogenic pre-cooling. Furthermore, this closed-loop damping yields a high-resolution force sensor with a practical bandwidth of 200 kHZ and the method has applications to other optomechanical sensors.

The final experiment contains results from a GHz-frequency mechanical resonator in a regime where the nonlinearity of the radiation-pressure interaction dominates the system dynamics. In this device we show self-oscillations of the mechanical element that are driven by multi-photon-phonon scattering. Control of the system allows us to initialize the mechanical oscillator into a stable high-amplitude attractor which would otherwise be inaccessible. To provide context, we begin this work by first presenting an intuitive overview of optomechanical systems and then providing an extended discussion of the principles underlying the design and fabrication of our optomechanical devices.

DNA-Mediated Charge Transport Devices for Protein Detection

Detection of biologically relevant targets, including small molecules, proteins, DNA, and RNA, is vital for fundamental research as well as clinical diagnostics. Sensors with biological elements provide a natural foundation for such devices because of the inherent recognition capabilities of biomolecules. Electrochemical DNA platforms are simple, sensitive, and do not require complex target labeling or expensive instrumentation. Sensitivity and specificity are added to DNA electrochemical platforms when the physical properties of DNA are harnessed. The inherent structure of DNA, with its stacked core of aromatic bases, enables DNA to act as a wire via DNA-mediated charge transport (DNA CT). DNA CT is not only robust over long molecular distances of at least 34 nm, but is also especially sensitive to anything that perturbs proper base stacking, including DNA mismatches, lesions, or DNA-binding proteins that distort the π-stack. Electrochemical sensors based on DNA CT have previously been used for single-nucleotide polymorphism detection, hybridization assays, and DNA-binding protein detection. Here, improvements to (i) the structure of DNA monolayers and (ii) the signal amplification with DNA CT platforms for improved sensitivity and detection are described.

First, improvements to the control over DNA monolayer formation are reported through the incorporation of copper-free click chemistry into DNA monolayer assembly. As opposed to conventional film formation involving the self-assembly of thiolated DNA, copper-free click chemistry enables DNA to be tethered to a pre-formed mixed alkylthiol monolayer. The total amount of DNA in the final film is directly related to the amount of azide in the underlying alkylthiol monolayer. DNA monolayers formed with this technique are significantly more homogeneous and lower density, with a larger amount of individual helices exposed to the analyte solution. With these improved monolayers, significantly more sensitive detection of the transcription factor TATA binding protein (TBP) is achieved.

Using low-density DNA monolayers, two-electrode DNA arrays were designed and fabricated to enable the placement of multiple DNA sequences onto a single underlying electrode. To pattern DNA onto the primary electrode surface of these arrays, a copper precatalyst for click chemistry was electrochemically activated at the secondary electrode. The location of the secondary electrode relative to the primary electrode enabled the patterning of up to four sequences of DNA onto a single electrode surface. As opposed to conventional electrochemical readout from the primary, DNA-modified electrode, a secondary microelectrode, coupled with electrocatalytic signal amplification, enables more sensitive detection with spatial resolution on the DNA array electrode surface. Using this two-electrode platform, arrays have been formed that facilitate differentiation between well-matched and mismatched sequences, detection of transcription factors, and sequence-selective DNA hybridization, all with the incorporation of internal controls.

For effective clinical detection, the two working electrode platform was multiplexed to contain two complementary arrays, each with fifteen electrodes. This platform, coupled with low density DNA monolayers and electrocatalysis with readout from a secondary electrode, enabled even more sensitive detection from especially small volumes (4 μL per well). This multiplexed platform has enabled the simultaneous detection of two transcription factors, TBP and CopG, with surface dissociation constants comparable to their solution dissociation constants.

With the sensitivity and selectivity obtained from the multiplexed, two working electrode array, an electrochemical signal-on assay for activity of the human methyltransferase DNMT1 was incorporated. DNMT1 is the most abundant human methyltransferase, and its aberrant methylation has been linked to the development of cancer. However, current methods to monitor methyltransferase activity are either ineffective with crude samples or are impractical to develop for clinical applications due to a reliance on radioactivity. Electrochemical detection of methyltransferase activity, in contrast, circumvents these issues. The signal-on detection assay translates methylation events into electrochemical signals via a methylation-specific restriction enzyme. Using the two working electrode platform combined with this assay, DNMT1 activity from tumor and healthy adjacent tissue lysate were evaluated. Our electrochemical measurements revealed significant differences in methyltransferase activity between tumor tissue and healthy adjacent tissue.

As differential activity was observed between colorectal tumor tissue and healthy adjacent tissue, ten tumor sets were subsequently analyzed for DNMT1 activity both electrochemically and by tritium incorporation. These results were compared to expression levels of DNMT1, measured by qPCR, and total DNMT1 protein content, measured by Western blot. The only trend detected was that hyperactivity was observed in the tumor samples as compared to the healthy adjacent tissue when measured electrochemically. These advances in DNA CT-based platforms have propelled this class of sensors from the purely academic realm into the realm of clinically relevant detection.