Coding Strategies and Implementations of Compressive Sensing

This dissertation studies the coding strategies of computational imaging to overcome the limitation of conventional sensing techniques. The information capacity of conventional sensing is limited by the physical properties of optics, such as aperture size, detector pixels, quantum efficiency, and sampling rate. These parameters determine the spatial, depth, spectral, temporal, and polarization sensitivity of each imager. To increase sensitivity in any dimension can significantly compromise the others.

This research implements various coding strategies subject to optical multidimensional imaging and acoustic sensing in order to extend their sensing abilities. The proposed coding strategies combine hardware modification and signal processing to exploiting bandwidth and sensitivity from conventional sensors. We discuss the hardware architecture, compression strategies, sensing process modeling, and reconstruction algorithm of each sensing system.

Optical multidimensional imaging measures three or more dimensional information of the optical signal. Traditional multidimensional imagers acquire extra dimensional information at the cost of degrading temporal or spatial resolution. Compressive multidimensional imaging multiplexes the transverse spatial, spectral, temporal, and polarization information on a two-dimensional (2D) detector. The corresponding spectral, temporal and polarization coding strategies adapt optics, electronic devices, and designed modulation techniques for multiplex measurement. This computational imaging technique provides multispectral, temporal super-resolution, and polarization imaging abilities with minimal loss in spatial resolution and noise level while maintaining or gaining higher temporal resolution. The experimental results prove that the appropriate coding strategies may improve hundreds times more sensing capacity.

Human auditory system has the astonishing ability in localizing, tracking, and filtering the selected sound sources or information from a noisy environment. Using engineering efforts to accomplish the same task usually requires multiple detectors, advanced computational algorithms, or artificial intelligence systems. Compressive acoustic sensing incorporates acoustic metamaterials in compressive sensing theory to emulate the abilities of sound localization and selective attention. This research investigates and optimizes the sensing capacity and the spatial sensitivity of the acoustic sensor. The well-modeled acoustic sensor allows localizing multiple speakers in both stationary and dynamic auditory scene; and distinguishing mixed conversations from independent sources with high audio recognition rate.

Stimulus Integration and Parsing in the Primate Auditory Midbrain

Integrating information from multiple sources is a crucial function of the brain. Examples of such integration include multiple stimuli of different modalties, such as visual and auditory, multiple stimuli of the same modality, such as auditory and auditory, and integrating stimuli from the sensory organs (i.e. ears) with stimuli delivered from brain-machine interfaces.

The overall aim of this body of work is to empirically examine stimulus integration in these three domains to inform our broader understanding of how and when the brain combines information from multiple sources.

First, I examine visually-guided auditory, a problem with implications for the general problem in learning of how the brain determines what lesson to learn (and what lessons not to learn). For example, sound localization is a behavior that is partially learned with the aid of vision. This process requires correctly matching a visual location to that of a sound. This is an intrinsically circular problem when sound location is itself uncertain and the visual scene is rife with possible visual matches. Here, we develop a simple paradigm using visual guidance of sound localization to gain insight into how the brain confronts this type of circularity. We tested two competing hypotheses. 1: The brain guides sound location learning based on the synchrony or simultaneity of auditory-visual stimuli, potentially involving a Hebbian associative mechanism. 2: The brain uses a ‘guess and check’ heuristic in which visual feedback that is obtained after an eye movement to a sound alters future performance, perhaps by recruiting the brain’s reward-related circuitry. We assessed the effects of exposure to visual stimuli spatially mismatched from sounds on performance of an interleaved auditory-only saccade task. We found that when humans and monkeys were provided the visual stimulus asynchronously with the sound but as feedback to an auditory-guided saccade, they shifted their subsequent auditory-only performance toward the direction of the visual cue by 1.3-1.7 degrees, or 22-28% of the original 6 degree visual-auditory mismatch. In contrast when the visual stimulus was presented synchronously with the sound but extinguished too quickly to provide this feedback, there was little change in subsequent auditory-only performance. Our results suggest that the outcome of our own actions is vital to localizing sounds correctly. Contrary to previous expectations, visual calibration of auditory space does not appear to require visual-auditory associations based on synchrony/simultaneity.

My next line of research examines how electrical stimulation of the inferior colliculus influences perception of sounds in a nonhuman primate. The central nucleus of the inferior colliculus is the major ascending relay of auditory information before it reaches the forebrain, and thus an ideal target for understanding low-level information processing prior to the forebrain, as almost all auditory signals pass through the central nucleus of the inferior colliculus before reaching the forebrain. Thus, the inferior colliculus is the ideal structure to examine to understand the format of the inputs into the forebrain and, by extension, the processing of auditory scenes that occurs in the brainstem. Therefore, the inferior colliculus was an attractive target for understanding stimulus integration in the ascending auditory pathway.

Moreover, understanding the relationship between the auditory selectivity of neurons and their contribution to perception is critical to the design of effective auditory brain prosthetics. These prosthetics seek to mimic natural activity patterns to achieve desired perceptual outcomes. We measured the contribution of inferior colliculus (IC) sites to perception using combined recording and electrical stimulation. Monkeys performed a frequency-based discrimination task, reporting whether a probe sound was higher or lower in frequency than a reference sound. Stimulation pulses were paired with the probe sound on 50% of trials (0.5-80 µA, 100-300 Hz, n=172 IC locations in 3 rhesus monkeys). Electrical stimulation tended to bias the animals’ judgments in a fashion that was coarsely but significantly correlated with the best frequency of the stimulation site in comparison to the reference frequency employed in the task. Although there was considerable variability in the effects of stimulation (including impairments in performance and shifts in performance away from the direction predicted based on the site’s response properties), the results indicate that stimulation of the IC can evoke percepts correlated with the frequency tuning properties of the IC. Consistent with the implications of recent human studies, the main avenue for improvement for the auditory midbrain implant suggested by our findings is to increase the number and spatial extent of electrodes, to increase the size of the region that can be electrically activated and provide a greater range of evoked percepts.

My next line of research employs a frequency-tagging approach to examine the extent to which multiple sound sources are combined (or segregated) in the nonhuman primate inferior colliculus. In the single-sound case, most inferior colliculus neurons respond and entrain to sounds in a very broad region of space, and many are entirely spatially insensitive, so it is unknown how the neurons will respond to a situation with more than one sound. I use multiple AM stimuli of different frequencies, which the inferior colliculus represents using a spike timing code. This allows me to measure spike timing in the inferior colliculus to determine which sound source is responsible for neural activity in an auditory scene containing multiple sounds. Using this approach, I find that the same neurons that are tuned to broad regions of space in the single sound condition become dramatically more selective in the dual sound condition, preferentially entraining spikes to stimuli from a smaller region of space. I will examine the possibility that there may be a conceptual linkage between this finding and the finding of receptive field shifts in the visual system.

In chapter 5, I will comment on these findings more generally, compare them to existing theoretical models, and discuss what these results tell us about processing in the central nervous system in a multi-stimulus situation. My results suggest that the brain is flexible in its processing and can adapt its integration schema to fit the available cues and the demands of the task.

The Neural Basis of Involuntary Episodic Memories

Involuntary episodic memories are memories that come into consciousness without preceding retrieval effort. These memories are commonplace and are relevant to multiple mental disorders. However, they are vastly understudied. We use a novel paradigm to elicit involuntary memories in the laboratory so that we can study their neural basis. In session one, an encoding session, sounds are presented with picture pairs or alone. In session two, in the scanner, sounds-picture pairs and unpaired sounds are reencoded. Immediately following, participants are split into two groups: a voluntary and an involuntary group. Both groups perform a sound localization task in which they hear the sounds and indicate the side from which they are coming. The voluntary group additionally tries to remember the pictures that were paired with the sounds. Looking at neural activity, we find a main effect of condition (paired vs. unpaired sounds) showing similar activity in both groups for voluntary and involuntary memories in regions typically associated with retrieval. There is also a main effect of group (voluntary vs. involuntary) in the dorsolateral prefrontal cortex, a region typically associated with cognitive control. Turning to connectivity similarities and differences between groups again, there is a main effect of condition showing paired > unpaired sounds are associated with a recollection network. In addition, three group differences were found: (1) increased connectivity between the pulvinar nucleus of the thalamus and the recollection network for the voluntary group, (2) a higher association between the voluntary group and a network that includes regions typically found in frontoparietal and cingulo-opercular networks, and (3) shorter path length for about half of the nodes in these networks for the voluntary group. Finally, we use the same paradigm to compare involuntary memories in people with posttraumatic stress disorder (PTSD) to trauma-controls. This study also included the addition of emotional pictures. There were two main findings. (1) A similar pattern of activity was found for paired > unpaired sounds for both groups but this activity was delayed in the PTSD group. (2) A similar pattern of activity was found for high > low emotion stimuli but it occurred early in the PTSD group compared to the control group. Our results suggest that involuntary and voluntary memories share the same neural representation but that voluntary memories are associated with additional cognitive control processes. They also suggest that disorders associated with cognitive deficits, like PTSD, can affect the processing of involuntary memories.

One sound or two? Object-related negativity indexes echo perception.

The ability to isolate a single sound source among concurrent sources and reverberant energy is necessary for understanding the auditory world. The precedence effect describes a related experimental finding, that when presented with identical sounds from two locations with a short onset asynchrony (on the order of milliseconds), listeners report a single source with a location dominated by the lead sound. Single-cell recordings in multiple animal models have indicated that there are low-level mechanisms that may contribute to the precedence effect, yet psychophysical studies in humans have provided evidence that top-down cognitive processes have a great deal of influence on the perception of simulated echoes. In the present study, event-related potentials evoked by click pairs at and around listeners' echo thresholds indicate that perception of the lead and lag sound as individual sources elicits a negativity between 100 and 250 msec, previously termed the object-related negativity (ORN). Even for physically identical stimuli, the ORN is evident when listeners report hearing, as compared with not hearing, a second sound source. These results define a neural mechanism related to the conscious perception of multiple auditory objects.

Bacteria localization and chorion thinning among preterm premature rupture of membranes.

OBJECTIVE: Bacterial colonization of the fetal membranes and its role in pathogenesis of membrane rupture is poorly understood. Prior retrospective work revealed chorion layer thinning in preterm premature rupture of membranes (PPROM) subjects. Our objective was to prospectively examine fetal membrane chorion thinning and to correlate to bacterial presence in PPROM, preterm, and term subjects. STUDY DESIGN: Paired membrane samples (membrane rupture and membrane distant) were prospectively collected from: PPROM = 14, preterm labor (PTL = 8), preterm no labor (PTNL = 8), term labor (TL = 10), and term no labor (TNL = 8), subjects. Sections were probed with cytokeratin to identify fetal trophoblast layer of the chorion using immunohistochemistry. Fluorescence in situ hybridization was performed using broad range 16 s ribosomal RNA probe. Images were evaluated, chorion and choriodecidua were measured, and bacterial fluorescence scored. Chorion thinning and bacterial presence were compared among and between groups using Student's t-test, linear mixed effect model, and Poisson regression model (SAS Cary, NC). RESULTS: In all groups, the fetal chorion cellular layer was thinner at rupture compared to distant site (147.2 vs. 253.7 µm, p<0.0001). Further, chorion thinning was greatest among PPROM subjects compared to all other groups combined, regardless of site sampled [PPROM(114.9) vs. PTL(246.0) vs. PTNL(200.8) vs. TL(217.9) vs. TNL(246.5)]. Bacteria counts were highest among PPROM subjects compared to all other groups regardless of site sampled or histologic infection [PPROM(31) vs. PTL(9) vs. PTNL(7) vs. TL(7) vs. TNL(6)]. Among all subjects at both sites, bacterial counts were inversely correlated with chorion thinning, even excluding histologic chorioamnionitis (p<0.0001 and p = 0.05). CONCLUSIONS: Fetal chorion was uniformly thinner at rupture site compared to distant sites. In PPROM fetal chorion, we demonstrated pronounced global thinning. Although cause or consequence is uncertain, bacterial presence is greatest and inversely correlated with chorion thinning among PPROM subjects.

Control of cyclin B1 localization through regulated binding of the nuclear export factor CRM1.

Activation of the Cyclin B/Cdc2 kinase complex triggers entry into mitosis in all eukaryotic cells. Cyclin B1 localization changes dramatically during the cell cycle, precipitously transiting from the cytoplasm to the nucleus at the beginning of mitosis. Presumably, this relocalization promotes the phosphorylation of nuclear targets critical for chromatin condensation and nuclear envelope breakdown. We show here that the previously characterized cytoplasmic retention sequence of Cyclin B1, responsible for its interphase cytoplasmic localization, is actually an autonomous nuclear export sequence, capable of directing nuclear export of a heterologous protein, and able to bind specifically to the recently identified export mediator, CRM1. We propose that the observed cytoplasmic localization of Cyclin B1 during interphase reflects the equilibrium between ongoing nuclear import and rapid CRM1-mediated export. In support of this hypothesis, we found that treatment of cells with leptomycin B, which disrupted Cyclin B1-CRM1 interactions, led to a marked nuclear accumulation of Cyclin B1. In mitosis, Cyclin B1 undergoes phosphorylation at several sites, a subset of which have been proposed to play a role in Cyclin B1 accumulation in the nucleus. Both CRM1 binding and the ability to direct nuclear export were affected by mutation of these phosphorylation sites; thus, we propose that Cyclin B1 phosphorylation at the G2/M transition prevents its interaction with CRM1, thereby reducing nuclear export and facilitating nuclear accumulation.

Sound the alarm: fraud in neuroscience.

We expect scientists to follow a code of honor and conduct and to report their research honestly and accurately, but so-called scientific misconduct, which includes plagiarism, faked data, and altered images, has led to a tenfold increase in the number of retractions over the past decade. Among the reasons for this troubling upsurge is increased competition for journal placement, grant money, and prestigious appointments. The solutions are not easy, but reform and greater vigilance is needed.

Localization of Metal Electrodes in the Intact Rat Brain Using Registration of 3D Microcomputed Tomography Images to a Magnetic Resonance Histology Atlas.

Simultaneous neural recordings taken from multiple areas of the rodent brain are garnering growing interest due to the insight they can provide about spatially distributed neural circuitry. The promise of such recordings has inspired great progress in methods for surgically implanting large numbers of metal electrodes into intact rodent brains. However, methods for localizing the precise location of these electrodes have remained severely lacking. Traditional histological techniques that require slicing and staining of physical brain tissue are cumbersome, and become increasingly impractical as the number of implanted electrodes increases. Here we solve these problems by describing a method that registers 3-D computerized tomography (CT) images of intact rat brains implanted with metal electrode bundles to a Magnetic Resonance Imaging Histology (MRH) Atlas. Our method allows accurate visualization of each electrode bundle's trajectory and location without removing the electrodes from the brain or surgically implanting external markers. In addition, unlike physical brain slices, once the 3D images of the electrode bundles and the MRH atlas are registered, it is possible to verify electrode placements from many angles by "re-slicing" the images along different planes of view. Further, our method can be fully automated and easily scaled to applications with large numbers of specimens. Our digital imaging approach to efficiently localizing metal electrodes offers a substantial addition to currently available methods, which, in turn, may help accelerate the rate at which insights are gleaned from rodent network neuroscience.

Bottlenose dolphins exchange signature whistles when meeting at sea.

The bottlenose dolphin, Tursiops truncatus, is one of very few animals that, through vocal learning, can invent novel acoustic signals and copy whistles of conspecifics. Furthermore, receivers can extract identity information from the invented part of whistles. In captivity, dolphins use such signature whistles while separated from the rest of their group. However, little is known about how they use them at sea. If signature whistles are the main vehicle to transmit identity information, then dolphins should exchange these whistles in contexts where groups or individuals join. We used passive acoustic localization during focal boat follows to observe signature whistle use in the wild. We found that stereotypic whistle exchanges occurred primarily when groups of dolphins met and joined at sea. A sequence analysis verified that most of the whistles used during joins were signature whistles. Whistle matching or copying was not observed in any of the joins. The data show that signature whistle exchanges are a significant part of a greeting sequence that allows dolphins to identify conspecifics when encountering them in the wild.