10 resultados para Extrinsic reward
em CaltechTHESIS
Resumo:
During inflammation and infection, hematopoietic stem and progenitor cells (HSPCs) are stimulated to proliferate and differentiate into mature immune cells, especially of the myeloid lineage. MicroRNA-146a (miR-146a) is a critical negative regulator of inflammation. Deletion of the gene encoding miR-146a—expressed in all blood cell types—produces effects that appear as dysregulated inflammatory hematopoiesis, leading to a decline in the number and quality of hematopoietic stem cells (HSCs), excessive myeloproliferation, and, ultimately, to exhaustion of the HSCs and hematopoietic neoplasms. Six-week-old deleted mice are normal, with no effect on cell numbers, but by 4 months bone marrow hypercellularity can be seen, and by 8 months marrow exhaustion is becoming evident. The ability of HSCs to replenish the entire hematopoietic repertoire in a myelo-ablated mouse also declines precipitously as miR-146a-deficient mice age. In the absence of miR-146a, LPS-mediated serial inflammatory stimulation accelerates the effects of aging. This chronic inflammatory stress on HSCs in deleted mice involves a molecular axis consisting of upregulation of the signaling protein TRAF6 leading to excessive activity of the transcription factor NF-κB and overproduction of the cytokine IL-6. At the cellular level, transplant studies show that the defects are attributable to both an intrinsic problem in the miR-146a-deficient HSCs and extrinsic effects of miR-146a-deficient lymphocytes and non-hematopoietic cells. This study has identified a microRNA, miR-146a, to be a critical regulator of HSC homeostasis during chronic inflammatory challenge in mice and has provided a molecular connection between chronic inflammation and the development of bone marrow failure and myeloproliferative neoplasms. This may have implications for human hematopoietic malignancies, such as myelodysplastic syndrome, which frequently displays downregulated miR-146a expression.
Resumo:
This thesis introduces fundamental equations and numerical methods for manipulating surfaces in three dimensions via conformal transformations. Conformal transformations are valuable in applications because they naturally preserve the integrity of geometric data. To date, however, there has been no clearly stated and consistent theory of conformal transformations that can be used to develop general-purpose geometry processing algorithms: previous methods for computing conformal maps have been restricted to the flat two-dimensional plane, or other spaces of constant curvature. In contrast, our formulation can be used to produce---for the first time---general surface deformations that are perfectly conformal in the limit of refinement. It is for this reason that we commandeer the title Conformal Geometry Processing.
The main contribution of this thesis is analysis and discretization of a certain time-independent Dirac equation, which plays a central role in our theory. Given an immersed surface, we wish to construct new immersions that (i) induce a conformally equivalent metric and (ii) exhibit a prescribed change in extrinsic curvature. Curvature determines the potential in the Dirac equation; the solution of this equation determines the geometry of the new surface. We derive the precise conditions under which curvature is allowed to evolve, and develop efficient numerical algorithms for solving the Dirac equation on triangulated surfaces.
From a practical perspective, this theory has a variety of benefits: conformal maps are desirable in geometry processing because they do not exhibit shear, and therefore preserve textures as well as the quality of the mesh itself. Our discretization yields a sparse linear system that is simple to build and can be used to efficiently edit surfaces by manipulating curvature and boundary data, as demonstrated via several mesh processing applications. We also present a formulation of Willmore flow for triangulated surfaces that permits extraordinarily large time steps and apply this algorithm to surface fairing, geometric modeling, and construction of constant mean curvature (CMC) surfaces.
Resumo:
Therapy employing epidural electrostimulation holds great potential for improving therapy for patients with spinal cord injury (SCI) (Harkema et al., 2011). Further promising results from combined therapies using electrostimulation have also been recently obtained (e.g., van den Brand et al., 2012). The devices being developed to deliver the stimulation are highly flexible, capable of delivering any individual stimulus among a combinatorially large set of stimuli (Gad et al., 2013). While this extreme flexibility is very useful for ensuring that the device can deliver an appropriate stimulus, the challenge of choosing good stimuli is quite substantial, even for expert human experimenters. To develop a fully implantable, autonomous device which can provide useful therapy, it is necessary to design an algorithmic method for choosing the stimulus parameters. Such a method can be used in a clinical setting, by caregivers who are not experts in the neurostimulator's use, and to allow the system to adapt autonomously between visits to the clinic. To create such an algorithm, this dissertation pursues the general class of active learning algorithms that includes Gaussian Process Upper Confidence Bound (GP-UCB, Srinivas et al., 2010), developing the Gaussian Process Batch Upper Confidence Bound (GP-BUCB, Desautels et al., 2012) and Gaussian Process Adaptive Upper Confidence Bound (GP-AUCB) algorithms. This dissertation develops new theoretical bounds for the performance of these and similar algorithms, empirically assesses these algorithms against a number of competitors in simulation, and applies a variant of the GP-BUCB algorithm in closed-loop to control SCI therapy via epidural electrostimulation in four live rats. The algorithm was tasked with maximizing the amplitude of evoked potentials in the rats' left tibialis anterior muscle. These experiments show that the algorithm is capable of directing these experiments sensibly, finding effective stimuli in all four animals. Further, in direct competition with an expert human experimenter, the algorithm produced superior performance in terms of average reward and comparable or superior performance in terms of maximum reward. These results indicate that variants of GP-BUCB may be suitable for autonomously directing SCI therapy.
Resumo:
Humans are particularly adept at modifying their behavior in accordance with changing environmental demands. Through various mechanisms of cognitive control, individuals are able to tailor actions to fit complex short- and long-term goals. The research described in this thesis uses functional magnetic resonance imaging to characterize the neural correlates of cognitive control at two levels of complexity: response inhibition and self-control in intertemporal choice. First, we examined changes in neural response associated with increased experience and skill in response inhibition; successful response inhibition was associated with decreased neural response over time in the right ventrolateral prefrontal cortex, a region widely implicated in cognitive control, providing evidence for increased neural efficiency with learned automaticity. We also examined a more abstract form of cognitive control using intertemporal choice. In two experiments, we identified putative neural substrates for individual differences in temporal discounting, or the tendency to prefer immediate to delayed rewards. Using dynamic causal models, we characterized the neural circuit between ventromedial prefrontal cortex, an area involved in valuation, and dorsolateral prefrontal cortex, a region implicated in self-control in intertemporal and dietary choice, and found that connectivity from dorsolateral prefrontal cortex to ventromedial prefrontal cortex increases at the time of choice, particularly when delayed rewards are chosen. Moreover, estimates of the strength of connectivity predicted out-of-sample individual rates of temporal discounting, suggesting a neurocomputational mechanism for variation in the ability to delay gratification. Next, we interrogated the hypothesis that individual differences in temporal discounting are in part explained by the ability to imagine future reward outcomes. Using a novel paradigm, we imaged neural response during the imagining of primary rewards, and identified negative correlations between activity in regions associated the processing of both real and imagined rewards (lateral orbitofrontal cortex and ventromedial prefrontal cortex, respectively) and the individual temporal discounting parameters estimated in the previous experiment. These data suggest that individuals who are better able to represent reward outcomes neurally are less susceptible to temporal discounting. Together, these findings provide further insight into role of the prefrontal cortex in implementing cognitive control, and propose neurobiological substrates for individual variation.
Resumo:
Modern robots are increasingly expected to function in uncertain and dynamically challenging environments, often in proximity with humans. In addition, wide scale adoption of robots requires on-the-fly adaptability of software for diverse application. These requirements strongly suggest the need to adopt formal representations of high level goals and safety specifications, especially as temporal logic formulas. This approach allows for the use of formal verification techniques for controller synthesis that can give guarantees for safety and performance. Robots operating in unstructured environments also face limited sensing capability. Correctly inferring a robot's progress toward high level goal can be challenging.
This thesis develops new algorithms for synthesizing discrete controllers in partially known environments under specifications represented as linear temporal logic (LTL) formulas. It is inspired by recent developments in finite abstraction techniques for hybrid systems and motion planning problems. The robot and its environment is assumed to have a finite abstraction as a Partially Observable Markov Decision Process (POMDP), which is a powerful model class capable of representing a wide variety of problems. However, synthesizing controllers that satisfy LTL goals over POMDPs is a challenging problem which has received only limited attention.
This thesis proposes tractable, approximate algorithms for the control synthesis problem using Finite State Controllers (FSCs). The use of FSCs to control finite POMDPs allows for the closed system to be analyzed as finite global Markov chain. The thesis explicitly shows how transient and steady state behavior of the global Markov chains can be related to two different criteria with respect to satisfaction of LTL formulas. First, the maximization of the probability of LTL satisfaction is related to an optimization problem over a parametrization of the FSC. Analytic computation of gradients are derived which allows the use of first order optimization techniques.
The second criterion encourages rapid and frequent visits to a restricted set of states over infinite executions. It is formulated as a constrained optimization problem with a discounted long term reward objective by the novel utilization of a fundamental equation for Markov chains - the Poisson equation. A new constrained policy iteration technique is proposed to solve the resulting dynamic program, which also provides a way to escape local maxima.
The algorithms proposed in the thesis are applied to the task planning and execution challenges faced during the DARPA Autonomous Robotic Manipulation - Software challenge.
Resumo:
By using techniques of rapid quenching from the melt, metastable phases have been obtained in ternary alloys which contain tellurium as a major component and two of the three noble metals (Cu, Ag, Au) as minor components. The metastable phases found in this investigation are either simple cubic or amorphous. The formation of the simple cubic phase is discussed. The electrical resistance and the thermoelectric power of the simple cubic alloy (Au30Te70) have been measured and interpreted in terms of atomic bondings. The semiconducting properties of a metastable amorphous alloy (Au5Cu25Te70) have been measured. The experimental results are discussed in connection with a theoretical consideration of the validity of band theory in an amorphous solid. The existence of extrinsic conduction in an amorphous semiconductor is suggested by the result of electrical resistance and thermoelectric power measurements.
Resumo:
Nicotinic acetylcholine receptors (nAChRs) are pentameric, ligand-gated, cation channels found throughout the central and peripheral nervous system, whose endogenous ligand is acetylcholine, but which can also be acted upon by nicotine. The subunit compositions of nAChR determine their physiological and pharmacological properties, with different subunits expressed in different combinations or areas throughout the brain. The behavioral and physiological effects of nicotine are elicited by its agonistic and desensitizing actions selectively on neuronal nAChRs. The midbrain is of particular interest due to its population of nAChRs expressed on dopaminergic neurons, which are important for reward and reinforcement, and possibly contribute to nicotine dependence. The α6-subunit is found on dopaminergic neurons but very few other regions of the brain, making it an interesting drug target. We assayed a novel nicotinic agonist, called TI-299423 or TC299, for its possible selectivity for α6-containing nAChRs. Our goal was to isolate the role of α6-containing nAChRs in nicotine reward and reinforcement, and provide insight into the search for more effective smoking cessation compounds. This was done using a variety of in vitro and behavioral assays, aimed dually at understanding TI-299423’s exact mechanism of action and its downstream effects. Additionally, we looked at the effects of another compound, menthol, on nicotine reward. Understanding how reward is generated in the cholinergic system and how that is modulated by other compounds contributes to a better understand of our complex neural circuitry and provides insight for the future development of therapeutics.
Resumo:
Nicotinic acetylcholine receptors are pentameric ligand-gated ion channels mediating fast synaptic transmission throughout the peripheral and central nervous systems. They have been implicated in various processes related to cognitive functions, learning and memory, arousal, reward, motor control and analgesia. Therefore, these receptors present alluring potential therapeutic targets for the treatment of pain, epilepsy, Alzheimer’s disease, Parkinson’s disease, Tourette’s syndrome, schizophrenia, anxiety, depression and nicotine addiction. The work detailed in this thesis focuses on binding studies of neuronal nicotinic receptors and aims to further our knowledge of subtype specific functional and structural information.
Chapter 1 is an introductory chapter describing the structure and function of nicotinic acetylcholine receptors as well as the methodologies used for the dissertation work described herein. There are several different subtypes of nicotinic acetylcholine receptors known to date and the subtle variations in their structure and function present a challenging area of study. The work presented in this thesis deals specifically with the α4β2 subtype of nicotinic acetylcholine receptor. This subtype assembles into 2 closely related stoichiometries, termed throughout this thesis as A3B2 and A2B3 after their respective subunit composition. Chapter 2 describes binding studies of select nicotinic agonists on A3B2 and A2B3 receptors determined by whole-cell recording. Three key binding interactions, a cation-π and two hydrogen bonds, were probed for four nicotinic agonists, acetylcholine, nicotine, smoking cessation drug varenicline (Chantix®) and the related natural product cytisine.
Results from the binding studies presented in Chapter 2 show that the major difference in binding of these four agonists to A3B2 and A2B3 receptors lies in one of the two hydrogen bond interactions where the agonist acts as the hydrogen bond acceptor and the backbone NH of a conserved leucine residue in the receptor acts as the hydrogen bond donor. Chapter 3 focuses on studying the effect of modulating the hydrogen bond acceptor ability of nicotine and epibatidine on A3B2 receptor function determined by whole-cell recording. Finally, Chapter 4 describes single-channel recording studies of varenicline binding to A2B3 and A3B2 receptors.
Resumo:
A person living in an industrialized society has almost no choice but to receive information daily with negative implications for himself or others. His attention will often be drawn to the ups and downs of economic indicators or the alleged misdeeds of leaders and organizations. Reacting to new information is central to economics, but economics typically ignores the affective aspect of the response, for example, of stress or anger. These essays present the results of considering how the affective aspect of the response can influence economic outcomes.
The first chapter presents an experiment in which individuals were presented with information about various non-profit organizations and allowed to take actions that rewarded or punished those organizations. When social interaction was introduced into this environment an asymmetry between rewarding and punishing appeared. The net effects of punishment became greater and more variable, whereas the effects of reward were unchanged. The individuals were more strongly influenced by negative social information and used that information to target unpopular organizations. These behaviors contributed to an increase in inequality among the outcomes of the organizations.
The second and third chapters present empirical studies of reactions to negative information about local economic conditions. Economic factors are among the most prevalent stressors, and stress is known to have numerous negative effects on health. These chapters document localized, transient effects of the announcement of information about large-scale job losses. News of mass layoffs and shut downs of large military bases are found to decrease birth weights and gestational ages among babies born in the affected regions. The effect magnitudes are close to those estimated in similar studies of disasters.
Resumo:
Hair cells from the bull frog's sacculus, a vestibular organ responding to substrate-borne vibration, possess electrically resonant membrane properties which maximize the sensitivity of each cell to a particular frequency of mechanical input. The electrical resonance of these cells and its underlying ionic basis were studied by applying gigohm-seal recording techniques to solitary hair cells enzymatically dissociated from the sacculus. The contribution of electrical resonance to frequency selectivity was assessed from microelectrode recordings from hair cells in an excised preparation of the sacculus.
Electrical resonance in the hair cell is demonstrated by damped membrane-potential oscillations in response to extrinsic current pulses applied through the recording pipette. This response is analyzed as that of a damped harmonic oscillator. Oscillation frequency rises with membrane depolarization, from 80-160 Hz at resting potential to asymptotic values of 200-250 Hz. The sharpness of electrical tuning, denoted by the electrical quality factor, Qe, is a bell-shaped function of membrane voltage, reaching a maximum value around eight at a membrane potential slightly positive to the resting potential.
In whole cells, three time-variant ionic currents are activated at voltages more positive than -60 to -50 mV; these are identified as a voltage-dependent, non-inactivating Ca current (Ica), a voltage-dependent, transient K current (Ia), and a Ca-dependent K current (Ic). The C channel is identified in excised, inside-out membrane patches on the basis of its large conductance (130-200 pS), its selective permeability to Kover Na or Cl, and its activation by internal Ca ions and membrane depolarization. Analysis of open- and closed-lifetime distributions suggests that the C channel can assume at least two open and three closed kinetic states.
Exposing hair cells to external solutions that inhibit the Ca or C conductances degrades the electrical resonance properties measured under current-clamp conditions, while blocking the A conductance has no significant effect, providing evidence that only the Ca and C conductances participate in the resonance mechanism. To test the sufficiency of these two conductances to account for electrical resonance, a mathematical model is developed that describes Ica, Ic, and intracellular Ca concentration during voltage-clamp steps. Ica activation is approximated by a third-order Hodgkin-Huxley kinetic scheme. Ca entering the cell is assumed to be confined to a small submembrane compartment which contains an excess of Ca buffer; Ca leaves this space with first-order kinetics. The Ca- and voltage-dependent activation of C channels is described by a five-state kinetic scheme suggested by the results of single-channel observations. Parameter values in the model are adjusted to fit the waveforms of Ica and Ic evoked by a series of voltage-clamp steps in a single cell. Having been thus constrained, the model correctly predicts the character of voltage oscillations produced by current-clamp steps, including the dependencies of oscillation frequency and Qe on membrane voltage. The model shows quantitatively how the Ca and C conductances interact, via changes in intracellular Ca concentration, to produce electrical resonance in a vertebrate hair cell.