Detecting actions of fruit flies

In this thesis we describe a system that tracks fruit flies in video and automatically detects and classifies their actions. We introduce Caltech Fly-vs-Fly Interactions, a new dataset that contains hours of video showing pairs of fruit flies engaging in social interactions, and is published with complete expert annotations and articulated pose trajectory features. We compare experimentally the value of a frame-level feature representation with the more elaborate notion of bout features that capture the structure within actions. Similarly, we compare a simple sliding window classifier architecture with a more sophisticated structured output architecture, and find that window based detectors outperform the much slower structured counterparts, and approach human performance. In addition we test the top performing detector on the CRIM13 mouse dataset, finding that it matches the performance of the best published method.

Flight dynamics in Drosophila through a dynamically-scaled robotic approach

Flies are particularly adept at balancing the competing demands of delay tolerance, performance, and robustness during flight, which invites thoughtful examination of their multimodal feedback architecture. This dissertation examines stabilization requirements for inner-loop feedback strategies in the flapping flight of Drosophila, the fruit fly, against the backdrop of sensorimotor transformations present in the animal. Flies have evolved multiple specializations to reduce sensorimotor latency, but sensory delay during flight is still significant on the timescale of body dynamics. I explored the effect of sensor delay on flight stability and performance for yaw turns using a dynamically-scaled robot equipped with a real-time feedback system that performed active turns in response to measured yaw torque. The results show a fundamental tradeoff between sensor delay and permissible feedback gain, and suggest that fast mechanosensory feedback provides a source of active damping that compliments that contributed by passive effects. Presented in the context of these findings, a control architecture whereby a haltere-mediated inner-loop proportional controller provides damping for slower visually-mediated feedback is consistent with tethered-flight measurements, free-flight observations, and engineering design principles. Additionally, I investigated how flies adjust stroke features to regulate and stabilize level forward flight. The results suggest that few changes to hovering kinematics are actually required to meet steady-state lift and thrust requirements at different flight speeds, and the primary driver of equilibrium velocity is the aerodynamic pitch moment. This finding is consistent with prior hypotheses and observations regarding the relationship between body pitch and flight speed in fruit flies. The results also show that the dynamics may be stabilized with additional pitch damping, but the magnitude of required damping increases with flight speed. I posit that differences in stroke deviation between the upstroke and downstroke might play a critical role in this stabilization. Fast mechanosensory feedback of the pitch rate could enable active damping, which would inherently exhibit gain scheduling with flight speed if pitch torque is regulated by adjusting stroke deviation. Such a control scheme would provide an elegant solution for flight stabilization across a wide range of flight speeds.

Octopamine neurons mediate flight-induced modulation of visual processing in Drosophila melanogaster

Activity-dependent modulation of sensory systems has been documented in many organisms, and is likely to be essential for appropriate processing of information during different behavioral states. However, the mechanisms underlying these phenomena, and often their functional consequences, remain poorly characterized. I investigated the role of octopamine neurons in the flight-dependent modulation observed in visual interneurons in the fruit fly Drosophila melanogaster. The vertical system (VS) cells exhibit a boost in their response to visual motion during flight compared to quiescence. Pharmacological application of octopamine evokes responses in quiescent flies that mimic those observed during flight, and octopamine neurons that project to the optic lobes increase in activity during flight. Using genetic tools to manipulate the activity of octopamine neurons, I find that they are both necessary and sufficient for the flight-induced visual boost. This work provides the first evidence that endogenous release of octopamine is involved in state-dependent modulation of visual interneurons in flies. Further, I investigated the role of a single pair of octopamine neurons that project to the optic lobes, and found no evidence that chemical synaptic transmission via these neurons is necessary for the flight boost. However, I found some evidence that activation of these neurons may contribute to the flight boost. Wind stimuli alone are sufficient to generate transient increases in the VS cell response to motion vision, but result in no increase in baseline membrane potential. These results suggest that the flight boost originates not from a central command signal during flight, but from mechanosensory stimuli relayed via the octopamine system. Lastly, in an attempt to understand the functional consequences of the flight boost observed in visual interneurons, we measured the effect of inactivating octopamine neurons in freely flying flies. We found that flies whose octopamine neurons we silenced accelerate less than wild-type flies, consistent with the hypothesis that the flight boost we observe in VS cells is indicative of a gain control mechanism mediated by octopamine neurons. Together, this work serves as the basis for a mechanistic and functional understanding of octopaminergic modulation of vision in flying flies.

Complex behavior and perception in Drosophila emerges from iterative feedback-regulated reflexes

For a hungry fruit fly, locating and landing on a fermenting fruit where it can feed, find mates, and lay eggs, is an essential and difficult task requiring the integration of both olfactory and visual cues. Understanding how flies accomplish this will help provide a comprehensive ethological context for the expanding knowledge of their neural circuits involved in processing olfaction and vision, as well as inspire novel engineering solutions for control and estimation in computationally limited robotic applications. In this thesis, I use novel high throughput methods to develop a detailed overview of how flies track odor plumes, land, and regulate flight speed. Finally, I provide an example of how these insights can be applied to robotic applications to simplify complicated estimation problems. To localize an odor source, flies exhibit three iterative, reflex-driven behaviors. Upon encountering an attractive plume, flies increase their flight speed and turn upwind using visual cues. After losing the plume, flies begin zigzagging crosswind, again using visual cues to control their heading. After sensing an attractive odor, flies become more attracted to small visual features, which increases their chances of finding the plume source. Their changes in heading are largely controlled by open-loop maneuvers called saccades, which they direct towards and away from visual features. If a fly decides to land on an object, it begins to decelerate so as to maintain a stereotypical ratio of expansion to retinal size. Once they reach a stereotypical distance from the target, flies extend their legs in preparation for touchdown. Although it is unclear what cues they use to trigger this behavior, previous studies have indicated that it is likely under visual control. In Chapter 3, I use a nonlinear control theoretic analysis and robotic testbed to propose a novel and putative mechanism for how a fly might visually estimate distance by actively decelerating according to a visual control law. Throughout these behaviors, a common theme is the visual control of flight speed. Using genetic tools I show that the neuromodulator octopamine plays an important role in regulating flight speed, and propose a neural circuit for how this controller might be implemented in the flies brain. Two general biological and engineering principles are evident across my experiments: (1) complex behaviors, such as foraging, can emerge from the interactions of simple independent sensory-motor modules; (2) flies control their behavior in such a way that simplifies complex estimation problems.

Neuronal mechanism of state control in Drosophila melanogaster

The changes in internal states, such as fear, hunger and sleep affect behavioral responses in animals. In most of the cases, these state-dependent influences are “pleiotropic”: one state affects multiple sensory modalities and behaviors; “scalable”: the strengths and choices of such modulations differ depending on the imminence of demands; and “persistent”: once the state is switched on the effects last even after the internal demands are off. These prominent features of state-control enable animals to adjust their behavioral responses depending on their internal demands. Here, we studied the neuronal mechanisms of state-controls by investigating energy-deprived state (hunger state) and social-deprived state of fruit flies, Drosophila melanogaster, as prototypic models. To approach these questions, we developed two novel methods: a genetically based method to map sites of neuromodulation in the brain and optogenetic tools in Drosophila.

These methods, and genetic perturbations, reveal that the effect of hunger to alter behavioral sensitivity to gustatory cues is mediate by two distinct neuromodulatory pathways. The neuropeptide F (NPF) – dopamine (DA) pathway increases sugar sensitivity under mild starvation, while the adipokinetic hormone (AKH)- short neuropeptide F (sNPF) pathway decreases bitter sensitivity under severe starvation. These two pathways are recruited under different levels of energy demands without any cross interaction. Effects of both of the pathways are mediated by modulation of the gustatory sensory neurons, which reinforce the concept that sensory neurons constitute an important locus for state-dependent control of behaviors. Our data suggests that multiple independent neuromodulatory pathways are underlying pleiotropic and scalable effects of the hunger state.

In addition, using optogenetic tool, we show that the neural control of male courtship song can be separated into probabilistic/biasing, and deterministic/command-like components. The former, but not the latter, neurons are subject to functional modulation by social experience, supporting the idea that they constitute a locus of state-dependent influence. Interestingly, moreover, brief activation of the former, but not the latter, neurons trigger persistent behavioral response for more than 10 min. Altogether, these findings and new tools described in this dissertation offer new entry points for future researchers to understand the neuronal mechanism of state control.

Studies on hymenopteran parasitism of Drosophila

Studies on Hymenopteran Parasitism of Drosophila

Flies of the genus Drosophila are subject to attack by a number of parasitic forms. Sturtevant (1921) has listed records of parasitism by protozoa (Leptomonas), fungi (Muiaria and Stigmatomyces), nematodes, mites and v~rious hymenoptera. According to Sturtevant, Perkins (1913) has bred at least five species of hymenoptera, belonging to the proctotrupoid, cynipoid and chalcidoid groups, upon Drosophiline flies. H.S. Smith has bred an unidentified proctotrupoid and a chalcidoid, Pachy crepoideus dubius Ashmead* from both Drosophila melanogaster ani D. hydei. Kieffer ( 1913) has described three species of hymenoptera from Africa collected by Silvestri and stated by him to be parasitic on Drosophila, species not given. They are Trichopria (Planopria) rhopalica (Diapriidae), Ashmeadopria drosophilae (Diapriidae), and the insect which forms the subject matter of the present investigation, Eucoila drosophilae (Figitidae).

There are in addition a number of predacious enemies among wasps, spiders, flies and beetles.

The present account is concerned with parasitism of various species of Drosophila by Eucoila drosophilae Kieff. The wasps were found b y Dr. w. P. Spencer who exposed traps in an effort to collect Drosophila at Long Lake, Ohio, in Sept. 1934 . Drosophila larvae from the trap gave a large number of pupae from which wasps emerged in considerable proportions. Since that time stock s have been maintained in culture on Drosophila melanogaster.

How resources control aggression in Drosophila

How animals use sensory information to weigh the risks vs. benefits of behavioral decisions remains poorly understood. Inter-male aggression is triggered when animals perceive both the presence of an appetitive resource, such as food or females, and of competing conspecific males. How such signals are detected and integrated to control the decision to fight is not clear. Here we use the vinegar fly, Drosophila melanogaster, to investigate the manner in which food and females promotes aggression.

In the first chapter, we explore how food controls aggression. As in many other species, food promotes aggression in flies, but it is not clear whether food increases aggression per se, or whether aggression is a secondary consequence of increased social interactions caused by aggregation of flies on food. Furthermore, nothing is known about how animals evaluate the quality and quantity of food in the context of competition. We show that food promotes aggression independently of any effect to increase the frequency of contact between males. Food increases aggression but not courtship between males, suggesting that the effect of food on aggression is specific. Next, we show that flies tune the level of aggression according to absolute amount of food rather than other parameters, such as area or concentration of food. Sucrose, a sugar molecule present in many fruits, is sufficient to promote aggression, and detection of sugar via gustatory receptor neurons is necessary for food-promoted aggression. Furthermore, we show that while food is necessary for aggression, too much food decreases aggression. Finally, we show that flies exhibit strategies consistent with a territorial strategy. These data suggest that flies use sweet-sensing gustatory information to guide their decision to fight over a limited quantity of a food resource.

Following up on the findings of the first chapter, we asked how the presence of a conspecific female resource promotes male-male aggression. In the absence of food, group-housed male flies, who normally do not fight even in the presence of food, fight in the presence of females. Unlike food, the presence of females strongly influences proximity between flies. Nevertheless, as group-housed flies do not fight even when they are in small chambers, it is unlikely that the presence of female indirectly increases aggression by first increasing proximity. Unlike food, the presence of females also leads to large increases in locomotion and in male-female courtship behaviors, suggesting that females may influence aggression as well as general arousal. Female cuticular hydrocarbons are required for this effect, as females that do not produce CH pheromones are unable to promote male-male aggression. In particular, 7,11-HD––a female-specific cuticular hydrocarbon pheromone critical for male-female courtship––is sufficient to mediate this effect when it is perfumed onto pheromone-deficient females or males. Recent studies showed that ppk23⁺ GRNs label two population of GRNs, one of which detects male cuticular hydrocarbons and another labeled by ppk23 and ppk25, which detects female cuticular hydrocarbons. I show that in particular, both of these GRNs control aggression, presumably via detection of female or male pheromones. To further investigate the ways in which these two classes of GRNs control aggression, I developed new genetic tools to independently test the male- and female-sensing GRNs. I show that ppk25-LexA and ppk25-GAL80 faithfully recapitulate the expression pattern of ppk25-GAL4 and label a subset of ppk23⁺ GRNs. These tools can be used in future studies to dissect the respective functions of male-sensing and female-sensing GRNs in male social behaviors.

Finally, in the last chapter, I discuss quantitative approaches to describe how varying quantities of food and females could control the level of aggression. Flies show an inverse-U shaped aggressive response to varying quantities of food and a flat aggressive response to varying quantities of females. I show how two simple game theoretic models, “prisoner’s dilemma” and “coordination game” could be used to describe the level of aggression we observe. These results suggest that flies may use strategic decision-making, using simple comparisons of costs and benefits.

In conclusion, male-male aggression in Drosophila is controlled by simple gustatory cues from food and females, which are detected by gustatory receptor neurons. Different quantities of resource cues lead to different levels of aggression, and flies show putative territorial behavior, suggesting that fly aggression is a highly strategic adaptive behavior. How these resource cues are integrated with male pheromone cues and give rise to this complex behavior is an interesting subject, which should keep researchers busy in the coming years.

Circadian clock mutants of Drosophila melanogaster

Three mutants of Drosophila melanogaster have been isolated in which the free-running period of the circadian eclosion rhythm and the adult locomotor activity rhythm is affected. One mutant is arrhythmic, another has a short period of 19 hours, and the third has a long period of 28 hours. The mutants retain their phenotypes over the temperature range 18° to 25° C. All three mutants map near the tip of the X chromosome (distal to the centromere). By deficiency mapping, the short-period mutation has been localized to the 3B1-2 region. Complementation tests show that all three mutations affect the same functional gene.

Analysis of activity rhythms of individual mosaic flies indicates that the site of action of the short-period mutation is probably located in the head of the fly. A few activity patterns of split-head and mixed-head mosaics appear to possess both mutant and heterozygous components, suggesting that the fly head may contain two complete clocks capable of maintaining their periodicities independently.

The short-period mutation affects both the duration of the light-insensitive part of the oscillation and the degree to which the clock can be reset during the light-sensitive part of the oscillation.

Both the short-period and long-period mutant eclosion rhythms can be entrained to a period of 24 hours by a 12:12 light-dark cycle having a light intensity at least two orders of magnitude greater than that required to entrain the normal rhythm. The arrhythmic mutant does not entrain under these conditions. In the presence of a temperature cycle, however, the arrhythmic mutant does entrain, but its rhythm damps out when the temperature cycle is removed.

Evidence is presented that Pittendrigh's two-oscillator model for the clock in D. pseudoobscura applies to D. melanogaster as well. The three clock mutations primarily affect the light- sensitive driving oscillator. The arrhythmic mutation appears to have eliminated the driving oscillator while leaving the temperature-sensitive driven oscillator relatively intact.