Real-time analytics on large dynamic graphs

In today's fast-paced and interconnected digital world, the data generated by an increasing number of applications is being modeled as dynamic graphs. The graph structure encodes relationships among data items, while the structural changes to the graphs as well as the continuous stream of information produced by the entities in these graphs make them dynamic in nature. Examples include social networks where users post status updates, images, videos, etc.; phone call networks where nodes may send text messages or place phone calls; road traffic networks where the traffic behavior of the road segments changes constantly, and so on. There is a tremendous value in storing, managing, and analyzing such dynamic graphs and deriving meaningful insights in real-time. However, a majority of the work in graph analytics assumes a static setting, and there is a lack of systematic study of the various dynamic scenarios, the complexity they impose on the analysis tasks, and the challenges in building efficient systems that can support such tasks at a large scale. In this dissertation, I design a unified streaming graph data management framework, and develop prototype systems to support increasingly complex tasks on dynamic graphs. In the first part, I focus on the management and querying of distributed graph data. I develop a hybrid replication policy that monitors the read-write frequencies of the nodes to decide dynamically what data to replicate, and whether to do eager or lazy replication in order to minimize network communication and support low-latency querying. In the second part, I study parallel execution of continuous neighborhood-driven aggregates, where each node aggregates the information generated in its neighborhoods. I build my system around the notion of an aggregation overlay graph, a pre-compiled data structure that enables sharing of partial aggregates across different queries, and also allows partial pre-computation of the aggregates to minimize the query latencies and increase throughput. Finally, I extend the framework to support continuous detection and analysis of activity-based subgraphs, where subgraphs could be specified using both graph structure as well as activity conditions on the nodes. The query specification tasks in my system are expressed using a set of active structural primitives, which allows the query evaluator to use a set of novel optimization techniques, thereby achieving high throughput. Overall, in this dissertation, I define and investigate a set of novel tasks on dynamic graphs, design scalable optimization techniques, build prototype systems, and show the effectiveness of the proposed techniques through extensive evaluation using large-scale real and synthetic datasets.

FRAMEWORK SYNTHESIS FOR SYMBOLIC EXECUTION OF EVENT-DRIVEN FRAMEWORKS

Symbolic execution is a powerful program analysis technique, but it is very challenging to apply to programs built using event-driven frameworks, such as Android. The main reason is that the framework code itself is too complex to symbolically execute. The standard solution is to manually create a framework model that is simpler and more amenable to symbolic execution. However, developing and maintaining such a model by hand is difficult and error-prone. We claim that we can leverage program synthesis to introduce a high-degree of automation to the process of framework modeling. To support this thesis, we present three pieces of work. First, we introduced SymDroid, a symbolic executor for Android. While Android apps are written in Java, they are compiled to Dalvik bytecode format. Instead of analyzing an app’s Java source, which may not be available, or decompiling from Dalvik back to Java, which requires significant engineering effort and introduces yet another source of potential bugs in an analysis, SymDroid works directly on Dalvik bytecode. Second, we introduced Pasket, a new system that takes a first step toward automatically generating Java framework models to support symbolic execution. Pasket takes as input the framework API and tutorial programs that exercise the framework. From these artifacts and Pasket's internal knowledge of design patterns, Pasket synthesizes an executable framework model by instantiating design patterns, such that the behavior of a synthesized model on the tutorial programs matches that of the original framework. Lastly, in order to scale program synthesis to framework models, we devised adaptive concretization, a novel program synthesis algorithm that combines the best of the two major synthesis strategies: symbolic search, i.e., using SAT or SMT solvers, and explicit search, e.g., stochastic enumeration of possible solutions. Adaptive concretization parallelizes multiple sub-synthesis problems by partially concretizing highly influential unknowns in the original synthesis problem. Thanks to adaptive concretization, Pasket can generate a large-scale model, e.g., thousands lines of code. In addition, we have used an Android model synthesized by Pasket and found that the model is sufficient to allow SymDroid to execute a range of apps.

Ledroit Park, A Portrait In Black And White: A Study Of Historic Districts, Social Change, And The Process Of Neighborhood Placemaking

This research examines the process of placemaking in LeDroit Park, a residential Washington, DC, neighborhood with a historic district at its core. Unpacking the entwined physical and social evolution of the small community within the context of the Nation’s Capital, this analysis provides insight into the role of urban design and development as well as historic designation on shaping collective identity. Initially planned and designed in 1873 as a gated suburb just beyond the formal L’Enfant-designed city boundary, LeDroit Park was intended as a retreat for middle and upper-class European Americans from the growing density and social diversity of the city. With a mixture of large romantic revival mansions and smaller frame cottages set on grassy plots evocative of an idealized rural village, the physical design was intentionally inwardly-focused. This feeling of refuge was underscored with a physical fence that surrounded the development, intended to prevent African Americans from nearby Howard University and the surrounding neighborhood, from using the community’s private streets to access the City of Washington. Within two decades of its founding, LeDroit Park was incorporated into the District of Columbia, the surrounding fence was demolished, and the neighborhood was racially integrated. Due to increasingly stringent segregation laws and customs in the city, this period of integration lasted less than twenty years, and LeDroit Park developed into an elite African American enclave, using the urban design as a bulwark against the indignities of a segregated city. Throughout the 20th century housing infill and construction increased density, yet the neighborhood never lost the feeling of security derived from the neighborhood plan. Highlighting the architecture and street design, neighbors successfully received historic district designation in 1974 in order to halt campus expansion. After a stalemate that lasted two decades, the neighborhood began another period of transformation, both racial and socio-economic, catalyzed by a multi-pronged investment program led by Howard University. Through interviews with long-term and new community members, this investigation asserts that the 140-year development history, including recent physical interventions, is integral to placemaking, shaping the material character as well as the social identity of residents.

Adaptive Mechanisms of an Estuarine Synechococcus based on Genomics, Transcriptomics, and Proteomics

Picocyanobacteria are important phytoplankton and primary producers in the ocean. Although extensive work has been conducted for picocyanobacteria (i.e. Synechococcus and Prochlorococcus) in coastal and oceanic waters, little is known about those found in estuaries like the Chesapeake Bay. Synechococcus CB0101, an estuarine isolate, is more tolerant to shifts in temperature, salinity, and metal toxicity than coastal and oceanic Synechococcus strains, WH7803 and WH7805. Further, CB0101 has a greater sensitivity to high light intensity, likely due to its adaptation to low light environments. A complete and annotated genome sequence of CB0101 was completed to explore its genetic capacity and to serve as a basis for further molecular analysis. Comparative genomics between CB0101, WH7803, and WH7805 show that CB0101 contains more genes involved in regulation, sensing, and stress response. At the transcript and protein level, CB0101 regulates its metabolic pathways, transport systems, and sensing mechanisms when nitrate and phosphate are limited. Zinc toxicity led to oxidative stress and a global down regulation of photosystems and the translation machinery. From the stress response studies seven chromosomal toxin-antitoxin (TA) genes, were identified in CB0101, which led to the discovery of TA genes in several marine Synechococcus strains. The activation of the relB2/relE1 TA system allows CB0101 to arrest its growth under stressful conditions, but the growth arrest is reversible, once the stressful environment dissipates. The genome of CB0101 contains a relatively large number of genomic island (GI) genes compared to known marine Synechococcus genomes. Interestingly, a massive shutdown (255 out of 343) of GI genes occurred after CB0101 was infected by a lytic phage. On the other hand, phage-encoded host-like proteins (hli, psbA, ThyX) were highly expressed upon phage infection. This research provides new evidence that estuarine Synechococcus like CB0101 have inherited unique genetic machinery, which allows them to be versatile in the estuarine environment.