Improving resource management in virtualized data centers using application performance models

The rapid growth of virtualized data centers and cloud hosting services is making the management of physical resources such as CPU, memory, and I/O bandwidth in data center servers increasingly important. Server management now involves dealing with multiple dissimilar applications with varying Service-Level-Agreements (SLAs) and multiple resource dimensions. The multiplicity and diversity of resources and applications are rendering administrative tasks more complex and challenging. This thesis aimed to develop a framework and techniques that would help substantially reduce data center management complexity.^ We specifically addressed two crucial data center operations. First, we precisely estimated capacity requirements of client virtual machines (VMs) while renting server space in cloud environment. Second, we proposed a systematic process to efficiently allocate physical resources to hosted VMs in a data center. To realize these dual objectives, accurately capturing the effects of resource allocations on application performance is vital. The benefits of accurate application performance modeling are multifold. Cloud users can size their VMs appropriately and pay only for the resources that they need; service providers can also offer a new charging model based on the VMs performance instead of their configured sizes. As a result, clients will pay exactly for the performance they are actually experiencing; on the other hand, administrators will be able to maximize their total revenue by utilizing application performance models and SLAs. ^ This thesis made the following contributions. First, we identified resource control parameters crucial for distributing physical resources and characterizing contention for virtualized applications in a shared hosting environment. Second, we explored several modeling techniques and confirmed the suitability of two machine learning tools, Artificial Neural Network and Support Vector Machine, to accurately model the performance of virtualized applications. Moreover, we suggested and evaluated modeling optimizations necessary to improve prediction accuracy when using these modeling tools. Third, we presented an approach to optimal VM sizing by employing the performance models we created. Finally, we proposed a revenue-driven resource allocation algorithm which maximizes the SLA-generated revenue for a data center.^

Fair and QoS-oriented resource management in heterogeneous networks

In this paper, a heterogeneous network composed of femtocells deployed within a macrocell network is considered, and a quality-of-service (QoS)-oriented fairness metric which captures important characteristics of tiered network architectures is proposed. Using homogeneous Poisson processes, the sum capacities in such networks are expressed in closed form for co-channel, dedicated channel, and hybrid resource allocation methods. Then a resource splitting strategy that simultaneously considers capacity maximization, fairness constraints, and QoS constraints is proposed. Detailed computer simulations utilizing 3GPP simulation assumptions show that a hybrid allocation strategy with a well-designed resource split ratio enjoys the best cell-edge user performance, with minimal degradation in the sum throughput of macrocell users when compared with that of co-channel operation.

Ecohydrology, evapotranspiration and hydrogeochemistry of carbonate mangrove wetlands

Coastal environments can be highly susceptible to environmental changes caused by anthropogenic pressures and natural events. Both anthropogenic and natural perturbations may directly affect the amount and the quality of water flowing through the ecosystem, both in the surface and subsurface and can subsequently, alter ecological communities and functions. The Florida Everglades and the Sian Ka'an Biosphere Reserve (Mexico) are two large ecosystems with an extensive coastal mangrove ecotone that represent a historically altered and pristine environment, respectively. Rising sea levels, climate change, increased water demand, and salt water intrusion are growing concerns in these regions and underlies the need for a better understanding of the present conditions. The goal of my research was to better understand various ecohydrological, environmental, and hydrogeochemical interactions and relationships in carbonate mangrove wetlands. A combination of aqueous geochemical analyses and visible and near-infrared reflectance data were employed to explore relationships between surface and subsurface water chemistry and spectral biophysical stress in mangroves. Optical satellite imagery and field collected meteorological data were used to estimate surface energy and evapotranspiration and measure variability associated with hurricanes and restoration efforts. Furthermore, major ionic and nutrient concentrations, and stable isotopes of hydrogen and oxygen were used to distinguish water sources and infer coastal groundwater discharge by applying the data to a combined principal component analysis-end member mixing model. Spectral reflectance measured at the field and satellite scales were successfully used to estimate surface and subsurface water chemistry and model chloride concentrations along the southern Everglades. Satellite imagery indicated that mangrove sites that have less tidal flushing and hydrogeomorphic heterogeneity tend to have more variable evapotranspiration and soil heat flux in response to storms and restoration. Lastly, water chemistry and multivariate analyses indicated two distinct fresh groundwater sources that discharge to the phosphorus-limited estuaries and bays of the Sian Ka'an Biopshere Reserve; and that coastal groundwater discharge was an important source for phosphorus. The results of the study give us a better understanding of the ecohydrological and hydrogeological processes in carbonate mangrove environments that can be then be extrapolated to similar coastal ecosystems in the Caribbean.

Fuzzy Modeling and Control Based Virtual Machine Resource Management

Virtual machines (VMs) are powerful platforms for building agile datacenters and emerging cloud systems. However, resource management for a VM-based system is still a challenging task. First, the complexity of application workloads as well as the interference among competing workloads makes it difficult to understand their VMs’ resource demands for meeting their Quality of Service (QoS) targets; Second, the dynamics in the applications and system makes it also difficult to maintain the desired QoS target while the environment changes; Third, the transparency of virtualization presents a hurdle for guest-layer application and host-layer VM scheduler to cooperate and improve application QoS and system efficiency. This dissertation proposes to address the above challenges through fuzzy modeling and control theory based VM resource management. First, a fuzzy-logic-based nonlinear modeling approach is proposed to accurately capture a VM’s complex demands of multiple types of resources automatically online based on the observed workload and resource usages. Second, to enable fast adaption for resource management, the fuzzy modeling approach is integrated with a predictive-control-based controller to form a new Fuzzy Modeling Predictive Control (FMPC) approach which can quickly track the applications’ QoS targets and optimize the resource allocations under dynamic changes in the system. Finally, to address the limitations of black-box-based resource management solutions, a cross-layer optimization approach is proposed to enable cooperation between a VM’s host and guest layers and further improve the application QoS and resource usage efficiency. The above proposed approaches are prototyped and evaluated on a Xen-based virtualized system and evaluated with representative benchmarks including TPC-H, RUBiS, and TerraFly. The results demonstrate that the fuzzy-modeling-based approach improves the accuracy in resource prediction by up to 31.4% compared to conventional regression approaches. The FMPC approach substantially outperforms the traditional linear-model-based predictive control approach in meeting application QoS targets for an oversubscribed system. It is able to manage dynamic VM resource allocations and migrations for over 100 concurrent VMs across multiple hosts with good efficiency. Finally, the cross-layer optimization approach further improves the performance of a virtualized application by up to 40% when the resources are contended by dynamic workloads.

Freshwater Resource Supply Modeling for Developed and Undeveloped Watersheds

Globally, the current state of freshwater resource management is insufficient and impeding the chance at a sustainable future. Human interference within the natural hydrologic cycle is becoming dangerously irreversible and the need to redefine resource managerial approaches is imminent. This research involves the development of a coupled natural-human freshwater resource supply model using a System Dynamics approach. The model was applied to two case studies, Somalia, Africa and the Phoenix Active Management Area in Arizona, USA. It is suggested that System Dynamic modeling would be an invaluable tool for achieving sustainable freshwater resource management in individual watersheds. Through a series of thought experiments, a thorough understanding of the systems’ dynamic behaviors is obtainable for freshwater resource managers and policy-makers to examine various courses of action for alleviating freshwater supply concerns. This thesis reviews the model, its development and an analysis of several thought experiments applied to the case studies.

Improving Resource Management in Virtualized Data Centers using Application Performance Models

The rapid growth of virtualized data centers and cloud hosting services is making the management of physical resources such as CPU, memory, and I/O bandwidth in data center servers increasingly important. Server management now involves dealing with multiple dissimilar applications with varying Service-Level-Agreements (SLAs) and multiple resource dimensions. The multiplicity and diversity of resources and applications are rendering administrative tasks more complex and challenging. This thesis aimed to develop a framework and techniques that would help substantially reduce data center management complexity. We specifically addressed two crucial data center operations. First, we precisely estimated capacity requirements of client virtual machines (VMs) while renting server space in cloud environment. Second, we proposed a systematic process to efficiently allocate physical resources to hosted VMs in a data center. To realize these dual objectives, accurately capturing the effects of resource allocations on application performance is vital. The benefits of accurate application performance modeling are multifold. Cloud users can size their VMs appropriately and pay only for the resources that they need; service providers can also offer a new charging model based on the VMs performance instead of their configured sizes. As a result, clients will pay exactly for the performance they are actually experiencing; on the other hand, administrators will be able to maximize their total revenue by utilizing application performance models and SLAs. This thesis made the following contributions. First, we identified resource control parameters crucial for distributing physical resources and characterizing contention for virtualized applications in a shared hosting environment. Second, we explored several modeling techniques and confirmed the suitability of two machine learning tools, Artificial Neural Network and Support Vector Machine, to accurately model the performance of virtualized applications. Moreover, we suggested and evaluated modeling optimizations necessary to improve prediction accuracy when using these modeling tools. Third, we presented an approach to optimal VM sizing by employing the performance models we created. Finally, we proposed a revenue-driven resource allocation algorithm which maximizes the SLA-generated revenue for a data center.

Geospatial Data Indexing Analysis and Visualization via Web Services with Autonomic Resource Management

With the exponential growth of the usage of web-based map services, the web GIS application has become more and more popular. Spatial data index, search, analysis, visualization and the resource management of such services are becoming increasingly important to deliver user-desired Quality of Service. First, spatial indexing is typically time-consuming and is not available to end-users. To address this, we introduce TerraFly sksOpen, an open-sourced an Online Indexing and Querying System for Big Geospatial Data. Integrated with the TerraFly Geospatial database [1-9], sksOpen is an efficient indexing and query engine for processing Top-k Spatial Boolean Queries. Further, we provide ergonomic visualization of query results on interactive maps to facilitate the user’s data analysis. Second, due to the highly complex and dynamic nature of GIS systems, it is quite challenging for the end users to quickly understand and analyze the spatial data, and to efficiently share their own data and analysis results with others. Built on the TerraFly Geo spatial database, TerraFly GeoCloud is an extra layer running upon the TerraFly map and can efficiently support many different visualization functions and spatial data analysis models. Furthermore, users can create unique URLs to visualize and share the analysis results. TerraFly GeoCloud also enables the MapQL technology to customize map visualization using SQL-like statements [10]. Third, map systems often serve dynamic web workloads and involve multiple CPU and I/O intensive tiers, which make it challenging to meet the response time targets of map requests while using the resources efficiently. Virtualization facilitates the deployment of web map services and improves their resource utilization through encapsulation and consolidation. Autonomic resource management allows resources to be automatically provisioned to a map service and its internal tiers on demand. v-TerraFly are techniques to predict the demand of map workloads online and optimize resource allocations, considering both response time and data freshness as the QoS target. The proposed v-TerraFly system is prototyped on TerraFly, a production web map service, and evaluated using real TerraFly workloads. The results show that v-TerraFly can accurately predict the workload demands: 18.91% more accurate; and efficiently allocate resources to meet the QoS target: improves the QoS by 26.19% and saves resource usages by 20.83% compared to traditional peak load-based resource allocation.

Estimation of ravine sediment production and sediment dynamics in the lower Le Sueur River watershed, Minnesota

This study focuses on quantifying explicitly the sediment budget of deeply incised ravines in the lower Le Sueur River watershed, in southern Minnesota. High-rate-gully-erosion equations along with the Universal Soil Loss Equation (USLE) were implemented in a numerical modeling approach that is based on a time-integration of the sediment balance equations. The model estimates the rates of ravine width and depth change and the amount of sediment periodically flushing from the ravines. Components of the sediment budget of the ravines were simulated with the model and results suggest that the ravine walls are the major sediment source in the ravines. A sensitivity analysis revealed that the erodibility coefficients of the gully bed and wall, the local slope angle and the Manning’s coefficient are the key parameters controlling the rate of sediment production. Recommendations to guide further monitoring efforts in the watershed and increased detail modeling approaches are highlighted as a result of this modeling effort.

Assessment of the occurrence and potential effects of pharmaceuticals and personal care products in South Florida waters and sediments

A LLE-GC-MS method was developed to detect PPCPs in surface water samples from Big Cypress National Park, Everglades National Park and Biscayne National Park in South Florida. The most frequently found PPCPs were caffeine, DEET and triclosan with detected maximum concentration of 169 ng/L, 27.9 ng/L and 10.9 ng/L, respectively. The detection frequencies of hormones were less than PPCPs. Detected maximal concentrations of estrone, 17β-estradiol, coprostan-3-ol, coprostane and coprostan-3-one were 5.98 ng/L, 3.34 ng/L, 16.5 ng/L, 13.5 ng/L and 6.79 ng/L, respectively. An ASE-SPE-GC-MS method was developed and applied to the analysis of the sediment and soil area where reclaimed water was used for irrigation. Most analytes were below detection limits, even though some of analytes were detected in the reclaimed water at relatively high concentrations corroborating the fact that PPCPs do not significantly partition to mineral phases. An online SPE-HPLC-APPI-MS/MS method and an online SPE-HPLC-HESI-MS/MS method were developed to analyze reclaimed water and drinking water samples. In the reclaimed water study, reclaimed water samples were collected from the sprinkler for a year-long period at Florida International University Biscayne Bay Campus, where reclaimed water was reused for irrigation. Analysis results showed that several analytes were continuously detected in all reclaimed water samples. Coprostanol, bisphenol A and DEET's maximum concentration exceeded 10 μg/L (ppb). The four most frequently detected compounds were diphenhydramine (100%), DEET (98%), atenolol (98%) and carbamazepine (96%). In the study of drinking water, 54 tap water samples were collected from the Miami-Dade area. The maximum concentrations of salicylic acid, ibuprofen and DEET were 521 ng/L, 301 ng/L and 290 ng/L, respectively. The three most frequently detected compounds were DEET (93%), carbamazepine (43%) and salicylic acid (37%), respectively. Because the source of drinking water in Miami-Dade County is the relatively pristine Biscayne aquifer, these findings suggest the presence of wastewater intrusions into the delivery system or the onset of direct influence of surface waters into the shallow aquifer.

Determination of vertical and horizontal pathways of injected fresh wastewater into a deep saline aquifer (Florida, USA) using natural chemical tracers

Two deep-well injection sites in south Florida, USA, inject an average of 430 million liters per day (MLD) of treated domestic fresh wastewater into a deep saline aquifer 900 m below land surface. Elevated levels of NH3 (highest concentration 939 µmol) in the overlying aquifer above ambient concentrations (concentration less than 30 µmol) were evidence of the upward migration of injected fluids. Three pathways were distinguished based on ammonium, chloride and bromide ratios, and temperature. At the South District Wastewater Treatment Plant, the tracer ratios showed that the injectate remained chemically distinct as it migrated upwards through rapid vertical pathways via density-driven buoyancy. The warmer injectate (mean 28°C) retained the temperature signal as it vertically migrated upwards; however, the temperature signal did not persist as the injectate moved horizontally into the overlying aquifers. Once introduced, the injectate moved slowly horizontally through the aquifer and mixed with ambient water. At the North District Wastewater Treatment Plant, data provide strong evidence of a one-time pulse of injectate into the overlying aquifers due to improper well construction. No evidence of rapid vertical pathways was observed at the North District Wastewater Treatment Plant.

Simulating everglades national park hydrology and phosphorus transport under existing and future scenarios using numerical modeling

The Florida Everglades has a long history of anthropogenic changes which have impacted the quantity and quality of water entering the system. Since the construction of Tamiami Trail in the 1920's, overland flow to the Florida Everglades has decreased significantly, impacting ecosystems from the wetlands to the estuary. The MIKE Marsh Model of Everglades National Park (M3ENP) is a numerical model, which simulates Everglades National Park (ENP) hydrology using MIKE SHE/MIKE 11software. This model has been developed to determine the parameters that effect Everglades hydrology and understand the impact of specific flow changes on the hydrology of the system. ^ As part of the effort to return flows to the historical levels, several changes to the existing water management infrastructure have been implemented or are in the design phase. Bridge construction scenarios were programed into the M3ENP model to review the effect of these structural changes and evaluate the potential impacts on water levels and hydroperiods in the receiving Northeast Shark Slough ecosystem. These scenarios have shown critical water level increases in an area which has been in decline due to low water levels. Results from this work may help guide future decisions for restoration designs. ^ Excess phosphorus entering Everglades National Park in South Florida may promote the growth of more phosphorus-opportunistic species and alter the food chain from the bottom up. Two phosphorus transport methods were developed into the M3ENP hydrodynamic model to determine the factors affecting phosphorus transport and the impact of bridge construction on water quality. Results showed that while phosphorus concentrations in surface waters decreased overall, some areas within ENP interior may experience an increase in phosphorus loading which the addition of bridges to Tamiami Trail. Finally, phosphorus data and modeled water level data was used to evaluate the spectral response of Everglades vegetation to increasing phosphorus availability using Landsat imagery.^

Water Governance in Haiti: An Assessment of Laws and Institutional Capacities

The Republic of Haiti struggles to sustainably manage its water resources. Public health is compromised by low levels of water supply, sanitation, and hygiene, and water resources are often contaminated and unsustainably allocated. While poor governance is often blamed for these shortcomings, the laws and institutions regulating water resources in Haiti are poorly understood, especially by the international community. This study brings together and analyzes Haitian water laws, assesses institutional capacities, and provides a case study of water management in northern Haiti in order to provide a more complete picture of the sector. Funded by the Inter-American Development Bank as part of the Water Availability, Quality and Integrated Water Resources Management in Northern Haiti (HA-T1179) Project, this study took place from January-July 2015, with the help of local experts and participating stakeholders. The results indicate that Haiti’s water law framework is highly fragmented, with overlapping mandates and little coordination between ministries at the national level, and ambiguous but unrealistic roles for subnational governments. A capacity assessment of institutions in northern Haiti illustrates that while local stakeholders are engaged, human and financial resources are insufficient to carry out statutory responsibilities. The findings suggest that water resources management planning should engage local governments and community fixtures while supplementing capacities with national or international support.