Improving resource management in virtualized data centers using application performance models

The rapid growth of virtualized data centers and cloud hosting services is making the management of physical resources such as CPU, memory, and I/O bandwidth in data center servers increasingly important. Server management now involves dealing with multiple dissimilar applications with varying Service-Level-Agreements (SLAs) and multiple resource dimensions. The multiplicity and diversity of resources and applications are rendering administrative tasks more complex and challenging. This thesis aimed to develop a framework and techniques that would help substantially reduce data center management complexity.^ We specifically addressed two crucial data center operations. First, we precisely estimated capacity requirements of client virtual machines (VMs) while renting server space in cloud environment. Second, we proposed a systematic process to efficiently allocate physical resources to hosted VMs in a data center. To realize these dual objectives, accurately capturing the effects of resource allocations on application performance is vital. The benefits of accurate application performance modeling are multifold. Cloud users can size their VMs appropriately and pay only for the resources that they need; service providers can also offer a new charging model based on the VMs performance instead of their configured sizes. As a result, clients will pay exactly for the performance they are actually experiencing; on the other hand, administrators will be able to maximize their total revenue by utilizing application performance models and SLAs. ^ This thesis made the following contributions. First, we identified resource control parameters crucial for distributing physical resources and characterizing contention for virtualized applications in a shared hosting environment. Second, we explored several modeling techniques and confirmed the suitability of two machine learning tools, Artificial Neural Network and Support Vector Machine, to accurately model the performance of virtualized applications. Moreover, we suggested and evaluated modeling optimizations necessary to improve prediction accuracy when using these modeling tools. Third, we presented an approach to optimal VM sizing by employing the performance models we created. Finally, we proposed a revenue-driven resource allocation algorithm which maximizes the SLA-generated revenue for a data center.^

Fuzzy Modeling and Control Based Virtual Machine Resource Management

Virtual machines (VMs) are powerful platforms for building agile datacenters and emerging cloud systems. However, resource management for a VM-based system is still a challenging task. First, the complexity of application workloads as well as the interference among competing workloads makes it difficult to understand their VMs’ resource demands for meeting their Quality of Service (QoS) targets; Second, the dynamics in the applications and system makes it also difficult to maintain the desired QoS target while the environment changes; Third, the transparency of virtualization presents a hurdle for guest-layer application and host-layer VM scheduler to cooperate and improve application QoS and system efficiency. This dissertation proposes to address the above challenges through fuzzy modeling and control theory based VM resource management. First, a fuzzy-logic-based nonlinear modeling approach is proposed to accurately capture a VM’s complex demands of multiple types of resources automatically online based on the observed workload and resource usages. Second, to enable fast adaption for resource management, the fuzzy modeling approach is integrated with a predictive-control-based controller to form a new Fuzzy Modeling Predictive Control (FMPC) approach which can quickly track the applications’ QoS targets and optimize the resource allocations under dynamic changes in the system. Finally, to address the limitations of black-box-based resource management solutions, a cross-layer optimization approach is proposed to enable cooperation between a VM’s host and guest layers and further improve the application QoS and resource usage efficiency. The above proposed approaches are prototyped and evaluated on a Xen-based virtualized system and evaluated with representative benchmarks including TPC-H, RUBiS, and TerraFly. The results demonstrate that the fuzzy-modeling-based approach improves the accuracy in resource prediction by up to 31.4% compared to conventional regression approaches. The FMPC approach substantially outperforms the traditional linear-model-based predictive control approach in meeting application QoS targets for an oversubscribed system. It is able to manage dynamic VM resource allocations and migrations for over 100 concurrent VMs across multiple hosts with good efficiency. Finally, the cross-layer optimization approach further improves the performance of a virtualized application by up to 40% when the resources are contended by dynamic workloads.

Numerical groundwater flow modeling in the Wakal River basin, India

Increasing dependence on groundwater in the Wakal River basin, India, jeopardizes water supply sustainability. A numerical groundwater model was developed to better understand the aquifer system and to evaluate its potential in terms of quantity and replenishment. Potential artificial recharge areas were delineated using landscape and hydrogeologic parameters, Geographic Information System (GIS), and remote sensing. Groundwater models are powerful tools for recharge estimation when transmissivity is known. Proper recharge must be applied to reproduce field-measured heads. The model showed that groundwater levels could decline significantly if there are two drought years in every four years that result in reduced recharge, and groundwater withdrawal is increased by 15%. The effect of such drought is currently uncertain however, because runoff from the basin is unknown. Remote sensing and GIS revealed areas with slopes less than 5%, forest cover, and Normalized Difference Vegetative Index greater than 0.5 that are suitable recharge sites.

Attitudes towards the Marine Environment and Implications for Marine Resource Management in Seward, Alaska

This research, conducted in 2006-2008, examines the ways in which various groups involved with the marine resources of Seward, Alaska construct attitudes towards the environment. Participant observation and semi-structured interviews are used to assess how commercial halibut fishers, tour boat operators, local residents and government officials understand the marine environment based on their previous experiences. This study also explores how ideologies relate to the current practices of each group. Two theories orient the analyses: The first, cultural modeling provided a theoretical and methodological framework for pursuing a more comprehensive analysis of resource management. The second, Theory of Reasoned Action (Ajzen and Fishbein 1980), guided the analysis of the ways in which each participant’s ideology towards the marine environment relates to their practice. Aside from contributing to a better understanding of a coastal community’s ideologies and practices, this dissertation sought to better understand the role of ecological ideologies and behaviors in fisheries management. The research illustrates certain domains where ideologies and practices concerning Pacific halibut and the marine environment differ among commercial fishers, government, and management officials, tour boat operators and residents of Seward, AK. These differences offer insights into how future collaborative efforts between government officials, managers and local marine resource users might better incorporate local ideology into management, and provide ecological information to local marine resource users in culturally appropriate ways.

Assessment of the occurrence and potential effects of pharmaceuticals and personal care products in South Florida waters and sediments

A LLE-GC-MS method was developed to detect PPCPs in surface water samples from Big Cypress National Park, Everglades National Park and Biscayne National Park in South Florida. The most frequently found PPCPs were caffeine, DEET and triclosan with detected maximum concentration of 169 ng/L, 27.9 ng/L and 10.9 ng/L, respectively. The detection frequencies of hormones were less than PPCPs. Detected maximal concentrations of estrone, 17β-estradiol, coprostan-3-ol, coprostane and coprostan-3-one were 5.98 ng/L, 3.34 ng/L, 16.5 ng/L, 13.5 ng/L and 6.79 ng/L, respectively. An ASE-SPE-GC-MS method was developed and applied to the analysis of the sediment and soil area where reclaimed water was used for irrigation. Most analytes were below detection limits, even though some of analytes were detected in the reclaimed water at relatively high concentrations corroborating the fact that PPCPs do not significantly partition to mineral phases. An online SPE-HPLC-APPI-MS/MS method and an online SPE-HPLC-HESI-MS/MS method were developed to analyze reclaimed water and drinking water samples. In the reclaimed water study, reclaimed water samples were collected from the sprinkler for a year-long period at Florida International University Biscayne Bay Campus, where reclaimed water was reused for irrigation. Analysis results showed that several analytes were continuously detected in all reclaimed water samples. Coprostanol, bisphenol A and DEET's maximum concentration exceeded 10 μg/L (ppb). The four most frequently detected compounds were diphenhydramine (100%), DEET (98%), atenolol (98%) and carbamazepine (96%). In the study of drinking water, 54 tap water samples were collected from the Miami-Dade area. The maximum concentrations of salicylic acid, ibuprofen and DEET were 521 ng/L, 301 ng/L and 290 ng/L, respectively. The three most frequently detected compounds were DEET (93%), carbamazepine (43%) and salicylic acid (37%), respectively. Because the source of drinking water in Miami-Dade County is the relatively pristine Biscayne aquifer, these findings suggest the presence of wastewater intrusions into the delivery system or the onset of direct influence of surface waters into the shallow aquifer.

Determination of vertical and horizontal pathways of injected fresh wastewater into a deep saline aquifer (Florida, USA) using natural chemical tracers

Two deep-well injection sites in south Florida, USA, inject an average of 430 million liters per day (MLD) of treated domestic fresh wastewater into a deep saline aquifer 900 m below land surface. Elevated levels of NH3 (highest concentration 939 µmol) in the overlying aquifer above ambient concentrations (concentration less than 30 µmol) were evidence of the upward migration of injected fluids. Three pathways were distinguished based on ammonium, chloride and bromide ratios, and temperature. At the South District Wastewater Treatment Plant, the tracer ratios showed that the injectate remained chemically distinct as it migrated upwards through rapid vertical pathways via density-driven buoyancy. The warmer injectate (mean 28°C) retained the temperature signal as it vertically migrated upwards; however, the temperature signal did not persist as the injectate moved horizontally into the overlying aquifers. Once introduced, the injectate moved slowly horizontally through the aquifer and mixed with ambient water. At the North District Wastewater Treatment Plant, data provide strong evidence of a one-time pulse of injectate into the overlying aquifers due to improper well construction. No evidence of rapid vertical pathways was observed at the North District Wastewater Treatment Plant.

Ecohydrology, evapotranspiration and hydrogeochemistry of carbonate mangrove wetlands

Coastal environments can be highly susceptible to environmental changes caused by anthropogenic pressures and natural events. Both anthropogenic and natural perturbations may directly affect the amount and the quality of water flowing through the ecosystem, both in the surface and subsurface and can subsequently, alter ecological communities and functions. The Florida Everglades and the Sian Ka'an Biosphere Reserve (Mexico) are two large ecosystems with an extensive coastal mangrove ecotone that represent a historically altered and pristine environment, respectively. Rising sea levels, climate change, increased water demand, and salt water intrusion are growing concerns in these regions and underlies the need for a better understanding of the present conditions. The goal of my research was to better understand various ecohydrological, environmental, and hydrogeochemical interactions and relationships in carbonate mangrove wetlands. A combination of aqueous geochemical analyses and visible and near-infrared reflectance data were employed to explore relationships between surface and subsurface water chemistry and spectral biophysical stress in mangroves. Optical satellite imagery and field collected meteorological data were used to estimate surface energy and evapotranspiration and measure variability associated with hurricanes and restoration efforts. Furthermore, major ionic and nutrient concentrations, and stable isotopes of hydrogen and oxygen were used to distinguish water sources and infer coastal groundwater discharge by applying the data to a combined principal component analysis-end member mixing model. Spectral reflectance measured at the field and satellite scales were successfully used to estimate surface and subsurface water chemistry and model chloride concentrations along the southern Everglades. Satellite imagery indicated that mangrove sites that have less tidal flushing and hydrogeomorphic heterogeneity tend to have more variable evapotranspiration and soil heat flux in response to storms and restoration. Lastly, water chemistry and multivariate analyses indicated two distinct fresh groundwater sources that discharge to the phosphorus-limited estuaries and bays of the Sian Ka'an Biopshere Reserve; and that coastal groundwater discharge was an important source for phosphorus. The results of the study give us a better understanding of the ecohydrological and hydrogeological processes in carbonate mangrove environments that can be then be extrapolated to similar coastal ecosystems in the Caribbean.

Simulating everglades national park hydrology and phosphorus transport under existing and future scenarios using numerical modeling

The Florida Everglades has a long history of anthropogenic changes which have impacted the quantity and quality of water entering the system. Since the construction of Tamiami Trail in the 1920's, overland flow to the Florida Everglades has decreased significantly, impacting ecosystems from the wetlands to the estuary. The MIKE Marsh Model of Everglades National Park (M3ENP) is a numerical model, which simulates Everglades National Park (ENP) hydrology using MIKE SHE/MIKE 11software. This model has been developed to determine the parameters that effect Everglades hydrology and understand the impact of specific flow changes on the hydrology of the system. ^ As part of the effort to return flows to the historical levels, several changes to the existing water management infrastructure have been implemented or are in the design phase. Bridge construction scenarios were programed into the M3ENP model to review the effect of these structural changes and evaluate the potential impacts on water levels and hydroperiods in the receiving Northeast Shark Slough ecosystem. These scenarios have shown critical water level increases in an area which has been in decline due to low water levels. Results from this work may help guide future decisions for restoration designs. ^ Excess phosphorus entering Everglades National Park in South Florida may promote the growth of more phosphorus-opportunistic species and alter the food chain from the bottom up. Two phosphorus transport methods were developed into the M3ENP hydrodynamic model to determine the factors affecting phosphorus transport and the impact of bridge construction on water quality. Results showed that while phosphorus concentrations in surface waters decreased overall, some areas within ENP interior may experience an increase in phosphorus loading which the addition of bridges to Tamiami Trail. Finally, phosphorus data and modeled water level data was used to evaluate the spectral response of Everglades vegetation to increasing phosphorus availability using Landsat imagery.^