PLAStiCC: Predictive Look-Ahead Scheduling for Continuous dataflows on Clouds

Scalable stream processing and continuous dataflow systems are gaining traction with the rise of big data due to the need for processing high velocity data in near real time. Unlike batch processing systems such as MapReduce and workflows, static scheduling strategies fall short for continuous dataflows due to the variations in the input data rates and the need for sustained throughput. The elastic resource provisioning of cloud infrastructure is valuable to meet the changing resource needs of such continuous applications. However, multi-tenant cloud resources introduce yet another dimension of performance variability that impacts the application's throughput. In this paper we propose PLAStiCC, an adaptive scheduling algorithm that balances resource cost and application throughput using a prediction-based lookahead approach. It not only addresses variations in the input data rates but also the underlying cloud infrastructure. In addition, we also propose several simpler static scheduling heuristics that operate in the absence of accurate performance prediction model. These static and adaptive heuristics are evaluated through extensive simulations using performance traces obtained from Amazon AWS IaaS public cloud. Our results show an improvement of up to 20% in the overall profit as compared to the reactive adaptation algorithm.

A survey of cloud storage facilities

There are many applications such as software for processing customer records in telecom, patient records in hospitals, email processing software accessing a single email in a mailbox etc. which require to access a single record in a database consisting of millions of records. A basic feature of these applications is that they need to access data sets which are very large but simple. Cloud computing provides computing requirements for these kinds of new generation of applications involving very large data sets which cannot possibly be handled efficiently using traditional computing infrastructure. In this paper, we describe storage services provided by three well-known cloud service providers and give a comparison of their features with a view to characterize storage requirements of very large data sets as examples and we hope that it would act as a catalyst for the design of storage services for very large data set requirements in future. We also give a brief overview of other kinds of storage that have come up in the recent past for cloud computing.

Experiments on a plume with off-source heating: Implications for cloud fluid dynamics

We report here on a series of laboratory experiments on plumes, undertaken with the object of simulating the effect of the heat release that occurs in clouds on condensation of water vapor. The experimental technique used for this purpose relies on ohmic heating generated in an electrically conducting plume fluid subjected to a suitable alternating voltage across specified axial stations in the plume flow [Bhat et al., 1989]. The present series of experiments achieves a value of the Richardson number that is toward the lower end of the range that characteristics cumulus clouds. It is found that the buoyancy enhancement due to heating disrupts the eddy structures in the flow and reduces the dilution owing to entrainment of ambient fluid that would otherwise have occurred in the central region of the plume. Heating also reduces the spread rate of the plume, but as it accelerates the flow as well, the overall specific mass flux in the plume does not show a very significant change at the heat input employed in the experiment. However, there is some indication that the entrainment rate (proportional to the streamwise derivative of the mass flux) is slightly higher immediately after heat injection and slightly lower farther downstream. The measurements support a previous proposal for a cloud scenario [Bhat and Narasimha, 1996] and demonstrate how fresh insights into certain aspects of the fluid dynamics of clouds may be derived from the experimental techniques employed here.

Satellite-derived direct radiative effect of aerosols dependent on cloud cover

Aerosols from biomass burning can alter the radiative balance of the Earth by reflecting and absorbing solar radiation(1). Whether aerosols exert a net cooling or a net warming effect will depend on the aerosol type and the albedo of the underlying surface(2). Here, we use a satellite-based approach to quantify the direct, top-of-atmosphere radiative effect of aerosol layers advected over the partly cloudy boundary layer of the southeastern Atlantic Ocean during July-October of 2006 and 2007. We show that the warming effect of aerosols increases with underlying cloud coverage. This relationship is nearly linear, making it possible to define a critical cloud fraction at which the aerosols switch from exerting a net cooling to a net warming effect. For this region and time period, the critical cloud fraction is about 0.4, and is strongly sensitive to the amount of solar radiation the aerosols absorb and the albedo of the underlying clouds. We estimate that the regional-mean warming effect of aerosols is three times higher when large-scale spatial covariation between cloud cover and aerosols is taken into account. These results demonstrate the importance of cloud prediction for the accurate quantification of aerosol direct effects.

Fraunhoffer Diffraction Analysis of Satellite Cloud Imagery

The Fraunhoffer diffraction analysis of cloud-covered satellite imagery has shown that the diffraction pattern follows approximately cosine squared distribution. The overshooting tops of clouds and the shadows cast by them contribute much to the diffraction of light, particularly in the high-frequency range. Indeed, cloud-covered imagery can be distinguished from cloud-free imagery on the basis of rate of decay of the diffracted light power in the high-frequency band.

Modeling Finite Buffer Effects on TCP Traffic over an IEEE 802.11 Infrastructure WLAN

The network scenario is that of an infrastructure IEEE 802.11 WLAN with a single AP with which several stations (STAs) are associated. The AP has a finite size buffer for storing packets. In this scenario, we consider TCP controlled upload and download file transfers between the STAs and a server on the wireline LAN (e.g., 100 Mbps Ethernet) to which the AP is connected. In such a situation, it is known (see, for example, (3), [9]) that because of packet loss due to finite buffers at the Ap, upload file transfers obtain larger throughputs than download transfers. We provide an analytical model for estimating the upload and download throughputs as a function of the buffer size at the AP. We provide models for the undelayed and delayed ACK cases for a TCP that performs loss recovery only by timeout, and also for TCP Reno.

On the Maximum Cloud Zone and the ITCZ over Indian, Longitudes during the Southwest Monsoon

An investigation is presented of the daily variation of the maximum cloud zone (MCZ) and the 7W mb trough in the Northern Hemisphere over the Indian longitudes 70–90°E during April–October for 1973–77. It is found that during June–September there are two favorable locations for a MCZ over these longitudes–on a majority of days the MCZ is present in the monsoon zone north of 15°N, and often a secondary MCZ occurs in the equatorial region (0–10°N). The monsoon MCZ gets established by northward movement of the MCZ occurring over the equatorial Indian ocean in April and May. The secondary MCZ appears intermittently, and is characterized by long spells of persistence only when the monsoon MCZ is absent. In each of the seasons studied, the MCZ temporarily disappeared from the mean summer monsoon location (15–28°N) about four weeks after it was established near the beginning of July. It is reestablished by the northward movement of the secondary MCZ, which becomes active during the absence of the monsoon MCZ, in a manner strikingly similar to that observed in the spring to summer transition. A break in monsoon conditions prevails just prior to the temporary disappearance of the monsoon MCZ. Thus we conclude that the monsoon MCZ cannot survive for longer than a month without reestablishment by the secondary MCZ. Possible underlying mechanisms are also discussed.

Modeling finite buffer effects on TCP traffic over an IEEE 802.11 infrastructure WLAN

The network scenario is that of an infrastructure IEEE 802.11 WLAN with a single AP with which several stations (STAs) are associated. The AP has a finite size buffer for storing packets. In this scenario, we consider TCP-controlled upload and download file transfers between the STAs and a server on the wireline LAN (e.g., 100 Mbps Ethernet) to which the AP is connected. In such a situation, it is well known that because of packet losses due to finite buffers at the AP, upload file transfers obtain larger throughputs than download transfers. We provide an analytical model for estimating the upload and download throughputs as a function of the buffer size at the AP. We provide models for the undelayed and delayed ACK cases for a TCP that performs loss recovery only by timeout, and also for TCP Reno. The models are validated incomparison with NS2 simulations.

Rainfall changes over tropical montane cloud forests of southern Western Ghats, India

The southern Western Ghats tropical montane cloud forest sites (Gavi, Periyar, High wavys and Venniyar), which are characterized by frequent or seasonal cloud cover at the vegetation level, are considered one of the most threatened ecosystems in India and the world. Three out of four montane cloud forest sites studied in the southern Western Ghats had experienced diminishing trends of seasonal average and total rainfall, especially during summer monsoon season. The highest level of reduction for summer monsoon season was observed at Gavi rainforest station (>20 mm/14 years) in Kerala followed by Venniyar (>20 mm/20 years) site in Tamil Nadu. Average annual and total precipitation increased during the study period irrespective of the seasons over Periyar area, and the greatest values were recorded for season 2 (>25 mm/28 years). Positive trends for winter monsoon rainfall has been observed for three stations (Periyar, High wavys and Venniyar) except Gavi, and the trend was positive and significant (90%) for Periyar and High wavys. Increase in summer monsoon rainfall was observed for Periyar site and the trend was found to be significant (95%).

Starbursts triggered by cloud compression in interacting galaxies

We propose a physical mechanism for the triggering of starbursts in interacting spiral galaxies by shock compression of the pre-existing disk giant molecular clouds (GMCs). We show that as a disk GMC tumbles into the central region of a galaxy following a galactic tidal encounter, it undergoes a radiative shock compression by the pre-existing high pressure of the central molecular intercloud medium. The shocked outer shell of a GMC becomes gravitationally unstable, which results in a burst of star formation in the initially stable GMC. In the case of colliding galaxies with physical overlap such as Arp 244, the cloud compression is shown to occur due to the hot, high-pressure remnant gas resulting from the collisions of atomic hydrogen gas clouds from the two galaxies. The resulting values of infrared luminosity agree with observations. The main mode of triggered star formation is via clusters of stars, thus we can naturally explain the formation of young, luminous star clusters observed in starburst galaxies.

OPSM - Opportunistic Power Save mode for Infrastructure IEEE 802.11 WLAN

We focus on the energy spent in radio communication by the stations (STAs) in an IEEE 802.11 infrastructure WLAN. All the STAs are engaged in web browsing, which is characterized by a short file downloads over TCP, with short duration of inactivity or think time in between two file downloads. Under this traffic, Static PSM (SPSM) performs better than CAM, since the STAs in SPSM can switch to low power state (sleep) during think times while in CAM they have to be in the active state all the time. In spite of this gain, performance of SPSM degrades due to congestion, as the number of STAs associated with the access point (AP) increases. To address this problem, we propose an algorithm, which we call opportunistic PSM (OPSM). We show through simulations that OPSM performs better than SPSM under the aforementioned TCP traffic. The performance gain achieved by OPSM over SPSM increases as the mean file size requested by the STAs or the number of STAs associated with the AP increases. We implemented OPSM in NS-2.33, and to compare the performance of OPSM and SPSM, we evaluate the number of file downloads that can be completed with a given battery capacity and the average time taken to download a file.

Aerosol radiative forcing over land: effect of surface and cloud reflection

It is now clearly understood that atmospheric aerosols have a significant impact on climate due to their important role in modifying the incoming solar and outgoing infrared radiation. The question of whether aerosol cools (negative forcing) or warms (positive forcing) the planet depends on the relative dominance of absorbing aerosols. Recent investigations over the tropical Indian Ocean have shown that, irrespective of the comparatively small percentage contribution in optical depth (similar to11%), soot has an important role in the overall radiative forcing. However, when the amount of absorbing aerosols such as soot are significant, aerosol optical depth and chemical composition are not the only determinants of aerosol climate effects, but the altitude of the aerosol layer and the altitude and type of clouds are also important. In this paper, the aerosol forcing in the presence of clouds and the effect of different surface types (ocean, soil, vegetation, and different combinations of soil and vegetation) are examined based on model simulations, demonstrating that aerosol forcing changes sign from negative (cooling) to positive (warming) when reflection from below (either due to land or clouds) is high.

Analytical models for energy consumption in infrastructure WLAN STAs carrying TCP traffic

We develop analytical models for estimating the energy spent by stations (STAs) in infrastructure WLANs when performing TCP controlled file downloads. We focus on the energy spent in radio communication when the STAs are in the Continuously Active Mode (CAM), or in the static Power Save Mode (PSM). Our approach is to develop accurate models for obtaining the fraction of times the STA radios spend in idling, receiving and transmitting. We discuss two traffic models for each mode of operation: (i) each STA performs one large file download, and (ii) the STAs perform short file transfers. We evaluate the rate of STA energy expenditure with long file downloads, and show that static PSM is worse than just using CAM. For short file downloads we compute the number of file downloads that can be completed with given battery capacity, and show that PSM performs better than CAM for this case. We provide a validation of our analytical models using the NS-2 simulator.