Constraints upon the response of fish and crayfish to environmental flow releases in a regulated headwater stream network

In dry climate zones, headwater streams are often regulated for water extraction causing intermittency in perennial streams and prolonged drying in intermittent streams. Regulation thereby reduces aquatic habitat downstream of weirs that also form barriers to migration by stream fauna. Environmental flow releases may restore streamflow in rivers, but are rarely applied to headwaters. We sampled fish and crayfish in four regulated headwater streams before and after the release of summer-autumn environmental flows, and in four nearby unregulated streams, to determine whether their abundances increased in response to flow releases. Historical data of fish and crayfish occurrence spanning a 30 year period was compared with contemporary data (electrofishing surveys, Victoria Range, Australia; summer 2008 to summer 2010) to assess the longer-term effects of regulation and drought. Although fish were recorded in regulated streams before 1996, they were not recorded in the present study upstream or downstream of weirs despite recent flow releases. Crayfish (Geocharax sp. nov. 1) remained in the regulated streams throughout the study, but did not become more abundant in response to flow releases. In contrast, native fish (Gadopsis marmoratus, Galaxias oliros, Galaxias maculatus) and crayfish remained present in unregulated streams, despite prolonged drought conditions during 2006-2010, and the assemblages of each of these streams remained essentially unchanged over the 30 year period. Flow release volumes may have been too small or have operated for an insufficient time to allow fish to recolonise regulated streams. Barriers to dispersal may also be preventing recolonisation. Indefinite continuation of annual flow releases, that prevent the unnatural cessation of flow caused by weirs, may eventually facilitate upstream movement of fish and crayfish in regulated channels; but other human-made dispersal barriers downstream need to be identified and ameliorated, to allow native fish to fulfil their life cycles in these headwater streams.

Stream quantiles via maximal entropy histograms

We address the problem of estimating the running quantile of a data stream when the memory for storing observations is limited.We (i) highlight the limitations of approaches previously described in the literature which make them unsuitable for non-stationary streams, (ii) describe a novel principle for the utilization of the available storage space, and (iii) introduce two novel algorithms which exploit the proposed principle. Experiments on three large realworld data sets demonstrate that the proposed methods vastly outperform the existing alternatives.

A general communication cost optimization framework for big data stream processing in geo-distributed data centers

With the explosion of big data, processing large numbers of continuous data streams, i.e., big data stream processing (BDSP), has become a crucial requirement for many scientific and industrial applications in recent years. By offering a pool of computation, communication and storage resources, public clouds, like Amazon's EC2, are undoubtedly the most efficient platforms to meet the ever-growing needs of BDSP. Public cloud service providers usually operate a number of geo-distributed datacenters across the globe. Different datacenter pairs are with different inter-datacenter network costs charged by Internet Service Providers (ISPs). While, inter-datacenter traffic in BDSP constitutes a large portion of a cloud provider's traffic demand over the Internet and incurs substantial communication cost, which may even become the dominant operational expenditure factor. As the datacenter resources are provided in a virtualized way, the virtual machines (VMs) for stream processing tasks can be freely deployed onto any datacenters, provided that the Service Level Agreement (SLA, e.g., quality-of-information) is obeyed. This raises the opportunity, but also a challenge, to explore the inter-datacenter network cost diversities to optimize both VM placement and load balancing towards network cost minimization with guaranteed SLA. In this paper, we first propose a general modeling framework that describes all representative inter-task relationship semantics in BDSP. Based on our novel framework, we then formulate the communication cost minimization problem for BDSP into a mixed-integer linear programming (MILP) problem and prove it to be NP-hard. We then propose a computation-efficient solution based on MILP. The high efficiency of our proposal is validated by extensive simulation based studies.

A data as a product model for future consumption of big stream data in clouds

Data is becoming the world’s new natural resourceand big data use grows quickly. The trend of computingtechnology is that everything is merged into the Internet and‘big data’ are integrated to comprise completeinformation for collective intelligence. With the increasingsize of big data, refining big data themselves to reduce data sizewhile keeping critical data (or useful information) is a newapproach direction. In this paper, we provide a novel dataconsumption model, which separates the consumption of datafrom the raw data, and thus enable cloud computing for bigdata applications. We define a new Data-as-a-Product (DaaP)concept; a data product is a small sized summary of theoriginal data and can directly answer users’ queries. Thus, weseparate the mining of big data into two classes of processingmodules: the refine modules to change raw big data into smallsizeddata products, and application-oriented mining modulesto discover desired knowledge further for applications fromwell-defined data products. Our practices of mining big streamdata, including medical sensor stream data, streams of textdata and trajectory data, demonstrated the efficiency andprecision of our DaaP model for answering users’ queries