SCLOPE: an algorithm for clustering data streams of categorical attributes

Clustering is a difficult problem especially when we consider the task in the context of a data stream of categorical attributes. In this paper, we propose SCLOPE, a novel algorithm based on CLOPErsquos intuitive observation about cluster histograms. Unlike CLOPE however, our algo- rithm is very fast and operates within the constraints of a data stream environment. In particular, we designed SCLOPE according to the recent CluStream framework. Our evaluation of SCLOPE shows very promising results. It consistently outperforms CLOPE in speed and scalability tests on our data sets while maintaining high cluster purity; it also supports cluster analysis that other algorithms in its class do not.

Hierarchial patterns of invertebrate assemblage structure in stony upland streams change with time and flow permanence

1. Studies in several parts of the world have examined variation in univariate descriptors of macroinvertebrate assemblage structure in perennially flowing stony streams across hierarchies of spatial scale using nested analyses of variance. However, few have investigated whether this spatial variation changes with time or whether these results are representative of habitats other than riffles or of other stream types, such as intermittently flowing streams.

2. We describe patterns in taxon richness and abundance from two sets of samples from stony streams in the Otway Range and the Grampians Range, Victoria, Australia, collected using hierarchical designs. Sampling of riffles was repeated in the Otways, to determine whether spatial patterns were consistent among times. In the Grampians, spatial patterns were compared between intermittent and perennially flowing streams (stream type) by sampling pools.

3. In the Otways streams, most variation in the dependent variables occurred between sample units. Patterns of variation among the other scales (streams, segments, riffles, groups of stones) were not consistent between sampling times, suggesting that they may have little ecological significance.

4. In the Grampians streams, variation in macroinvertebrate taxon richness and abundance differed significantly between replicate streams within each stream type but not between stream types or pools. The largest source of variation in taxon richness was stream type. Little variation occurred among sample units.

5. The pattern of most variation occurring among sample units is robust both to differences in the method of sampling and different dependent variables among studies and increasingly appears to be a property of riffles in stony, perennial upland streams. High variation among sample units (residual variation) limits the explanatory power of linear models and therefore, where samples are from a single sampling time, small but significant components of variation are unlikely to represent features of assemblage structure that will be stable over time.

ó-SCLOPE : clustering categorical streams using attribute selection

Clustering is a difficult problem especially when we consider the task in the context of a data stream of categorical attributes. In this paper, we propose σ-SCLOPE, a novel algorithm based on SCLOPE’s intuitive observation about cluster histograms. Unlike SCLOPE however, our algorithm consumes less memory per window and has a better clustering runtime for the same data stream in a given window. This positions σ-SCLOPE as a more attractive option over SCLOPE if a minor lost of clustering accuracy is insignificant in the application.

Effect of fire on Benthic Algal Assemblage structure and recolonisation in intermittent streams

Dry biofilm on rocks and other substrata forms an important drought refuge for benthic algae in intermittent streams following the cessation of flow. This dry biofilm is potentially susceptible to disturbance from bushfires, including direct burning and/or scorching and damage from radiant heat, particularly when streams are dry. Therefore, damage to dry biofilms by fire has the potential to influence algal recolonization and assemblage structure in intermittent streams following commencement of flow. The influence of fire on benthic algal assemblages and recolonization was examined in intermittent streams of the Grampians National Park, Victoria, Australia, using a field survey and manipulative field experiment. The field survey compared assemblages in two intermittent streams within a recently burnt area (within 5 months of the fire) with two intermittent streams within an unburnt area. The two burnt streams were still flowing during the fire so most biofilms were not likely to be directly exposed to flames. Considerable site-to-site and stream-to-stream variation was detected during the field survey, which may have obscured potential differences attributable to indirect effects of the fire. The manipulative field experiment occurred in two intermittent streams and consisted of five treatments chosen to replicate various characteristics of bushfires that may influence dry biofilms: dry biofilm exposed directly to fire; dry biofilm exposed to radiant heat; dry biofilm exposed to ash; and two procedural controls. After exposure to the different treatments, rocks were replaced in the streams and algae were sampled 7 days after flow commenced. Differences occurred across treatments, but treatment differences were inconsistent across the two streams. For example, direct exposure to fire reduced the abundance of recolonizing algae and altered assemblage structure in both streams, while radiant heat had an effect on assemblage structure in one stream only. The manipulative field experiment is likely to have represented the intensity of a small bushfire only. Nonetheless, significant differences across treatments were detected, so these experimental results suggest that fire can damage dry biofilms, and hence, influence algal recolonization and assemblage structure in intermittent streams.

Accounting for local government boundary changes in South Australia : valuing assets and liabilities or valuing revenue streams

This paper examines the methodologies adopted in the transfer of assets, liabilities, revenues and expenses resulting from boundary changes associated with municipal amalgamations in South Australia during the late 1990s. It investigates the methods employed for apportioning these financial elements, valuations used and financial settlements required. Significant transfers occurred in only three cases. In two cases, councils used simple, pragmatic methods to apportion assets and liabilities, similar to those used previously in Victoria. In the third case a transfer price was calculated based on the net present value of revenues. This method is quite different from previous methods examined and is appropriate where one council will make significant future gains at another council's loss because of net revenue transfers.

Rapid determination of papaver somniferum alkaloids in process streams using monolithic column high-performance liquid chromatography with chemiluminescence detection

We have combined high-performance liquid chromatography (HPLC) separations using a monolithic column with acidic potassium permanganate and tris(2,2′-bipyridyl)ruthenium(II) chemiluminescence detection in a rapid and highly sensitive method to monitor the process of extracting opiate alkaloids from Papaver somniferum. Due to the high flow rates allowed with the monolithic column and the inherent selectivity of the chemiluminescence reactions, the four predominant alkaloids – morphine, codeine, oripavine and thebaine – were determined in less than 2 min. The results obtained with numerous process samples compared favourable with those of the standard HPLC methodology. Limits of detection were 1 × 10⁻¹⁰ M, 5 × 10⁻¹⁰ M, 5 × 10⁻¹⁰ M and 1 × 10⁻⁹ M, for morphine, codeine, oripavine and thebaine, respectively.

Discovering frequent sets from data streams with CPU constraint

Data streams are usually generated in an online fashion characterized by huge volume, rapid unpredictable rates, and fast changing data characteristics. It has been hence recognized that mining over streaming data requires the problem of limited computational resources to be adequately addressed. Since the arrival rate of data streams can significantly increase and exceed the CPU capacity, the machinery must adapt to this change to guarantee the timeliness of the results. We present an online algorithm to approximate a set of frequent patterns from a sliding window over the underlying data stream - given apriori CPU capacity. The algorithm automatically detects overload situations and can adaptively shed unprocessed data to guarantee the timely results. We theoretically prove, using probabilistic and deterministic techniques, that the error on the output results is bounded within a pre-specified threshold. The empirical results on various datasets also confirmed the feasiblity of our proposal.

Adaptive load shedding for mining frequent patterns from data streams

Most algorithms that focus on discovering frequent patterns from data streams assumed that the machinery is capable of managing all the incoming transactions without any delay; or without the need to drop transactions. However, this assumption is often impractical due to the inherent characteristics of data stream environments. Especially under high load conditions, there is often a shortage of system resources to process the incoming transactions. This causes unwanted latencies that in turn, affects the applicability of the data mining models produced – which often has a small window of opportunity. We propose a load shedding algorithm to address this issue. The algorithm adaptively detects overload situations and drops transactions from data streams using a probabilistic model. We tested our algorithm on both synthetic and real-life datasets to verify the feasibility of our algorithm.