Scaling up data mining techniques to large datasets using parallel and distributed processing

Advances in hardware and software technology enable us to collect, store and distribute large quantities of data on a very large scale. Automatically discovering and extracting hidden knowledge in the form of patterns from these large data volumes is known as data mining. Data mining technology is not only a part of business intelligence, but is also used in many other application areas such as research, marketing and financial analytics. For example medical scientists can use patterns extracted from historic patient data in order to determine if a new patient is likely to respond positively to a particular treatment or not; marketing analysts can use extracted patterns from customer data for future advertisement campaigns; finance experts have an interest in patterns that forecast the development of certain stock market shares for investment recommendations. However, extracting knowledge in the form of patterns from massive data volumes imposes a number of computational challenges in terms of processing time, memory, bandwidth and power consumption. These challenges have led to the development of parallel and distributed data analysis approaches and the utilisation of Grid and Cloud computing. This chapter gives an overview of parallel and distributed computing approaches and how they can be used to scale up data mining to large datasets.

How well can we simulate past climates? Evaluating the models using global palaeoenvironmental datasets.

Global syntheses of palaeoenvironmental data are required to test climate models under conditions different from the present. Data sets for this purpose contain data from spatially extensive networks of sites. The data are either directly comparable to model output or readily interpretable in terms of modelled climate variables. Data sets must contain sufficient documentation to distinguish between raw (primary) and interpreted (secondary, tertiary) data, to evaluate the assumptions involved in interpretation of the data, to exercise quality control, and to select data appropriate for specific goals. Four data bases for the Late Quaternary, documenting changes in lake levels since 30 kyr BP (the Global Lake Status Data Base), vegetation distribution at 18 kyr and 6 kyr BP (BIOME 6000), aeolian accumulation rates during the last glacial-interglacial cycle (DIRTMAP), and tropical terrestrial climates at the Last Glacial Maximum (the LGM Tropical Terrestrial Data Synthesis) are summarised. Each has been used to evaluate simulations of Last Glacial Maximum (LGM: 21 calendar kyr BP) and/or mid-Holocene (6 cal. kyr BP) environments. Comparisons have demonstrated that changes in radiative forcing and orography due to orbital and ice-sheet variations explain the first-order, broad-scale (in space and time) features of global climate change since the LGM. However, atmospheric models forced by 6 cal. kyr BP orbital changes with unchanged surface conditions fail to capture quantitative aspects of the observed climate, including the greatly increased magnitude and northward shift of the African monsoon during the early to mid-Holocene. Similarly, comparisons with palaeoenvironmental datasets show that atmospheric models have underestimated the magnitude of cooling and drying of much of the land surface at the LGM. The inclusion of feedbacks due to changes in ocean- and land-surface conditions at both times, and atmospheric dust loading at the LGM, appears to be required in order to produce a better simulation of these past climates. The development of Earth system models incorporating the dynamic interactions among ocean, atmosphere, and vegetation is therefore mandated by Quaternary science results as well as climatological principles. For greatest scientific benefit, this development must be paralleled by continued advances in palaeodata analysis and synthesis, which in turn will help to define questions that call for new focused data collection efforts.

Seasonal intercomparison of observational rainfall datasets over India during the southwest monsoon season

The Indian monsoon is an important component of Earth's climate system, accurate forecasting of its mean rainfall being essential for regional food and water security. Accurate measurement of the rainfall is essential for various water-related applications, the evaluation of numerical models and detection and attribution of trends, but a variety of different gridded rainfall datasets are available for these purposes. In this study, six gridded rainfall datasets are compared against the India Meteorological Department (IMD) gridded rainfall dataset, chosen as the most representative of the observed system due to its high gauge density. The datasets comprise those based solely on rain gauge observations and those merging rain gauge data with satellite-derived products. Various skill scores and subjective comparisons are carried out for the Indian region during the south-west monsoon season (June to September). Relative biases and skill metrics are documented at all-India and sub-regional scales. In the gauge-based (land-only) category, Asian Precipitation-Highly-Resolved Observational Data Integration Towards Evaluation of water resources (APHRODITE) and Global Precipitation Climatology Center (GPCC) datasets perform better relative to the others in terms of a variety of skill metrics. In the merged category, the Global Precipitation Climatology Project (GPCP) dataset is shown to perform better than the Climate Prediction Center Merged Analysis of Precipitation (CMAP) for the Indian monsoon in terms of various metrics, when compared with the IMD gridded data. Most of the datasets have difficulty in representing rainfall over orographic regions including the Western Ghats mountains, in north-east India and the Himalayan foothills. The wide range of skill scores seen among the datasets and even the change of sign of bias found in some years are causes of concern. This uncertainty between datasets is largest in north-east India. These results will help those studying the Indian monsoon region to select an appropriate dataset depending on their application and focus of research.

Sea surface temperature datasets for climate applications from Phase 1 of the European Space Agency Climate Change Initiative (SST CCI)

Sea surface temperature (SST) datasets have been generated from satellite observations for the period 1991–2010, intended for use in climate science applications. Attributes of the datasets specifically relevant to climate applications are: first, independence from in situ observations; second, effort to ensure homogeneity and stability through the time-series; third, context-specific uncertainty estimates attached to each SST value; and, fourth, provision of estimates of both skin SST (the fundamental measure- ment, relevant to air-sea fluxes) and SST at standard depth and local time (partly model mediated, enabling comparison with his- torical in situ datasets). These attributes in part reflect requirements solicited from climate data users prior to and during the project. Datasets consisting of SSTs on satellite swaths are derived from the Along-Track Scanning Radiometers (ATSRs) and Advanced Very High Resolution Radiometers (AVHRRs). These are then used as sole SST inputs to a daily, spatially complete, analysis SST product, with a latitude-longitude resolution of 0.05°C and good discrimination of ocean surface thermal features. A product user guide is available, linking to reports describing the datasets’ algorithmic basis, validation results, format, uncer- tainty information and experimental use in trial climate applications. Future versions of the datasets will span at least 1982–2015, better addressing the need in many climate applications for stable records of global SST that are at least 30 years in length.

The Maunder minimum (1645-1715) was indeed a grand minimum: a reassessment of multiple datasets

Aims. Although the time of the Maunder minimum (1645–1715) is widely known as a period of extremely low solar activity, it is still being debated whether solar activity during that period might have been moderate or even higher than the current solar cycle (number 24). We have revisited all existing evidence and datasets, both direct and indirect, to assess the level of solar activity during the Maunder minimum. Methods. We discuss the East Asian naked-eye sunspot observations, the telescopic solar observations, the fraction of sunspot active days, the latitudinal extent of sunspot positions, auroral sightings at high latitudes, cosmogenic radionuclide data as well as solar eclipse observations for that period. We also consider peculiar features of the Sun (very strong hemispheric asymmetry of the sunspot location, unusual differential rotation and the lack of the K-corona) that imply a special mode of solar activity during the Maunder minimum. Results. The level of solar activity during the Maunder minimum is reassessed on the basis of all available datasets. Conclusions. We conclude that solar activity was indeed at an exceptionally low level during the Maunder minimum. Although the exact level is still unclear, it was definitely lower than during the Dalton minimum of around 1800 and significantly below that of the current solar cycle #24. Claims of a moderate-to-high level of solar activity during the Maunder minimum are rejected with a high confidence level.

PETS 2015: datasets and challenge

This paper presents the two datasets (ARENA and P5) and the challenge that form a part of the PETS 2015 workshop. The datasets consist of scenarios recorded by us- ing multiple visual and thermal sensors. The scenarios in ARENA dataset involve different staged activities around a parked vehicle in a parking lot in UK and those in P5 dataset involve different staged activities around the perimeter of a nuclear power plant in Sweden. The scenarios of each dataset are grouped into ‘Normal’, ‘Warning’ and ‘Alarm’ categories. The Challenge specifically includes tasks that account for different steps in a video understanding system: Low-Level Video Analysis (object detection and tracking), Mid-Level Video Analysis (‘atomic’ event detection) and High-Level Video Analysis (‘complex’ event detection). The evaluation methodology used for the Challenge includes well-established measures.

Tracking performance evaluation on PETS 2015 Challenge datasets

This paper presents a quantitative evaluation of a tracking system on PETS 2015 Challenge datasets using well-established performance measures. Using the existing tools, the tracking system implements an end-to-end pipeline that include object detection, tracking and post- processing stages. The evaluation results are presented on the provided sequences of both ARENA and P5 datasets of PETS 2015 Challenge. The results show an encouraging performance of the tracker in terms of accuracy but a greater tendency of being prone to cardinality error and ID changes on both datasets. Moreover, the analysis show a better performance of the tracker on visible imagery than on thermal imagery.

Datasets related to in-land water for limnology and remote sensing applications: distance-to-land, distance-to-water, water-body identifier and lake-centre co-ordinates

Datasets containing information to locate and identify water bodies have been generated from data locating static-water-bodies with resolution of about 300 m (1/360 deg) recently released by the Land Cover Climate Change Initiative (LC CCI) of the European Space Agency. The LC CCI water-bodies dataset has been obtained from multi-temporal metrics based on time series of the backscattered intensity recorded by ASAR on Envisat between 2005 and 2010. The new derived datasets provide coherently: distance to land, distance to water, water-body identifiers and lake-centre locations. The water-body identifier dataset locates the water bodies assigning the identifiers of the Global Lakes and Wetlands Database (GLWD), and lake centres are defined for in-land waters for which GLWD IDs were determined. The new datasets therefore link recent lake/reservoir/wetlands extent to the GLWD, together with a set of coordinates which locates unambiguously the water bodies in the database. Information on distance-to-land for each water cell and the distance-to-water for each land cell has many potential applications in remote sensing, where the applicability of geophysical retrieval algorithms may be affected by the presence of water or land within a satellite field of view (image pixel). During the generation and validation of the datasets some limitations of the GLWD database and of the LC CCI water-bodies mask have been found. Some examples of the inaccuracies/limitations are presented and discussed. Temporal change in water-body extent is common. Future versions of the LC CCI dataset are planned to represent temporal variation, and this will permit these derived datasets to be updated.

2014 World Cup live blog: political protestor invades field at US vs Belgium

Retirado do Vice News.

Potencialidades do uso do blog em educação.

This research is an interpretative analysis of the type participatory research, developed in a qualitative and quantitative use of the blog as a support to a specific discipline, in order to identify the potential evident from its use. The report discusses the changes that have occurred in contemporary society, relating to the development of information technologies and communication (ITC s), presents a brief review of the historical background of the Internet and its use as an aid to education, emphasizing some environments inserted media the Internet, focusing on the main blog - its concept, origin and categorization, and analyzes the concepts of using the blog from the dialogues with teachers and students of pedagogy course at the Federal University of Rio Grande do Norte. Started from the assumption that the use of technological resources, such as blogging, with strictly educational purposes, can extend the knowledge beyond the walls of the classroom, thus creating a dialogic and interactive environment. Using data collected through interviews, questionnaires and observation, we seek to understand the object of study as a supportive environment for the teaching of a subject, raising some theoretical and methodological questions about its application to educational practice, and possible contributions to the construction of knowledge. The results indicate that there are several capabilities that make the blog a space conducive to teaching and learning process, and relates the concepts of the study participants about their use, highlighting the most important places to be solved, so that teachers and students to take ownership of knowledge necessary for capacity building required by the contemporary social context, due to the advancement of science and technology.

O gênero blog: ação social e multimodalidade.

Studies about discursive genres peak at a necessity to understand how these genres work in a society that is more and more submissive to the technology of informatics such as ours. In acknowledging that the virtual context of the Internet provides the manifestation and development of new genres of the discourse, we perceived that the online journal, or blog, as commonly known, is responsible for a variety of linguistic phenomena that would normally take severaI years to consolidate. Since its appearance in 1997 blogs rush as a virtual version of the personal diary and in a short time due to communicative demands suffers several changes, making new categories of blog genres emerge. Facing such phenomena, this work intends at first to characterize blogs as a genre that exercises a social action, evidencing its formal, structural and pragmatic characteristics from the notion of recurrence and rhetoric in a discursive-semiotic perspective. The methodological postulates adopted by this research are considered of qualitative basis in the sense that they are not restricted to looking at the discursive events as a product, but mainly they take into consideration a group of situational, cultural and ideologic factors that are present in the constitution of genre.

A linguagem do blog escolar em um trabalho com multiletramentos: compartilhando sentidos

This research, part of Applied Linguistics field, aims to analyze the language of a school blog, developed with the participation of students, as a work based on the conception of multiliteracies, focusing on the construction of different meanings. The research is carried on from the building and maintenance of a school blog, the Ieceblog, with students of Ensino Fundamental II, since 2008, in a private school in Natal-RN. The investigation of the language produced on a school blog is justified due to the interactive conceptions of writing and reading on the virtual environment. Given the fact that new technologies are a reality in the schools opened to the practices of multiliteracies, it is assumed that text, image, video, audio, non-graphic signs and hypertext intensifies the produced interaction, in which the students become real authors. In this perspective, some voices belonging to the statements that are formed through the posts and comments chosen to the analyses and reflection on the blog space as locus of productions of senses inserted in the school and the world environment, as well as for the identification of the language resources used to intensify the senses that emerge from it. From the view of dialogism conceptualized by Bakhtin Circle, the qualitative interpretive-research deepens the experience of a school blog focusing on digital language in line with the vision of digital literacy. From the blog posts, a corpus that promote the exposure of different manifestations of language in the design of digital multiliteracies is elected. Thereby, the method used was the dialogical analysis of speech based on Bakhtin s studies and the Circle. The corpus was taken from the blog s posts in order to point up the different language manifestations in the following categories: (i) mood reinforced by the mockery, (ii) search for compliance with school sphere, (iii) conflicting social values and consistent complicity between sense and verbal imagery, and finally (iv) social practices that take place from and through the discursive genre. The study points to the tension between the active voices in several directions, revealing the distorted unit of posts which, under the analytical observation raises multiple meanings in a responsive manner. The analysis of the dialogue interaction in which intersperses the digital one becomes more apparent that the multiliteracies events that are mediated by language in addition to structure of the language and makes us rethink the students