Algorithms and tools of big data: A bibliographic review

66 p.

Detecting phase transitions in community structures using big data analysis of the engineering education research landscape: a European perspective

Comunicação apresentada na 44th SEFI Conference, 12-­15 September 2016, Tampere, Finland

The Amplifier - v. 16, no. 2

In this issue...Who's Who, Vietnam War, Student Union Building, Cultural Programs, Library, Loretta Peck, Radio 1370, Film Festival, Elizabeth Lochrie, Big Sky Techettes

The Amplifier - v. 17, no. 2

In this issue...Big Sky Resort, campus parking, Biology House, Cheerleaders, Silver bow County, Lewis and Clark, Yellowstone, Shell Oil Company, George Harrison

The Amplifier - v. 17, no. 10

In this issue...Theta Tau, Junior Prom, Larry Harkins, Montana Tech History Club, Big Horn Sheep, Coach Riley, Mineral Engineering, Moon Talk, Golf Team

Impact of a Changing Climate on Fine Particulate Concentrations in Butte, MT

A model was developed to assess the potential change in PM2.5 concentrations in Butte, Montana over the course of the 21st century as the result of climate change and changes in emissions. The EPA AERMOD regulatory model was run using NARCCAP climate data for the years of 2040, 2050, 2060 and 2070, and the results were compared to the NAAQS to determine if there is the potential for future impacts to human health. This model predicted an average annual concentration of 15.84 µg/m3 in the year 2050, which would exceed the primary NAAQS of 12 µg/m3 and is a large increase over the average concentration from 2010 – 2012 of 10.52 µg/m3. The effectiveness of a wood stove change out program was also evaluated to determine its efficacy, and modeled results predicted that by changing out 100% of inefficient stoves with an EPA approved model, concentrations could be reduced below the NAAQS.

Big Manistee River Tributaries as Potential Arctic Grayling Habitat

The Big Manistee River was one of the most well known Michigan rivers to historically support a population of Arctic grayling (Thymallus arctics). Overfishing, competition with introduced fish, and habitat loss due to logging are believed to have caused their decline and ultimate extirpation from the Big Manistee River around 1900 and from the State of Michigan by 1936. Grayling are a species of great cultural importance to Little River Band of Ottawa Indian tribal heritage and although past attempts to reintroduce Arctic grayling have been unsuccessful, a continued interest in their return led to the assessment of environmental conditions of tributaries within a 21 kilometer section of the Big Manistee River to determine if suitable habitat exists. Although data describing historical conditions in the Big Manistee River is limited, we reviewed the literature to determine abiotic conditions prior to Arctic grayling disappearance and the habitat conditions in rivers in western and northwestern North America where they currently exist. We assessed abiotic habitat metrics from 23 sites distributed across 8 tributaries within the Manistee River watershed. Data collected included basic water parameters, streambed substrate composition, channel profile and areal measurements of channel geomorphic unit, and stream velocity and discharge measurements. These environmental condition values were compared to literature values, habitat suitability thresholds, and current conditions of rivers with Arctic grayling populations to assess the feasibility of the abiotic habitat in Big Manistee River tributaries to support Arctic grayling. Although the historic grayling habitat in the region was disturbed during the era of major logging around the turn of the 20th century, our results indicate that some important abiotic conditions within Big Manistee River tributaries are within the range of conditions that support current and past populations of Arctic grayling. Seven tributaries contained between 20-30% pools by area, used by grayling for refuge. All but two tributaries were composed primarily of pebbles, with the remaining two dominated by fine substrates (sand, silt, clay). Basic water parameters and channel depth were within the ranges of those found for populations of Arctic grayling persisting in Montana, Alaska, and Canada for all tributaries. Based on the metrics analyzed in this study, suitable abiotic grayling habitat does exist in Big Manistee River tributaries.

Big-data e turismo Estrazione ed analisi dei dati dalla piattaforma Expedia

La mia tesi si occupa di trattare come, attraverso questo nuovo prodotto dell’informatica chiamato big data, si possano ottenere informazioni e fare previsioni sull’andamento del turismo.

Implementazione e benchmarking dell'algoritmo QDANet PRO per l'analisi di big data genomici

Dato il recente avvento delle tecnologie NGS, in grado di sequenziare interi genomi umani in tempi e costi ridotti, la capacità di estrarre informazioni dai dati ha un ruolo fondamentale per lo sviluppo della ricerca. Attualmente i problemi computazionali connessi a tali analisi rientrano nel topic dei Big Data, con databases contenenti svariati tipi di dati sperimentali di dimensione sempre più ampia. Questo lavoro di tesi si occupa dell'implementazione e del benchmarking dell'algoritmo QDANet PRO, sviluppato dal gruppo di Biofisica dell'Università di Bologna: il metodo consente l'elaborazione di dati ad alta dimensionalità per l'estrazione di una Signature a bassa dimensionalità di features con un'elevata performance di classificazione, mediante una pipeline d'analisi che comprende algoritmi di dimensionality reduction. Il metodo è generalizzabile anche all'analisi di dati non biologici, ma caratterizzati comunque da un elevato volume e complessità, fattori tipici dei Big Data. L'algoritmo QDANet PRO, valutando la performance di tutte le possibili coppie di features, ne stima il potere discriminante utilizzando un Naive Bayes Quadratic Classifier per poi determinarne il ranking. Una volta selezionata una soglia di performance, viene costruito un network delle features, da cui vengono determinate le componenti connesse. Ogni sottografo viene analizzato separatamente e ridotto mediante metodi basati sulla teoria dei networks fino all'estrapolazione della Signature finale. Il metodo, già precedentemente testato su alcuni datasets disponibili al gruppo di ricerca con riscontri positivi, è stato messo a confronto con i risultati ottenuti su databases omici disponibili in letteratura, i quali costituiscono un riferimento nel settore, e con algoritmi già esistenti che svolgono simili compiti. Per la riduzione dei tempi computazionali l'algoritmo è stato implementato in linguaggio C++ su HPC, con la parallelizzazione mediante librerie OpenMP delle parti più critiche.

Analyzing Tidal Fluctuations in the Big Pine Key Freshwater Lens with Time-Lapse Resistivity

The tidal influence on the Big Pine Key saltwater/freshwater interface was analyzed using time-lapse electrical resistivity imaging and shallow well measurements. The transition zone at the saltwater/freshwater interface was measured over part of a tidal cycle along three profiles. The resistivity was converted to salinity by deriving a formation factor for the Miami Oolite. A SEAWAT model was created to attempt to recreate the field measurements and test previously established hydrogeologic parameters. The results imply that the tide only affects the groundwater within 20 to 30 m of the coast. The effect is small and caused by flooding from the high tide. The low relief of the island means this effect is very sensitive to small changes in the magnitude. The SEAWAT model proved to be insufficient in modeling this effect. The study suggests that the extent of flooding is the largest influence on the salinity of the groundwater.

Exploring Hidden Coherent Feature Groups and Temporal Semantics for Multimedia Big Data Analysis

Thanks to the advanced technologies and social networks that allow the data to be widely shared among the Internet, there is an explosion of pervasive multimedia data, generating high demands of multimedia services and applications in various areas for people to easily access and manage multimedia data. Towards such demands, multimedia big data analysis has become an emerging hot topic in both industry and academia, which ranges from basic infrastructure, management, search, and mining to security, privacy, and applications. Within the scope of this dissertation, a multimedia big data analysis framework is proposed for semantic information management and retrieval with a focus on rare event detection in videos. The proposed framework is able to explore hidden semantic feature groups in multimedia data and incorporate temporal semantics, especially for video event detection. First, a hierarchical semantic data representation is presented to alleviate the semantic gap issue, and the Hidden Coherent Feature Group (HCFG) analysis method is proposed to capture the correlation between features and separate the original feature set into semantic groups, seamlessly integrating multimedia data in multiple modalities. Next, an Importance Factor based Temporal Multiple Correspondence Analysis (i.e., IF-TMCA) approach is presented for effective event detection. Specifically, the HCFG algorithm is integrated with the Hierarchical Information Gain Analysis (HIGA) method to generate the Importance Factor (IF) for producing the initial detection results. Then, the TMCA algorithm is proposed to efficiently incorporate temporal semantics for re-ranking and improving the final performance. At last, a sampling-based ensemble learning mechanism is applied to further accommodate the imbalanced datasets. In addition to the multimedia semantic representation and class imbalance problems, lack of organization is another critical issue for multimedia big data analysis. In this framework, an affinity propagation-based summarization method is also proposed to transform the unorganized data into a better structure with clean and well-organized information. The whole framework has been thoroughly evaluated across multiple domains, such as soccer goal event detection and disaster information management.

Semantics, knowledge and grids on big data

Semantics, knowledge and Grids represent three spaces where people interact, understand, learn and create. Grids represent the advanced cyber-infrastructures and evolution. Big data influence the evolution of semantics, knowledge and Grids. Exploring semantics, knowledge and Grids on big data helps accelerate the shift of scientific paradigm, the fourth industrial revolution, and the transformational innovation of technologies.

Aplicación de técnicas de Big Data Science para la gestión de crisis

A pesar de la existencia de una multitud de investigaciones sobre el análisis de sentimiento, existen pocos trabajos que traten el tema de su implantación práctica y real y su integración con la inteligencia de negocio y big data de tal forma que dichos análisis de sentimiento estén incorporados en una arquitectura (que soporte todo el proceso desde la obtención de datos hasta su explotación con las herramientas de BI) aplicada a la gestión de la crisis. Se busca, por medio de este trabajo, investigar cómo se pueden unir los mundos de análisis (de sentimiento y crisis) y de la tecnología (todo lo relacionado con la inteligencia de negocios, minería de datos y Big Data), y crear una solución de Inteligencia de Negocios que comprenda la minería de datos y el análisis de sentimiento (basados en grandes volúmenes de datos), y que ayude a empresas y/o gobiernos con la gestión de crisis. El autor se ha puesto a estudiar formas de trabajar con grandes volúmenes de datos, lo que se conoce actualmente como Big Data Science, o la ciencia de los datos aplicada a grandes volúmenes de datos (Big Data), y unir esta tecnología con el análisis de sentimiento relacionado a una situación real (en este trabajo la situación elegida fue la del proceso de impechment de la presidenta de Brasil, Dilma Rousseff). En esta unión se han utilizado técnicas de inteligencia de negocios para la creación de cuadros de mandos, rutinas de ETC (Extracción, Transformación y Carga) de los datos así como también técnicas de minería de textos y análisis de sentimiento. El trabajo ha sido desarrollado en distintas partes y con distintas fuentes de datos (datasets) debido a las distintas pruebas de tecnología a lo largo del proyecto. Uno de los datasets más importantes del proyecto son los tweets recogidos entre los meses de diciembre de 2015 y enero de 2016. Los mensajes recogidos contenían la palabra "Dilma" en el mensaje. Todos los twittees fueron recogidos con la API de Streaming del Twitter. Es muy importante entender que lo que se publica en la red social Twitter no se puede manipular y representa la opinión de la persona o entidad que publica el mensaje. Por esto se puede decir que hacer el proceso de minería de datos con los datos del Twitter puede ser muy eficiente y verídico. En 3 de diciembre de 2015 se aceptó la petición de apertura del proceso del impechment del presidente de Brasil, Dilma Rousseff. La petición fue aceptada por el presidente de la Cámara de los Diputados, el diputado Sr. Eduardo Cunha (PMDBRJ), y de este modo se creó una expectativa sobre el sentimiento de la población y el futuro de Brasil. También se ha recogido datos de las búsquedas en Google referentes a la palabra Dilma; basado en estos datos, el objetivo es llegar a un análisis global de sentimiento (no solo basado en los twittees recogidos). Utilizando apenas dos fuentes (Twitter y búsquedas de Google) han sido extraídos muchísimos datos, pero hay muchas otras fuentes donde es posible obtener informaciones con respecto de las opiniones de las personas acerca de un tema en particular. Así, una herramienta que pueda recoger, extraer y almacenar tantos datos e ilustrar las informaciones de una manera eficaz que ayude y soporte una toma de decisión, contribuye para la gestión de crisis.

Big data, smart homes and ambient assisted living

Objectives: To discuss how current research in the area of smart homes and ambient assisted living will be influenced by the use of big data. Methods: A scoping review of literature published in scientific journals and conference proceedings was performed, focusing on smart homes, ambient assisted living and big data over the years 2011-2014. Results: The health and social care market has lagged behind other markets when it comes to the introduction of innovative IT solutions and the market faces a number of challenges as the use of big data will increase. First, there is a need for a sustainable and trustful information chain where the needed information can be transferred from all producers to all consumers in a structured way. Second, there is a need for big data strategies and policies to manage the new situation where information is handled and transferred independently of the place of the expertise. Finally, there is a possibility to develop new and innovative business models for a market that supports cloud computing, social media, crowdsourcing etc. Conclusions: The interdisciplinary area of big data, smart homes and ambient assisted living is no longer only of interest for IT developers, it is also of interest for decision makers as customers make more informed choices among today's services. In the future it will be of importance to make information usable for managers and improve decision making, tailor smart home services based on big data, develop new business models, increase competition and identify policies to ensure privacy, security and liability.