Big Data : análisis y estudio de la plataforma Hadoop

Desde el inicio de los tiempos el ser humano ha tenido la necesidad de comprender y analizar todo lo que nos rodea, para ello se ha valido de diferentes herramientas como las pinturas rupestres, la biblioteca de Alejandría, bastas colecciones de libros y actualmente una enorme cantidad de información informatizada. Todo esto siempre se ha almacenado, según la tecnología de la época lo permitía, con la esperanza de que fuera útil mediante su consulta y análisis. En la actualidad continúa ocurriendo lo mismo. Hasta hace unos años se ha realizado el análisis de información manualmente o mediante bases de datos relacionales. Ahora ha llegado el momento de una nueva tecnología, Big Data, con la cual se puede realizar el análisis de extensas cantidades de datos de todo tipo en tiempos relativamente pequeños. A lo largo de este libro, se estudiarán las características y ventajas de Big Data, además de realizar un estudio de la plataforma Hadoop. Esta es una plataforma basada en Java y puede realizar el análisis de grandes cantidades de datos de diferentes formatos y procedencias. Durante la lectura de estas páginas se irá dotando al lector de los conocimientos previos necesarios para su mejor comprensión, así como de ubicarle temporalmente en el desarrollo de este concepto, de su uso, las previsiones y la evolución y desarrollo que se prevé tenga en los próximos años. ABSTRACT. Since the beginning of time, human being was in need of understanding and analyzing everything around him. In order to do that, he used different media as cave paintings, Alexandria library, big amount of book collections and nowadays massive amount of computerized information. All this information was stored, depending on the age and technology capability, with the expectation of being useful though it consulting and analysis. Nowadays they keep doing the same. In the last years, they have been processing the information manually or using relational databases. Now it is time for a new technology, Big Data, which is able to analyze huge amount of data in a, relatively, small time. Along this book, characteristics and advantages of Big Data will be detailed, so as an introduction to Hadoop platform. This platform is based on Java and can perform the analysis of massive amount of data in different formats and coming from different sources. During this reading, the reader will be provided with the prior knowledge needed to it understanding, so as the temporal location, uses, forecast, evolution and growth in the next years.

Data consistency management in an open smart home management platform

In this paper, the authors introduce a novel mechanism for data management in a middleware for smart home control, where a relational database and semantic ontology storage are used at the same time in a Data Warehouse. An annotation system has been designed for instructing the storage format and location, registering new ontology concepts and most importantly, guaranteeing the Data Consistency between the two storage methods. For easing the data persistence process, the Data Access Object (DAO) pattern is applied and optimized to enhance the Data Consistency assurance. Finally, this novel mechanism provides an easy manner for the development of applications and their integration with BATMP. Finally, an application named "Parameter Monitoring Service" is given as an example for assessing the feasibility of the system.

Human Immunodeficiency Virus Reverse Transcriptase and Protease Sequence Database: an expanded data model integrating natural language text and sequence analysis programs

The HIV Reverse Transcriptase and Protease Sequence Database is an on-line relational database that catalogs evolutionary and drug-related sequence variation in the human immunodeficiency virus (HIV) reverse transcriptase (RT) and protease enzymes, the molecular targets of anti-HIV therapy (http://hivdb.stanford.edu). The database contains a compilation of nearly all published HIV RT and protease sequences, including submissions from International Collaboration databases and sequences published in journal articles. Sequences are linked to data about the source of the sequence sample and the antiretroviral drug treatment history of the individual from whom the isolate was obtained. During the past year 3500 sequences have been added and the data model has been expanded to include drug susceptibility data on sequenced isolates. Database content has also been integrated with didactic text and the output of two sequence analysis programs.

PlasmoDB: An integrative database of the Plasmodium falciparum genome. Tools for accessing and analyzing finished and unfinished sequence data

The Plasmodium falciparum Genome Database (http://PlasmoDB.org) integrates sequence information, automated analyses and annotation data emerging from the P.falciparum genome sequencing consortium. To date, raw sequence coverage is available for >90% of the genome, and two chromosomes have been finished and annotated. Data in PlasmoDB are organized by chromosome (1–14), and can be accessed using a variety of tools for graphical and text-based browsing or downloaded in various file formats. The GUS (Genomics Unified Schema) implementation of PlasmoDB provides a multi-species genomic relational database, incorporating data from human and mouse, as well as P.falciparum. The relational schema uses a highly structured format to accommodate diverse data sets related to genomic sequence and gene expression. Tools have been designed to facilitate complex biological queries, including many that are specific to Plasmodium parasites and malaria as a disease. Additional projects seek to integrate genomic information with the rich data sets now becoming available for RNA transcription, protein expression, metabolic pathways, genetic and physical mapping, antigenic and population diversity, and phylogenetic relationships with other apicomplexan parasites. The overall goal of PlasmoDB is to facilitate Internet- and CD-ROM-based access to both finished and unfinished sequence information by the global malaria research community.

Intracranial EEG correlates of implicit relational inference within the hippocampus

Drawing inferences from past experiences enables adaptive behavior in future situations. Inference has been shown to depend on hippocampal processes. Usually, inference is considered a deliberate and effortful mental act which happens during retrieval, and requires the focus of our awareness. Recent fMRI studies hint at the possibility that some forms of hippocampus-dependent inference can also occur during encoding and possibly also outside of awareness. Here, we sought to further explore the feasibility of hippocampal implicit inference, and specifically address the temporal evolution of implicit inference using intracranial EEG. Presurgical epilepsy patients with hippocampal depth electrodes viewed a sequence of word pairs, and judged the semantic fit between two words in each pair. Some of the word pairs entailed a common word (e.g.,‘winter - red’, ‘red - cat’) such that an indirect relation was established in following word pairs (e.g, ‘winter - cat’). The behavioral results suggested that drawing inference implicitly from past experience is feasible because indirect relations seemed to foster ‘fit’ judgments while the absence of indirect relations fostered 'do not fit' judgments, even though the participants were unaware of the indirect relations. A event-related potential (ERP) difference emerging 400 ms post-stimulus was evident in the hippocampus during encoding, suggesting that indirect relations were already established automatically during encoding of the overlapping word pairs. Further ERP differences emerged later post-stimulus (1500 ms), were modulated by the participants' responses and were evident during encoding and test. Furthermore, response-locked ERP effects were evident at test. These ERP effects could hence be a correlate of the interaction of implicit memory with decision-making. Together, the data map out a time-course in which the hippocampus automatically integrates memories from discrete but related episodes to implicitly influence future decision making.

Ex ante evaluations of alternate data structures for end user queries: Theory and experimental test

The data structure of an information system can significantly impact the ability of end users to efficiently and effectively retrieve the information they need. This research develops a methodology for evaluating, ex ante, the relative desirability of alternative data structures for end user queries. This research theorizes that the data structure that yields the lowest weighted average complexity for a representative sample of information requests is the most desirable data structure for end user queries. The theory was tested in an experiment that compared queries from two different relational database schemas. As theorized, end users querying the data structure associated with the less complex queries performed better Complexity was measured using three different Halstead metrics. Each of the three metrics provided excellent predictions of end user performance. This research supplies strong evidence that organizations can use complexity metrics to evaluate, ex ante, the desirability of alternate data structures. Organizations can use these evaluations to enhance the efficient and effective retrieval of information by creating data structures that minimize end user query complexity.

Selecting optimal instantiations of data models - Theory and validation of an ex ante approach

The schema of an information system can significantly impact the ability of end users to efficiently and effectively retrieve the information they need. Obtaining quickly the appropriate data increases the likelihood that an organization will make good decisions and respond adeptly to challenges. This research presents and validates a methodology for evaluating, ex ante, the relative desirability of alternative instantiations of a model of data. In contrast to prior research, each instantiation is based on a different formal theory. This research theorizes that the instantiation that yields the lowest weighted average query complexity for a representative sample of information requests is the most desirable instantiation for end-user queries. The theory was validated by an experiment that compared end-user performance using an instantiation of a data structure based on the relational model of data with performance using the corresponding instantiation of the data structure based on the object-relational model of data. Complexity was measured using three different Halstead metrics: program length, difficulty, and effort. For a representative sample of queries, the average complexity using each instantiation was calculated. As theorized, end users querying the instantiation with the lower average complexity made fewer semantic errors, i.e., were more effective at composing queries. (c) 2005 Elsevier B.V. All rights reserved.

A design pattern for efficient retrieval of large data sets from remote data sources

Retrieving large amounts of information over wide area networks, including the Internet, is problematic due to issues arising from latency of response, lack of direct memory access to data serving resources, and fault tolerance. This paper describes a design pattern for solving the issues of handling results from queries that return large amounts of data. Typically these queries would be made by a client process across a wide area network (or Internet), with one or more middle-tiers, to a relational database residing on a remote server. The solution involves implementing a combination of data retrieval strategies, including the use of iterators for traversing data sets and providing an appropriate level of abstraction to the client, double-buffering of data subsets, multi-threaded data retrieval, and query slicing. This design has recently been implemented and incorporated into the framework of a commercial software product developed at Oracle Corporation.

High-performance human resource practices, citizenship behaviour and organizational performance:A relational perspective

Taking a relational perspective on the employment relationship, we examined processes (mediation and moderation) linking high-performance human resource practices and productivity and turnover, two indicators of organizational performance. Multilevel analysis of data from hotels in the People's Republic of China revealed that service-oriented organizational citizenship behavior (OCB) partially mediated the relationships between high-performance human resource practices and both performance indicators. Unemployment rate moderated the service-oriented OCB-turnover relationship, and business strategy (service quality) moderated the service-oriented OCB-productivity relationship. Copyright of the Academy of Management, all rights reserved.

What's in a handshake? Exploring business-to-business relational exchange

Despite its importance in the development of competitive advantage, attempts to unify diverse classifications of business-to-business relational exchange (B2B RE) have been largely unsuccessful. We used 18 semi-structured, in-depth interviews with managers from a range of industries to explore the B2B RE construct. Analysis of the data revealed that B2B RE comprises five key dimensions. These are communication, understanding, commitment, trust and power symmetry. The research identifies the importance of personal interaction in business relationships and provides additional insights into the importance of affective commitment. In addition we uncover a number of negative consequences of affective commitment, which have been previously unexplored. This research contributes to the domain of B2B research by synthesising and advancing knowledge in this area to provide a new conceptual framework of B2B RE and directions for future research.

The implementation of a functional query language front-end to a relational database system. Available in 2 volumes.

Database systems have a user interface one of the components of which will normally be a query language which is based on a particular data model. Typically data models provide primitives to define, manipulate and query databases. Often these primitives are designed to form self-contained query languages. This thesis describes a prototype implementation of a system which allows users to specify queries against the database in a query language whose primitives are not those provided by the actual model on which the database system is based, but those provided by a different data model. The implementation chosen is the Functional Query Language Front End (FQLFE). This uses the Daplex functional data model and query language. Using FQLFE, users can specify the underlying database (based on the relational model) in terms of Daplex. Queries against this specified view can then be made in Daplex. FQLFE transforms these queries into the query language (Quel) of the underlying target database system (Ingres). The automation of part of the Daplex function definition phase is also described and its implementation discussed.

Rules for mapping a conceptual model onto various data base management systems. Available in 2 volumes.

The design and implementation of data bases involve, firstly, the formulation of a conceptual data model by systematic analysis of the structure and information requirements of the organisation for which the system is being designed; secondly, the logical mapping of this conceptual model onto the data structure of the target data base management system (DBMS); and thirdly, the physical mapping of this structured model into storage structures of the target DBMS. The accuracy of both the logical and physical mapping determine the performance of the resulting systems. This thesis describes research which develops software tools to facilitate the implementation of data bases. A conceptual model describing the information structure of a hospital is derived using the Entity-Relationship (E-R) approach and this model forms the basis for mapping onto the logical model. Rules are derived for automatically mapping the conceptual model onto relational and CODASYL types of data structures. Further algorithms are developed for partly automating the implementation of these models onto INGRES, MIMER and VAX-11 DBMS.

Relational diversity, social integration and individual effectiveness: a social self-regulation perspective

Relational demographers and dissimilarity researchers contend that group members who are dissimilar (vs. similar) to their peers in terms of a given diversity attribute (e.g. demographics, attitudes, values or traits) feel less attached to their work group, experience less satisfying and more conflicted relationships with their colleagues, and consequently are less effective. However, qualitative reviews suggest empirical findings tend to be weak and inconsistent (Chattopadhyay, Tluchowska and George, 2004; Riordan, 2000; Tsui and Gutek, 1999), and that it remains unclear when, how and to what extent such differences (i.e. relational diversity) affect group members social integration (i.e. attachment with their work group, satisfaction and conflicted relationships with their peers) and effectiveness (Riordan, 2000). This absence of meta-analytically derived effect size estimates and the lack of an integrative theoretical framework leave practitioners with inconclusive advice regarding whether the effects elicited by relational diversity are practically relevant, and if so how these should be managed. The current research develops an integrative theoretical framework, which it tests by using meta-analysis techniques and adding two further empirical studies to the literature. The first study reports a meta-analytic integration of the results of 129 tests of the relationship between relational diversity with social integration and individual effectiveness. Using meta-analytic and structural equation modelling techniques, it shows different effects of surface- and deep-level relational diversity on social integration Specifically, low levels of interdependence accentuated the negative effects of surface-level relational diversity on social integration, while high levels of interdependence accentuated the negative effects of deep-level relational diversity on social integration. The second study builds on a social self-regulation framework (Abrams, 1994) and suggests that under high levels of interdependence relational diversity is not one but two things: visibility and separation. Using ethnicity as a prominent example it was proposed that separation has a negative effect on group members effectiveness leading for those high in visibility and low in separation to overall positive additive effects, while to overall negative additive effects for those low in visibility and high in separation. These propositions were sustained in a sample of 621 business students working in 135 ethnically diverse work groups in a business simulation course over a period of 24 weeks. The third study suggests visibility has a positive effect on group members self-monitoring, while separation has a negative effect. The study proposed that high levels of visibility and low levels of separation lead to overall positive additive effects on self-monitoring but overall negative additive effects for those low in visibility and high in separation. Results from four waves of data on 261 business students working in 69 ethnically diverse work groups in a business simulation course held over a period of 24 weeks support these propositions.

Relational incentive policies in cross-border interfirm relationships

How an exporter can effectively craft a distributor agreement that encourages its network of foreign distributors to respond in pro-relational ways? This is an important issue as previous research has shown that relationship quality was linked to export performance. However, research failed to propose managerial tools that allowed exporters to foster relational phenomena in cross-border relationships. In this study, we suggest that exporters can influence importers' attitudes and behaviors with relational incentives policies. We also show that the impact of these policies is impervious to the noise, i.e., psychic distance and information asymmetry, that characterizes international business relationships. Our hypotheses are tested via structural equations modeling with data from a sample of French exporters.

The franchise dilemma:entrepreneurial characteristics, relational contracting, and opportunism in hybrid governance

Using franchise data, we identify that entrepreneurial characteristics of franchisees partially impact on their opportunistic tendencies. Further, relational contracting increases franchisee opportunism by strengthening the opportunism-enhancing impact of entrepreneurial characteristics. These findings point to a key dilemma franchisors need to be aware of: Entrepreneurially minded franchisees who might be better at exploiting market opportunities for their units may also behave more opportunistically, if given the chance through a more relational contracting regime. At the same time, if they perceive the contractual framework as being too rigid, they may be less able to leverage their capabilities, become dissatisfied, and exit the system.