Dynamic data retrieval on the World Wide Web

Methods for accessing data on the Web have been the focus of active research over the past few years. In this thesis we propose a method for representing Web sites as data sources. We designed a Data Extractor data retrieval solution that allows us to define queries to Web sites and process resulting data sets. Data Extractor is being integrated into the MSemODB heterogeneous database management system. With its help database queries can be distributed over both local and Web data sources within MSemODB framework. ^ Data Extractor treats Web sites as data sources, controlling query execution and data retrieval. It works as an intermediary between the applications and the sites. Data Extractor utilizes a twofold “custom wrapper” approach for information retrieval. Wrappers for the majority of sites are easily built using a powerful and expressive scripting language, while complex cases are processed using Java-based wrappers that utilize specially designed library of data retrieval, parsing and Web access routines. In addition to wrapper development we thoroughly investigate issues associated with Web site selection, analysis and processing. ^ Data Extractor is designed to act as a data retrieval server, as well as an embedded data retrieval solution. We also use it to create mobile agents that are shipped over the Internet to the client's computer to perform data retrieval on behalf of the user. This approach allows Data Extractor to distribute and scale well. ^ This study confirms feasibility of building custom wrappers for Web sites. This approach provides accuracy of data retrieval, and power and flexibility in handling of complex cases. ^

Dynamic data retrieval on the world wide web

Methods for accessing data on the Web have been the focus of active research over the past few years. In this thesis we propose a method for representing Web sites as data sources. We designed a Data Extractor data retrieval solution that allows us to define queries to Web sites and process resulting data sets. Data Extractor is being integrated into the MSemODB heterogeneous database management system. With its help database queries can be distributed over both local and Web data sources within MSemODB framework. Data Extractor treats Web sites as data sources, controlling query execution and data retrieval. It works as an intermediary between the applications and the sites. Data Extractor utilizes a two-fold "custom wrapper" approach for information retrieval. Wrappers for the majority of sites are easily built using a powerful and expressive scripting language, while complex cases are processed using Java-based wrappers that utilize specially designed library of data retrieval, parsing and Web access routines. In addition to wrapper development we thoroughly investigate issues associated with Web site selection, analysis and processing. Data Extractor is designed to act as a data retrieval server, as well as an embedded data retrieval solution. We also use it to create mobile agents that are shipped over the Internet to the client's computer to perform data retrieval on behalf of the user. This approach allows Data Extractor to distribute and scale well. This study confirms feasibility of building custom wrappers for Web sites. This approach provides accuracy of data retrieval, and power and flexibility in handling of complex cases.

Data partitioning and replication management in distributed geographic information system (GIS) database

This research presents several components encompassing the scope of the objective of Data Partitioning and Replication Management in Distributed GIS Database. Modern Geographic Information Systems (GIS) databases are often large and complicated. Therefore data partitioning and replication management problems need to be addresses in development of an efficient and scalable solution. ^ Part of the research is to study the patterns of geographical raster data processing and to propose the algorithms to improve availability of such data. These algorithms and approaches are targeting granularity of geographic data objects as well as data partitioning in geographic databases to achieve high data availability and Quality of Service(QoS) considering distributed data delivery and processing. To achieve this goal a dynamic, real-time approach for mosaicking digital images of different temporal and spatial characteristics into tiles is proposed. This dynamic approach reuses digital images upon demand and generates mosaicked tiles only for the required region according to user's requirements such as resolution, temporal range, and target bands to reduce redundancy in storage and to utilize available computing and storage resources more efficiently. ^ Another part of the research pursued methods for efficient acquiring of GIS data from external heterogeneous databases and Web services as well as end-user GIS data delivery enhancements, automation and 3D virtual reality presentation. ^ There are vast numbers of computing, network, and storage resources idling or not fully utilized available on the Internet. Proposed "Crawling Distributed Operating System "(CDOS) approach employs such resources and creates benefits for the hosts that lend their CPU, network, and storage resources to be used in GIS database context. ^ The results of this dissertation demonstrate effective ways to develop a highly scalable GIS database. The approach developed in this dissertation has resulted in creation of TerraFly GIS database that is used by US government, researchers, and general public to facilitate Web access to remotely-sensed imagery and GIS vector information. ^

The Impact of the Internet in Six Latin American Countries

Access to the Internet has grown exponentially in Latin America over the past decade. The International Telecommunications Union (ITU) estimates that in 2009 there were 144.5 million Internet users in South America, 6.4 million in Central America, and 8.2 million in the Caribbean, or a total 159.2 million users in all of Latin America.1 At that time, ITU reported an estimated 31 million Internet users in Mexico, which would bring the overall number of users in Latin America to 190.2 million people. More recent estimates published by Internet World Stats place Internet access currently at an estimated 204.6 million out of a total population of 592.5 million in the region (this figure includes Mexico).2 According to those figures, 34.5 per cent of the Latin American population now enjoys Internet access. In recent years, universal access policies contributed to the vast increase in digital literacy and Internet use in Argentina, Brazil, Chile, Colombia, and Costa Rica. Whereas the latter was the first country in the region to adopt a policy of universal access, the most expansive and successful digital inclusion programs in the region have taken hold in Brazil and Chile. These two countries have allocated considerable resources to the promotion of digital literacy and Internet access among low income and poor populations; in both cases, civil society groups significantly assisted in the promotion of inclusion at the grassroots level. Digital literacy and Internet access have come to represent, particularly in the area of education, a welcome complementary resource for populations chronically underserved in nations with a long-standing record of inadequate public social services. Digital inclusion is vastly expanding throughout the region, thanks to stabilizing economies, increasingly affordable technology, and the rapid growth in the supply of cellular mobile telephony. A recent study by the global advertising agency Razorfish revealed significant shifts in the demographics of digital inclusion in the major economies of South America, where Web access is rapidly increasing amid the lower middle class and the working poor.3 Several researchers have suggested that Internet access will bring about greater civic participation and engagement, although skeptics remain unsure this could happen in Latin America. Yet, there have been some recent instances of political mobilization facilitated through the use of the Web and social media applications, starting in Chile when “smart mobs” nationwide demonstrated against former Chilean President Michelle Bachelet when she failed to enact education reforms in May 2006. The Internet has also been used by marginalized groups and by guerrillas groups to highlight their stories. In sum, Internet access in Latin is no longer a medium restricted to the elite. It is rather a public sphere upon which civil society has staked its claim. Some of the examples noted in this study point toward a developing trend whereby civil society, through online grassroots movements, is able to effectively pressure public officials, instill transparency and demand accountability in government. Access to the Internet has also made it possible for voices on the margins to participate in the conversation in a way that was never previously feasible. 1 International Telecommunications Union [ITU], “Information Technology Public & Report,” accessed May 15, 2011, http://www.itu.int/. 2 Internet World Stats, “Internet Usage Statistics for the Americas,” accessed March 24, 2011, http://www.internetworldstats.com/stats2.htm 3 J. Crump, “The finch and the fox,” London, UK (2010), http://www.slideshare.net/razorfishmarketing/the-finch-and-the-fox.

A semantic paradigm for intelligent data access

An implementation of Sem-ODB—a database management system based on the Semantic Binary Model is presented. A metaschema of Sem-ODB database as well as the top-level architecture of the database engine is defined. A new benchmarking technique is proposed which allows databases built on different database models to compete fairly. This technique is applied to show that Sem-ODB has excellent efficiency comparing to a relational database on a certain class of database applications. A new semantic benchmark is designed which allows evaluation of the performance of the features characteristic of semantic database applications. An application used in the benchmark represents a class of problems requiring databases with sparse data, complex inheritances and many-to-many relations. Such databases can be naturally accommodated by semantic model. A fixed predefined implementation is not enforced allowing the database designer to choose the most efficient structures available in the DBMS tested. The results of the benchmark are analyzed. ^ A new high-level querying model for semantic databases is defined. It is proven adequate to serve as an efficient native semantic database interface, and has several advantages over the existing interfaces. It is optimizable and parallelizable, supports the definition of semantic userviews and the interoperability of semantic databases with other data sources such as World Wide Web, relational, and object-oriented databases. The query is structured as a semantic database schema graph with interlinking conditionals. The query result is a mini-database, accessible in the same way as the original database. The paradigm supports and utilizes the rich semantics and inherent ergonomics of semantic databases. ^ The analysis and high-level design of a system that exploits the superiority of the Semantic Database Model to other data models in expressive power and ease of use to allow uniform access to heterogeneous data sources such as semantic databases, relational databases, web sites, ASCII files, and others via a common query interface is presented. The Sem-ODB engine is used to control all the data sources combined under a unified semantic schema. A particular application of the system to provide an ODBC interface to the WWW as a data source is discussed. ^

Design and implementation of a web based shop floor control system to maintain schedule feasibility in a dynamic job shop environment

The effective control of production activities in dynamic job shop with predetermined resource allocation for all the jobs entering the system is a unique manufacturing environment, which exists in the manufacturing industry. In this thesis a framework for an Internet based real time shop floor control system for such a dynamic job shop environment is introduced. The system aims to maintain the schedule feasibility of all the jobs entering the manufacturing system under any circumstance. The system is capable of deciding how often the manufacturing activities should be monitored to check for control decisions that need to be taken on the shop floor. The system will provide the decision maker real time notification to enable him to generate feasible alternate solutions in case a disturbance occurs on the shop floor. The control system is also capable of providing the customer with real time access to the status of the jobs on the shop floor. The communication between the controller, the user and the customer is through web based user friendly GUI. The proposed control system architecture and the interface for the communication system have been designed, developed and implemented.