Desarrollo de un sistema de arañas inteligentes para la extracción de datos de empresas

El presente proyecto de fin de grado es uno de los resultados generados en un proyecto de financiación privada por parte de Telefónica consistente en el desarrollo y posterior implantación de un sistema para minería de datos de empresas presentes en Internet. Este TFG surge a partir de un proyecto que el grupo de investigación AICU-LABS (Mercator) de la UPM ha desarrollado para Telefónica, y tiene como elemento principal el desarrollo de Agentes web (también llamados robots software, “softbots” o “crawlers”) capaces de obtener datos de empresas a partir de sus CIF a través de internet. El listado de empresas nos los proporciona Telefónica, y está compuesto por empresas que no son clientes de Telefónica en la actualidad. Nuestra misión es proporcionarles los datos necesarios (principalmente teléfono, correo electrónico y dirección de la empresa) para la creación de una base de datos de potenciales clientes. Para llevar a cabo esta tarea, se ha realizado una aplicación que, a partir de los CIF que nos proporcionan, busque información en internet y extraiga aquella que nos interese. Además se han desarrollado sistemas de validación de datos para ayudarnos a descartar datos no válidos y clasificar los datos según su calidad para así maximizar la calidad de los datos producidos por el robot. La búsqueda de datos se hará tanto en bases de datos online como, en caso de localizarlas, las propias páginas web de las empresas. ABSTRACT This Final Degree Project is one of the results obtained from a project funded by Telefónica. This project consists on the development and subsequent implantation of a system which performs data mining on companies operating on the Internet. This document arises from a project the research group AICU-LABS (Mercator) from the Universidad Politécnica de Madrid has developed for Telefónica. The main goal of this project is the creation of web agents (also known as “crawlers” or “web spiders”) able to obtain data from businesses through the Internet, knowing only their VAT identification number. The list of companies is given by Telefónica, and it is composed by companies that are not Telefónica’s customers today. Our mission is to provide the data required (mainly phone, email and address of the company) to create a database of potential customers. To perform this task, we’ve developed an application that, starting with the given VAT numbers, searches the web for information and extracts the data sought. In addition, we have developed data validation systems, that are capable of discarding low quality data and also sorting the data according to their quality, to maximize the quality of the results produced by the robot. We’ll use both the companies’ websites and external databases as our sources of information.

The anatomy of urban social networks and its implications in the searchability problem

The appearance of large geolocated communication datasets has recently increased our understanding of how social networks relate to their physical space. However, many recurrently reported properties, such as the spatial clustering of network communities, have not yet been systematically tested at different scales. In this work we analyze the social network structure of over 25 million phone users from three countries at three different scales: country, provinces and cities. We consistently find that this last urban scenario presents significant differences to common knowledge about social networks. First, the emergence of a giant component in the network seems to be controlled by whether or not the network spans over the entire urban border, almost independently of the population or geographic extension of the city. Second, urban communities are much less geographically clustered than expected. These two findings shed new light on the widely-studied searchability in self-organized networks. By exhaustive simulation of decentralized search strategies we conclude that urban networks are searchable not through geographical proximity as their country-wide counterparts, but through an homophily-driven community structure.

1984-1986: Thomas Miller Jenkins

15th President 1984-1986

1898-1901: John H. Jackson

2nd President 1898-1902

1902-1918: Benjamin F. Allen

4th President 1902-1918

1923-1927, 1929-1931: Nathan B. Young

7th & 9th President 1923-19271929-1931

1927-1929, 1937-1938: William B. Jason

8th President 1927-19291937-1938 (Acting President)

1956-1969: Earl Edgar Dawson

12th President 1956-1969

1875-1878: Samuel T. Mitchell

Principal In Office: 1875-1878

1866-1870: Richard Baxter Foster

1st PrincipalIn Office: 1866-1870

1918-1922: Clement Richardson

5th President 1918-1922

1931-1937: Charles W. Florence

10th President 1931-1937

1973-1982 (Nov.): James Frank

14th President 1973-1982 (Nov.)

1988-1996: Wendell G. Rayburn

16th President 1988-1996

2005-2012: Dr. Carolyn Mahoney

18th President 1st Female President 2005-2012 Dr. Carolyn R. Mahoney earned her B.S. degree in Mathematics from Siena College; M.S. and PhD. Degree in Mathematics from the Ohio State University. On February 1, 2005, Dr. Mahoney became the 18th president of Lincoln University of Missouri.