Search engine content analysis

Search engines have forever changed the way people access and discover knowledge, allowing information about almost any subject to be quickly and easily retrieved within seconds. As increasingly more material becomes available electronically the influence of search engines on our lives will continue to grow. This presents the problem of how to find what information is contained in each search engine, what bias a search engine may have, and how to select the best search engine for a particular information need. This research introduces a new method, search engine content analysis, in order to solve the above problem. Search engine content analysis is a new development of traditional information retrieval field called collection selection, which deals with general information repositories. Current research in collection selection relies on full access to the collection or estimations of the size of the collections. Also collection descriptions are often represented as term occurrence statistics. An automatic ontology learning method is developed for the search engine content analysis, which trains an ontology with world knowledge of hundreds of different subjects in a multilevel taxonomy. This ontology is then mined to find important classification rules, and these rules are used to perform an extensive analysis of the content of the largest general purpose Internet search engines in use today. Instead of representing collections as a set of terms, which commonly occurs in collection selection, they are represented as a set of subjects, leading to a more robust representation of information and a decrease of synonymy. The ontology based method was compared with ReDDE (Relevant Document Distribution Estimation method for resource selection) using the standard R-value metric, with encouraging results. ReDDE is the current state of the art collection selection method which relies on collection size estimation. The method was also used to analyse the content of the most popular search engines in use today, including Google and Yahoo. In addition several specialist search engines such as Pubmed and the U.S. Department of Agriculture were analysed. In conclusion, this research shows that the ontology based method mitigates the need for collection size estimation.

Peer to peer English/Chinese cross-language information retrieval

Peer to peer systems have been widely used in the internet. However, most of the peer to peer information systems are still missing some of the important features, for example cross-language IR (Information Retrieval) and collection selection / fusion features. Cross-language IR is the state-of-art research area in IR research community. It has not been used in any real world IR systems yet. Cross-language IR has the ability to issue a query in one language and receive documents in other languages. In typical peer to peer environment, users are from multiple countries. Their collections are definitely in multiple languages. Cross-language IR can help users to find documents more easily. E.g. many Chinese researchers will search research papers in both Chinese and English. With Cross-language IR, they can do one query in Chinese and get documents in two languages. The Out Of Vocabulary (OOV) problem is one of the key research areas in crosslanguage information retrieval. In recent years, web mining was shown to be one of the effective approaches to solving this problem. However, how to extract Multiword Lexical Units (MLUs) from the web content and how to select the correct translations from the extracted candidate MLUs are still two difficult problems in web mining based automated translation approaches. Discovering resource descriptions and merging results obtained from remote search engines are two key issues in distributed information retrieval studies. In uncooperative environments, query-based sampling and normalized-score based merging strategies are well-known approaches to solve such problems. However, such approaches only consider the content of the remote database but do not consider the retrieval performance of the remote search engine. This thesis presents research on building a peer to peer IR system with crosslanguage IR and advance collection profiling technique for fusion features. Particularly, this thesis first presents a new Chinese term measurement and new Chinese MLU extraction process that works well on small corpora. An approach to selection of MLUs in a more accurate manner is also presented. After that, this thesis proposes a collection profiling strategy which can discover not only collection content but also retrieval performance of the remote search engine. Based on collection profiling, a web-based query classification method and two collection fusion approaches are developed and presented in this thesis. Our experiments show that the proposed strategies are effective in merging results in uncooperative peer to peer environments. Here, an uncooperative environment is defined as each peer in the system is autonomous. Peer like to share documents but they do not share collection statistics. This environment is a typical peer to peer IR environment. Finally, all those approaches are grouped together to build up a secure peer to peer multilingual IR system that cooperates through X.509 and email system.

Enriching XML documents clustering by using concise structure and content

With the growing number of XML documents on theWeb it becomes essential to effectively organise these XML documents in order to retrieve useful information from them. A possible solution is to apply clustering on the XML documents to discover knowledge that promotes effective data management, information retrieval and query processing. However, many issues arise in discovering knowledge from these types of semi-structured documents due to their heterogeneity and structural irregularity. Most of the existing research on clustering techniques focuses only on one feature of the XML documents, this being either their structure or their content due to scalability and complexity problems. The knowledge gained in the form of clusters based on the structure or the content is not suitable for reallife datasets. It therefore becomes essential to include both the structure and content of XML documents in order to improve the accuracy and meaning of the clustering solution. However, the inclusion of both these kinds of information in the clustering process results in a huge overhead for the underlying clustering algorithm because of the high dimensionality of the data. The overall objective of this thesis is to address these issues by: (1) proposing methods to utilise frequent pattern mining techniques to reduce the dimension; (2) developing models to effectively combine the structure and content of XML documents; and (3) utilising the proposed models in clustering. This research first determines the structural similarity in the form of frequent subtrees and then uses these frequent subtrees to represent the constrained content of the XML documents in order to determine the content similarity. A clustering framework with two types of models, implicit and explicit, is developed. The implicit model uses a Vector Space Model (VSM) to combine the structure and the content information. The explicit model uses a higher order model, namely a 3- order Tensor Space Model (TSM), to explicitly combine the structure and the content information. This thesis also proposes a novel incremental technique to decompose largesized tensor models to utilise the decomposed solution for clustering the XML documents. The proposed framework and its components were extensively evaluated on several real-life datasets exhibiting extreme characteristics to understand the usefulness of the proposed framework in real-life situations. Additionally, this research evaluates the outcome of the clustering process on the collection selection problem in the information retrieval on the Wikipedia dataset. The experimental results demonstrate that the proposed frequent pattern mining and clustering methods outperform the related state-of-the-art approaches. In particular, the proposed framework of utilising frequent structures for constraining the content shows an improvement in accuracy over content-only and structure-only clustering results. The scalability evaluation experiments conducted on large scaled datasets clearly show the strengths of the proposed methods over state-of-the-art methods. In particular, this thesis work contributes to effectively combining the structure and the content of XML documents for clustering, in order to improve the accuracy of the clustering solution. In addition, it also contributes by addressing the research gaps in frequent pattern mining to generate efficient and concise frequent subtrees with various node relationships that could be used in clustering.

Document clustering evaluation : Divergence from a random baseline

Divergence from a random baseline is a technique for the evaluation of document clustering. It ensures cluster quality measures are performing work that prevents ineffective clusterings from giving high scores to clusterings that provide no useful result. These concepts are defined and analysed using intrinsic and extrinsic approaches to the evaluation of document cluster quality. This includes the classical clusters to categories approach and a novel approach that uses ad hoc information retrieval. The divergence from a random baseline approach is able to differentiate ineffective clusterings encountered in the INEX XML Mining track. It also appears to perform a normalisation similar to the Normalised Mutual Information (NMI) measure but it can be applied to any measure of cluster quality. When it is applied to the intrinsic measure of distortion as measured by RMSE, subtraction from a random baseline provides a clear optimum that is not apparent otherwise. This approach can be applied to any clustering evaluation. This paper describes its use in the context of document clustering evaluation.

Um estudo sobre a proposta para formação continuada de professores de leitura e escrita no Programa Pró-Letramento: 2005-2009

Pós-graduação em Educação - FFC

Portfolios and pedagogy : an examination of ideology and use

Portfolio use in writing studies contexts is becoming ubiquitous and, as such, portfolios are in danger of being rendered meaningless and thus require that we more fully theorize and historicize portfolios. To this end, I examine portfolios: both the standardized portfolio used for assessment purposes and the personalized portfolio used for entering the job market. I take a critical look at portfolios as a form of technology and acknowledge some of the dangers of blindly using portfolios for gaining employment in the current economic structure of fast capitalism. As educators in the writing studies fields, it is paramount that instructors have a critical awareness of the consequences of portfolio creation on students as designers, lifelong learners, and citizens of a larger society. I argue that a better understanding of the pedagogical implications for portfolio use is imperative before implementing them in the classroom, and that a social-epistemic approach provides a valuable rethinking of portfolio use for assessment purposes. Further, I argue for the notions of meditation and transformation to be added alongside collection, selection, and reflection because they enable portfolio designers and evaluators alike to thoughtfully consider new ways of meaning-making and innovation. Also important and included with meditation and transformation is the understanding that students are ideologically positioned in the educational system. For them to begin recognizing their situatedness is a step toward becoming designers of change. The portfolio can be a site for that change, and a way for them to document their own learning and ways of making meaning over a lifetime.

The Boston Handel and Haydn Society collection of church music : being a selection of the most approved psalm and hymn tunes, anthems, sentences, chants, &c. : together with many beautiful extracts from the works of Haydn, Mozart, Beethoven, and other eminent composers, harmonized for three and four voices, with a figured base for the organ and piano forte /

Mode of access: Internet.

Musical pastime; a new collection of popular melodies, arranged as duets for the violin and piano. Consisting of the latest and best marches, galops, waltzes. polkas, round and square dances &c. Together with a choice selection of solos upon classical airs, arranged for amateurs.

Mode of access: Internet.

A digested index to the reported decisions in the House of lords, Privy council, and in the courts of common law, equity, divorce, probate, admiralty, and ecclesiastical : with a selection from the Irish chancery and common law reports, and references to the statutes passed, and rules and orders of court promulgated; and a collection of cases overruled and impeached, during the year 1859 /

Mode of access: Internet.

The print collector; an introduction to the knowledge necessary for forming a collection of ancient prints. Containing suggestions as to the mode of commencing collector, the selection of specimens, the prices and care of prints. Also notices of the marks of proprietorship used by collectors, remarks on the ancient and modern practice of the art and a catalogue raisonné of books on engraving and prints.

Published anonymously. By Joseph Maberly.

Chess problems, a collection of original positions, forming one hundred ends of games won or drawn; to which is added a selection of games, including those played between the Leeds and Liverpool clubs, with remarks.

Mode of access: Internet.

Lowe's collection of reels, strathspeys, & jigs. : Being a new and complete selection of the best dancing tunes in their proper keys, as performed at the different assemblies and public meetings throughout Scotland. Carefully arranged with appropriate basses for the piano forte & violoncello. Book 1[-6].

Book 1: 3d ed.; book 2: 2d ed.