Document clustering algorithms, representations and evaluation for information retrieval

This thesis presents new methods for classification and thematic grouping of billions of web pages, at scales previously not achievable. This process is also known as document clustering, where similar documents are automatically associated with clusters that represent various distinct topic. These automatically discovered topics are in turn used to improve search engine performance by only searching the topics that are deemed relevant to particular user queries.

Analysis of resource efficiency : a production frontier approach

This article integrates the material/energy flow analysis into a production frontier framework to quantify resource efficiency (RE). The emergy content of natural resources instead of their mass content is used to construct aggregate inputs. Using the production frontier approach, aggregate inputs will be optimised relative to given output quantities to derive RE measures. This framework is superior to existing RE indicators currently used in the literature. Using the exergy/emergy content in constructing aggregate material or energy flows overcomes a criticism that mass content cannot be used to capture different quality of differing types of resources. Derived RE measures are both ‘qualitative’ and ‘quantitative’, whereas existing RE indicators are only qualitative. An empirical examination into the RE of 116 economies was undertaken to illustrate the practical applicability of the new framework. The results showed that economies, on average, could reduce the consumption of resources by more than 30% without any reduction in per capita gross domestic product (GDP). This calculation occurred after adjustments for differences in the purchasing power of national currencies. The existence of high variations in RE across economies was found to be positively correlated with participation of people in labour force, population density, urbanisation, and GDP growth over the past five years. The results also showed that economies of a higher income group achieved higher RE, and those economies that are more dependent on imports and primary industries would have lower RE performance.

A case study of utilising concept knowledge in a topic specific document collection

The use of ‘topic’ concepts has shown improved search performance, given a query, by bringing together relevant documents which use different terms to describe a higher level concept. In this paper, we propose a method for discovering and utilizing concepts in indexing and search for a domain specific document collection being utilized in industry. This approach differs from others in that we only collect focused concepts to build the concept space and that instead of turning a user’s query into a concept based query, we experiment with different techniques of combining the original query with a concept query. We apply the proposed approach to a real-world document collection and the results show that in this scenario the use of concept knowledge at index and search can improve the relevancy of results.

Clustering and labeling a web scale document collection using Wikipedia clusters

Clustering is an important technique in organising and categorising web scale documents. The main challenges faced in clustering the billions of documents available on the web are the processing power required and the sheer size of the datasets available. More importantly, it is nigh impossible to generate the labels for a general web document collection containing billions of documents and a vast taxonomy of topics. However, document clusters are most commonly evaluated by comparison to a ground truth set of labels for documents. This paper presents a clustering and labeling solution where the Wikipedia is clustered and hundreds of millions of web documents in ClueWeb12 are mapped on to those clusters. This solution is based on the assumption that the Wikipedia contains such a wide range of diverse topics that it represents a small scale web. We found that it was possible to perform the web scale document clustering and labeling process on one desktop computer under a couple of days for the Wikipedia clustering solution containing about 1000 clusters. It takes longer to execute a solution with finer granularity clusters such as 10,000 or 50,000. These results were evaluated using a set of external data.

Mapping geographical knowledge and skills needed for pre-service teachers in teacher education

In Australia, for more than two decades, a ‘social science’ integrated framework was the favoured approach for delivering subjects such as history and geography. However, such interdisciplinary approaches have continued to attract criticism from various parts of the academic and public spheres and since 2009, a return to teaching the disciplines has been heralded as the ‘new’ way forward. Using discourse analysis techniques associated with Foucauldian archaeology, the purpose of this paper is to examine the Australian Curriculum: Geography document to ascertain the discourses necessary for pre-service teachers to enact effective teaching of geography in a primary setting. Then, based on pre-service teachers’ online survey responses, the paper investigates if such future teachers have the knowledge and skills to interpret, deliver and enact the new geography curriculum in primary classrooms. Finally, as teacher educators, our interest lies in preparing pre-service teachers effectively for the classroom so the findings are used to inform the content of a teacher education course for pre-service primary teachers.

Blood in our hearts or blood on our hands? The viscosity, vitality and validity of Aboriginal 'blood talk'

Blood metaphors abound in everyday social discourse among both Aboriginal and non-Aboriginal people. However, ‘Aboriginal blood talk’, more specifically, is located within a contradictory and contested space in terms of the meanings and values that can be attributed to it by Aboriginal and non-Aboriginal people. In the colonial context, blood talk operated as a tool of oppression for Aboriginal people via blood quantum discourses, yet today, Aboriginal people draw upon notions of blood, namely bloodlines, in articulating their identities. This paper juxtaposes contemporary Aboriginal blood talk as expressed by Aboriginal people against colonial blood talk and critically examines the ongoing political and intellectual governance regarding the validity of this talk in articulating Aboriginalities.

Topical pattern based document modelling and relevance ranking

For traditional information filtering (IF) models, it is often assumed that the documents in one collection are only related to one topic. However, in reality users’ interests can be diverse and the documents in the collection often involve multiple topics. Topic modelling was proposed to generate statistical models to represent multiple topics in a collection of documents, but in a topic model, topics are represented by distributions over words which are limited to distinctively represent the semantics of topics. Patterns are always thought to be more discriminative than single terms and are able to reveal the inner relations between words. This paper proposes a novel information filtering model, Significant matched Pattern-based Topic Model (SPBTM). The SPBTM represents user information needs in terms of multiple topics and each topic is represented by patterns. More importantly, the patterns are organized into groups based on their statistical and taxonomic features, from which the more representative patterns, called Significant Matched Patterns, can be identified and used to estimate the document relevance. Experiments on benchmark data sets demonstrate that the SPBTM significantly outperforms the state-of-the-art models.

The David and Goliath principle: Cultural, ideological, and attitudinal underpinnings of the normative protection of low-status groups from criticism

Two studies documented the “David and Goliath” rule—the tendency for people to perceive criticism of “David” groups (groups with low power and status) as less normatively permissible than criticism of “Goliath” groups (groups with high power and status). The authors confirmed the existence of the David and Goliath rule across Western and Chinese cultures (Study 1). However, the rule was endorsed more strongly in Western than in Chinese cultures, an effect mediated by cultural differences in power distance. Study 2 identified the psychological underpinnings of this rule in an Australian sample. Lower social dominance orientation (SDO) was associated with greater endorsement of the rule, an effect mediated through the differential attribution of stereotypes. Specifically, those low in SDO were more likely to attribute traits of warmth and incompetence to David versus Goliath groups, a pattern of stereotypes that was related to the protection of David groups from criticism.

Performative digital asset management: To propose a framework and proof of concept model that effectively enables researchers to document, archive and curate their non-traditional research data

This cross disciplinary study was conducted as two research and development projects. The outcome is a multimodal and dynamic chronicle, which incorporates the tracking of spatial, temporal and visual elements of performative practice-led and design-led research journeys. The distilled model provides a strong new approach to demonstrate rigour in non-traditional research outputs including provenance and an 'augmented web of facticity'.

Patient safety and the RCA: A document analysis

This research examined the function of Queensland Health's Root Cause Analysis (RCA) to improve patient safety through an investigation of patient harm events where permanent harm and preventable death, Severity Assessment Code 1, were the outcome of healthcare. Unedited and highly legislated RCAs from across Queensland Health public hospitals from 2009, 2010 and 2011 comprised the data. A document analysis revealed the RCAs opposed organisational policy and dominant theoretical directives. If we accept the prevailing assumption that patient harm is a systemic issue, then the RCA is failing to address harm events in healthcare.

Document timespan normalisation and understanding temporality for clinical records search

Previous qualitative research has highlighted that temporality plays an important role in relevance for clinical records search. In this study, an investigation is undertaken to determine the effect that the timespan of events within a patient record has on relevance in a retrieval scenario. In addition, based on the standard practise of document length normalisation, a document timespan normalisation model that specifically accounts for timespans is proposed. Initial analysis revealed that in general relevant patient records tended to cover a longer timespan of events than non-relevant patient records. However, an empirical evaluation using the TREC Medical Records track supports the opposite view that shorter documents (in terms of timespan) are better for retrieval. These findings highlight that the role of temporality in relevance is complex and how to effectively deal with temporality within a retrieval scenario remains an open question.

Semi-supervised document clustering via loci

Document clustering is one of the prominent methods for mining important information from the vast amount of data available on the web. However, document clustering generally suffers from the curse of dimensionality. Providentially in high dimensional space, data points tend to be more concentrated in some areas of clusters. We take advantage of this phenomenon by introducing a novel concept of dynamic cluster representation named as loci. Clusters’ loci are efficiently calculated using documents’ ranking scores generated from a search engine. We propose a fast loci-based semi-supervised document clustering algorithm that uses clusters’ loci instead of conventional centroids for assigning documents to clusters. Empirical analysis on real-world datasets shows that the proposed method produces cluster solutions with promising quality and is substantially faster than several benchmarked centroid-based semi-supervised document clustering methods.

CCD: Connected Component Descriptor for Robust Mosaicing of Camera-Captured Document Images

We propose a robust method for mosaicing of document images using features derived from connected components. Each connected component is described using the Angular Radial Tran. form (ART). To ensure geometric consistency during feature matching, the ART coefficients of a connected component are augmented with those of its two nearest neighbors. The proposed method addresses two critical issues often encountered in correspondence matching: (i) The stability of features and (ii) Robustness against false matches due to the multiple instances of characters in a document image. The use of connected components guarantees a stable localization across images. The augmented features ensure a successful correspondence matching even in the presence of multiple similar regions within the page. We illustrate the effectiveness of the proposed method on camera captured document images exhibiting large variations in viewpoint, illumination and scale.

Scalable document hashing and retrieval

This thesis studies document signatures, which are small representations of documents and other objects that can be stored compactly and compared for similarity. This research finds that document signatures can be effectively and efficiently used to both search and understand relationships between documents in large collections, scalable enough to search a billion documents in a fraction of a second. Deliverables arising from the research include an investigation of the representational capacity of document signatures, the publication of an open-source signature search platform and an approach for scaling signature retrieval to operate efficiently on collections containing hundreds of millions of documents.