Is intent-aware expected reciprocal rank sufficient to evaluate diversity?

In this paper we define two models of users that require diversity in search results; these models are theoretically grounded in the notion of intrinsic and extrinsic diversity. We then examine Intent-Aware Expected Reciprocal Rank (ERR-IA), one of the official measures used to assess diversity in TREC 2011-12, with respect to the proposed user models. By analyzing ranking preferences as expressed by the user models and those estimated by ERR-IA, we investigate whether ERR-IA assesses document rankings according to the requirements of the diversity retrieval task expressed by the two models. Empirical results demonstrate that ERR-IA neglects query-intents coverage by attributing excessive importance to redundant relevant documents. ERR-IA behavior is contrary to the user models that require measures to first assess diversity through the coverage of intents, and then assess the redundancy of relevant intents. Furthermore, diversity should be considered separately from document relevance and the documents positions in the ranking.

Structured Learning for Non-Smooth Ranking losses

Learning to rank from relevance judgment is an active research area. Itemwise score regression, pairwise preference satisfaction, and listwise structured learning are the major techniques in use. Listwise structured learning has been applied recently to optimize important non-decomposable ranking criteria like AUC (area under ROC curve) and MAP(mean average precision). We propose new, almost-lineartime algorithms to optimize for two other criteria widely used to evaluate search systems: MRR (mean reciprocal rank) and NDCG (normalized discounted cumulative gain)in the max-margin structured learning framework. We also demonstrate that, for different ranking criteria, one may need to use different feature maps. Search applications should not be optimized in favor of a single criterion, because they need to cater to a variety of queries. E.g., MRR is best for navigational queries, while NDCG is best for informational queries. A key contribution of this paper is to fold multiple ranking loss functions into a multi-criteria max-margin optimization.The result is a single, robust ranking model that is close to the best accuracy of learners trained on individual criteria. In fact, experiments over the popular LETOR and TREC data sets show that, contrary to conventional wisdom, a test criterion is often not best served by training with the same individual criterion.

Translation Induction on Indian Language Corpora Using Translingual Themes from Other Languages

Identifying translations from comparable corpora is a well-known problem with several applications, e.g. dictionary creation in resource-scarce languages. Scarcity of high quality corpora, especially in Indian languages, makes this problem hard, e.g. state-of-the-art techniques achieve a mean reciprocal rank (MRR) of 0.66 for English-Italian, and a mere 0.187 for Telugu-Kannada. There exist comparable corpora in many Indian languages with other ``auxiliary'' languages. We observe that translations have many topically related words in common in the auxiliary language. To model this, we define the notion of a translingual theme, a set of topically related words from auxiliary language corpora, and present a probabilistic framework for translation induction. Extensive experiments on 35 comparable corpora using English and French as auxiliary languages show that this approach can yield dramatic improvements in performance (e.g. MRR improves by 124% to 0.419 for Telugu-Kannada). A user study on WikiTSu, a system for cross-lingual Wikipedia title suggestion that uses our approach, shows a 20% improvement in the quality of titles suggested.

Automatic classification of scientific records using the German Subject Heading Authority File (SWD)

The following paper deals with an automatic text classification method which does not require training documents. For this method the German Subject Heading Authority File (SWD), provided by the linked data service of the German National Library is used. Recently the SWD was enriched with notations of the Dewey Decimal Classification (DDC). In consequence it became possible to utilize the subject headings as textual representations for the notations of the DDC. Basically, we we derive the classification of a text from the classification of the words in the text given by the thesaurus. The method was tested by classifying 3826 OAI-Records from 7 different repositories. Mean reciprocal rank and recall were chosen as evaluation measure. Direct comparison to a machine learning method has shown that this method is definitely competitive. Thus we can conclude that the enriched version of the SWD provides high quality information with a broad coverage for classification of German scientific articles.

Recomendr-entity recommendation based on ad-hoc dimensions

The growing availability and popularity of opinion rich resources on the online web resources, such as review sites and personal blogs, has made it convenient to find out about the opinions and experiences of layman people. But, simultaneously, this huge eruption of data has made it difficult to reach to a conclusion. In this thesis, I develop a novel recommendation system, Recomendr that can help users digest all the reviews about an entity and compare candidate entities based on ad-hoc dimensions specified by keywords. It expects keyword specified ad-hoc dimensions/features as input from the user and based on those features; it compares the selected range of entities using reviews provided on the related User Generated Contents (UGC) e.g. online reviews. It then rates the textual stream of data using a scoring function and returns the decision based on an aggregate opinion to the user. Evaluation of Recomendr using a data set in the laptop domain shows that it can effectively recommend the best laptop as per user-specified dimensions such as price. Recomendr is a general system that can potentially work for any entities on which online reviews or opinionated text is available.

Unsupervised manifold learning using Reciprocal kNN Graphs in image re-ranking and rank aggregation tasks

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

“Diving into reading” : revisiting reciprocal teaching in the middle years

This paper reports on a project that was designed to support teachers to introduce reciprocal teaching and student-generated questioning in the middle years. It began as a solution to teachers expressing a concern that there was a lack of professional development related to the teaching of reading in the middle years.

Professional growth through working together : a study of reciprocal benefits for teacher and education advisor through classroom based professional development

Teacher professional development provided by education advisors as one-off, centrally offered sessions does not always result in change in teacher knowledge, beliefs, attitudes or practice in the classroom. As the mathematics education advisor in this study, I set out to investigate a particular method of professional development so as to influence change in a practising classroom teacher’s knowledge and practices. The particular method of professional development utilised in this study was based on several principles of effective teacher professional development and saw me working regularly in a classroom with the classroom teacher as well as providing ongoing support for her for a full school year. The intention was to document the effects of this particular method of professional development in terms of the classroom teacher’s and my professional growth to provide insights for others working as education advisors. The professional development for the classroom teacher consisted of two components. The first was the co-operative development and implementation of a mental computation instructional program for the Year 3 class. The second component was the provision of ongoing support for the classroom teacher by the education advisor. The design of the professional development and the mental computation instructional program were progressively refined throughout the year. The education advisor fulfilled multiple roles in the study as teacher in the classroom, teacher educator working with the classroom teacher and researcher. Examples of the professional growth of the classroom teacher and the education advisor which occurred as sequences of changes (growth networks, Hollingsworth, 1999) in the domains of the professional world of the classroom teacher and education advisor were drawn from the large body of data collected through regular face-to-face and email communications between the classroom teacher and the education advisor as well as from transcripts of a structured interview. The Interconnected Model of Professional Growth (Clarke & Hollingsworth, 2002; Hollingsworth, 1999) was used to summarise and represent each example of the classroom teacher’s professional growth. A modified version of this model was used to summarise and represent the professional growth of the education advisor. This study confirmed that the method of professional development utilised could lead to significant teacher professional growth related directly to her work in the classroom. Using the Interconnected Model of Professional Growth to summarise and represent the classroom teacher’s professional growth and the modified version for my professional growth assisted with the recognition of examples of how we both changed. This model has potential to be used more widely by education advisors when preparing, implementing, evaluating and following-up on planned teacher professional development activities. The mental computation instructional program developed and trialled in the study was shown to be a successful way of sequencing and managing the teaching of mental computation strategies and related number sense understandings to Year 3 students. This study was conducted in one classroom, with one teacher in one school. The strength of this study was the depth of teacher support provided made possible by the particular method of the professional development, and the depth of analysis of the process. In another school, or with another teacher, this might not have been as successful. While I set out to change my practice as an education advisor I did not expect the depth of learning I experienced in terms of my knowledge, beliefs, attitudes and practices as an educator of teachers. This study has changed the way in which I plan to work as an education advisor in the future.

Low rank Runge-Kutta methods, symplecticity and stochastic Hamiltonian problems with additive noise

In this paper we extend the ideas of Brugnano, Iavernaro and Trigiante in their development of HBVM($s,r$) methods to construct symplectic Runge-Kutta methods for all values of $s$ and $r$ with $s\geq r$. However, these methods do not see the dramatic performance improvement that HBVMs can attain. Nevertheless, in the case of additive stochastic Hamiltonian problems an extension of these ideas, which requires the simulation of an independent Wiener process at each stage of a Runge-Kutta method, leads to methods that have very favourable properties. These ideas are illustrated by some simple numerical tests for the modified midpoint rule.

An extension to the rank constraint for improving the reliability of stereo matching algorithms using the rank transform

The rank transform is one non-parametric transform which has been applied to the stereo matching problem The advantages of this transform include its invariance to radio metric distortion and its amenability to hardware implementation. This paper describes the derivation of the rank constraint for matching using the rank transform Previous work has shown that this constraint was capable of resolving ambiguous matches thereby improving match reliability A new matching algorithm incorporating this constraint was also proposed. This paper extends on this previous work by proposing a matching algorithm which uses a dimensional match surface in which the match score is computed for every possible template and match window combination. The principal advantage of this algorithm is that the use of the match surface enforces the left�right consistency and uniqueness constraints thus improving the algorithms ability to remove invalid matches Experimental results for a number of test stereo pairs show that the new algorithm is capable of identifying and removing a large number of in incorrect matches particularly in the case of occlusions

An accurate and reliable stereo matching algorithm incorporating the rank constraint

A fundamental problem faced by stereo vision algorithms is that of determining correspondences between two images which comprise a stereo pair. This paper presents work towards the development of a new matching algorithm, based on the rank transform. This algorithm makes use of both area-based and edge-based information, and is therefore referred to as a hybrid algorithm. In addition, this algorithm uses a number of matching constraints,including the novel rank constraint. Results obtained using a number of test pairs show that the matching algorithm is capable of removing a significant proportion of invalid matches. The accuracy of matching in the vicinity of edges is also improved.

A hybrid stereo matching algorithm incorporating the rank constraint

A fundamental problem faced by stereo vision algorithms is that of determining correspondences between two images which comprise a stereo pair. This paper presents work towards the development of a new matching algorithm, based on the rank transform. This algorithm makes use of both area-based and edge-based information, and is therefore referred to as a hybrid algorithm. In addition, this algorithm uses a number of matching constraints, including the novel rank constraint. Results obtained using a number of test pairs show that the matching algorithm is capable of removing most invalid matches. The accuracy of matching in the vicinity of edges is also improved.

A constraint to improve the reliability of stereo matching using the rank transform

The rank transform is a non-parametric technique which has been recently proposed for the stereo matching problem. The motivation behind its application to the matching problem is its invariance to certain types of image distortion and noise, as well as its amenability to real-time implementation. This paper derives an analytic expression for the process of matching using the rank transform, and then goes on to derive one constraint which must be satisfied for a correct match. This has been dubbed the rank order constraint or simply the rank constraint. Experimental work has shown that this constraint is capable of resolving ambiguous matches, thereby improving matching reliability. This constraint was incorporated into a new algorithm for matching using the rank transform. This modified algorithm resulted in an increased proportion of correct matches, for all test imagery used.