Using interaction data for improving the offline and online evaluation of search engines

This thesis investigates how web search evaluation can be improved using historical interaction data. Modern search engines combine offline and online evaluation approaches in a sequence of steps that a tested change needs to pass through to be accepted as an improvement and subsequently deployed. We refer to such a sequence of steps as an evaluation pipeline. In this thesis, we consider the evaluation pipeline to contain three sequential steps: an offline evaluation step, an online evaluation scheduling step, and an online evaluation step. In this thesis we show that historical user interaction data can aid in improving the accuracy or efficiency of each of the steps of the web search evaluation pipeline. As a result of these improvements, the overall efficiency of the entire evaluation pipeline is increased. Firstly, we investigate how user interaction data can be used to build accurate offline evaluation methods for query auto-completion mechanisms. We propose a family of offline evaluation metrics for query auto-completion that represents the effort the user has to spend in order to submit their query. The parameters of our proposed metrics are trained against a set of user interactions recorded in the search engine’s query logs. From our experimental study, we observe that our proposed metrics are significantly more correlated with an online user satisfaction indicator than the metrics proposed in the existing literature. Hence, fewer changes will pass the offline evaluation step to be rejected after the online evaluation step. As a result, this would allow us to achieve a higher efficiency of the entire evaluation pipeline. Secondly, we state the problem of the optimised scheduling of online experiments. We tackle this problem by considering a greedy scheduler that prioritises the evaluation queue according to the predicted likelihood of success of a particular experiment. This predictor is trained on a set of online experiments, and uses a diverse set of features to represent an online experiment. Our study demonstrates that a higher number of successful experiments per unit of time can be achieved by deploying such a scheduler on the second step of the evaluation pipeline. Consequently, we argue that the efficiency of the evaluation pipeline can be increased. Next, to improve the efficiency of the online evaluation step, we propose the Generalised Team Draft interleaving framework. Generalised Team Draft considers both the interleaving policy (how often a particular combination of results is shown) and click scoring (how important each click is) as parameters in a data-driven optimisation of the interleaving sensitivity. Further, Generalised Team Draft is applicable beyond domains with a list-based representation of results, i.e. in domains with a grid-based representation, such as image search. Our study using datasets of interleaving experiments performed both in document and image search domains demonstrates that Generalised Team Draft achieves the highest sensitivity. A higher sensitivity indicates that the interleaving experiments can be deployed for a shorter period of time or use a smaller sample of users. Importantly, Generalised Team Draft optimises the interleaving parameters w.r.t. historical interaction data recorded in the interleaving experiments. Finally, we propose to apply the sequential testing methods to reduce the mean deployment time for the interleaving experiments. We adapt two sequential tests for the interleaving experimentation. We demonstrate that one can achieve a significant decrease in experiment duration by using such sequential testing methods. The highest efficiency is achieved by the sequential tests that adjust their stopping thresholds using historical interaction data recorded in diagnostic experiments. Our further experimental study demonstrates that cumulative gains in the online experimentation efficiency can be achieved by combining the interleaving sensitivity optimisation approaches, including Generalised Team Draft, and the sequential testing approaches. Overall, the central contributions of this thesis are the proposed approaches to improve the accuracy or efficiency of the steps of the evaluation pipeline: the offline evaluation frameworks for the query auto-completion, an approach for the optimised scheduling of online experiments, a general framework for the efficient online interleaving evaluation, and a sequential testing approach for the online search evaluation. The experiments in this thesis are based on massive real-life datasets obtained from Yandex, a leading commercial search engine. These experiments demonstrate the potential of the proposed approaches to improve the efficiency of the evaluation pipeline.

An ontology for risk management of digital collections

Maintaining accessibility to and understanding of digital information over time is a complex challenge that often requires contributions and interventions from a variety of individuals and organizations. The processes of preservation planning and evaluation are fundamentally implicit and share similar complexity. Both demand comprehensive knowledge and understanding of every aspect of to-be-preserved content and the contexts within which preservation is undertaken. Consequently, means are required for the identification, documentation and association of those properties of data, representation and management mechanisms that in combination lend value, facilitate interaction and influence the preservation process. These properties may be almost limitless in terms of diversity, but are integral to the establishment of classes of risk exposure, and the planning and deployment of appropriate preservation strategies. We explore several research objectives within the course of this thesis. Our main objective is the conception of an ontology for risk management of digital collections. Incorporated within this are our aims to survey the contexts within which preservation has been undertaken successfully, the development of an appropriate methodology for risk management, the evaluation of existing preservation evaluation approaches and metrics, the structuring of best practice knowledge and lastly the demonstration of a range of tools that utilise our findings. We describe a mixed methodology that uses interview and survey, extensive content analysis, practical case study and iterative software and ontology development. We build on a robust foundation, the development of the Digital Repository Audit Method Based on Risk Assessment. We summarise the extent of the challenge facing the digital preservation community (and by extension users and creators of digital materials from many disciplines and operational contexts) and present the case for a comprehensive and extensible knowledge base of best practice. These challenges are manifested in the scale of data growth, the increasing complexity and the increasing onus on communities with no formal training to offer assurances of data management and sustainability. These collectively imply a challenge that demands an intuitive and adaptable means of evaluating digital preservation efforts. The need for individuals and organisations to validate the legitimacy of their own efforts is particularly prioritised. We introduce our approach, based on risk management. Risk is an expression of the likelihood of a negative outcome, and an expression of the impact of such an occurrence. We describe how risk management may be considered synonymous with preservation activity, a persistent effort to negate the dangers posed to information availability, usability and sustainability. Risk can be characterised according to associated goals, activities, responsibilities and policies in terms of both their manifestation and mitigation. They have the capacity to be deconstructed into their atomic units and responsibility for their resolution delegated appropriately. We continue to describe how the manifestation of risks typically spans an entire organisational environment, and as the focus of our analysis risk safeguards against omissions that may occur when pursuing functional, departmental or role-based assessment. We discuss the importance of relating risk-factors, through the risks themselves or associated system elements. To do so will yield the preservation best-practice knowledge base that is conspicuously lacking within the international digital preservation community. We present as research outcomes an encapsulation of preservation practice (and explicitly defined best practice) as a series of case studies, in turn distilled into atomic, related information elements. We conduct our analyses in the formal evaluation of memory institutions in the UK, US and continental Europe. Furthermore we showcase a series of applications that use the fruits of this research as their intellectual foundation. Finally we document our results in a range of technical reports and conference and journal articles. We present evidence of preservation approaches and infrastructures from a series of case studies conducted in a range of international preservation environments. We then aggregate this into a linked data structure entitled PORRO, an ontology relating preservation repository, object and risk characteristics, intended to support preservation decision making and evaluation. The methodology leading to this ontology is outlined, and lessons are exposed by revisiting legacy studies and exposing the resource and associated applications to evaluation by the digital preservation community.

Transition to Product Service Systems: methodology based on scenarios identification, modelling and evaluation

Part 4: Transition Towards Product-Service Systems

Evaluation of Neurodevelopment and Factors Affecting it in Children With Acyanotic Congenital Cardiac Disease

Background: The rate of congenital heart disease is 0.8% in all live births. The majority of this, however, is acyanotic congenital heart disease. The survival rate of children with cardiac disease has increased with the developments provided in recent years and their lifetime is extended. Objectives: This study aims to evaluate neurodevelopment of children with uncomplicated acyanotic congenital heart disease in preschool period and determine the factors affecting their neurodevelopmental process. Patients and Methods: 132 children with acyanotic congenital heart disease aged 6 - 72 months were involved in the study. Mental development and intelligence levels of patients under 2 years old were assessed by using Bayley Development Scale-III, and Stanford Binet Intelligence test was employed for patients over 2 years old. Denver Developmental Screening Test II was applied to all patients for their personal-social, fine motor, gross motor and language development. Results: The average age of patients (67 girls, 65 boys) included in the study was 35.2 ± 19.6 months. It was determined that there were subnormal mental level in 13 (10%) patients and at least one specific developmental disorder in 33 (25%) patients. Bayley Mental Development Scale score of patients who had received incubator care in perinatal period was found significantly low (88 ± 4.2) compared to those with no incubator care (93.17 ± 8.5) (P = 0.028). Low educational level of father was established to be linked with low mental development scores at the age of 2 and following that age (P < 0.05). Iron deficiency anemia was discovered to be related to low psychometric test scores at every age (P < 0.05). Conclusions: Neurodevelopmental problems in children with acyanotic congenital heart disease were found higher compared to those in society. Mental development and intelligence levels of patients were determined to be closely associated with receiving incubator care, father’s educational level and iron deficiency anemia.

What's so funny? An evaluation of the relations among humour use, mirth, and depressive symptomatology

Humour production and showing mirth (i.e., smiling and laughing) confer prosocial advantages. However, there is a paucity of literature evaluating how humour manifests in psychopathology. Humour and mirth may be especially relevant in depression, wherein profound impairments are evident in emotional and social functioning. Chapters 2 and 3 present correlational and predictive relations of depressive, social anxiety, and social anhedonia symptoms with humour styles, and consider the role of motivational systems and expressivity of positive affect as they relate to humour. As expected, symptoms and avoidance-based motivation were positively related to maladaptive humour styles and negatively related to adaptive humour styles. Interestingly, the pattern of relations shifted when considered among individuals in a depressive episode; acutely depressed individuals generally shy away from any humour style rather than gravitating toward specific styles. In a mediation model, the inverse relation between depressive symptoms and affiliative humour was fully mediated by approach-based motivation and expressivity of positive emotions. Chapters 4 and 5 examined subjective and observed mirth responses (facial affect and laughter) demonstrated by depressed and healthy comparison groups. Relative to non-depressed individuals, depressed persons reported less enjoyment, lower ratings of funniness, and fewer instances and shorter durations of positive facial affect and laughter when viewing humourous videos. There was no significant change in retrospective ratings of enjoyment and funniness at a one-week follow-up. The pattern of responsivity by depressed persons shifted when they viewed humourous videos while hearing others laughing. Both groups demonstrated more mirth when hearing others laugh; there were no differences between groups on mirthful behaviours. The one exception was that the total duration of laugher produced by depressed individuals was shorter than that produced by individuals in the healthy comparison group. This research project demonstrates that facets of depressive symptomatology are differentially associated with humour use and depressed individuals show blunted emotional responsivity to humourous stimuli. However, the pattern of reduced affective responsivity is context specific in that it fluctuates in response to hearing others’ laughter. These findings have important implications for the conceptualization of depression and the subsequent avenues for the treatment of individuals with depression.

New dimeric supramolecular structure of mixed (phthalocyaninato)(porphyrinato)-europium(III) sandwiches : preparation and spectroscopic characteristics

The mixed double-decker Eu\[Pc(15C5)4](TPP) (1) was obtained by base-catalysed tetramerisation of 4,5-dicyanobenzo-15-crown-5 using the half-sandwich complex Eu(TPP)(acac) (acac = acetylacetonate), generated in situ, as the template. For comparative studies, the mixed triple-decker complexes Eu2\[Pc(15C5)4](TPP)2 (2) and Eu2\[Pc(15C5)4]2(TPP) (3) were also synthesised by the raise-by-one-story method. These mixed ring sandwich complexes were characterised by various spectroscopic methods. Up to four one-electron oxidations and two one-electron reductions were revealed by cyclic voltammetry (CV) and differential pulse voltammetry (DPV). As shown by electronic absorption and infrared spectroscopy, supramolecular dimers (SM1 and SM3) were formed from the corresponding double-decker 1 and triple-decker 3 in the presence of potassium ions in MeOH/CHCl3.

Electrochemistry of Heteroleptic Tris(phthalocyaninato) Rare Earth(III) Complexes

The electrochemical characteristics of a series of heteroleptic tris(phthalocyaninato) complexes with identical rare earths or mixed rare earths (Pc)M(OOPc)M(OOPc) [M = Eu...Lu, Y; H2Pc = unsubstituted phthalocyanine, H2(OOPc) = 3,4,12,13,21,22,30,31-octakis(octyloxy)phthalocyanine] and (Pc)Eu(OOPc)Er(OOPc) have been recorded and studied comparatively by cyclic voltammetry (CV) and differential pulse voltammetry (DPV) in CH2Cl2 containing 0.1 M tetrabutylammonium perchlorate (TBAP). Up to five quasi-reversible one-electron oxidations and four one-electron reductions have been revealed. The half-wave potentials of the first, second and fifth oxidations depend on the size of the metal center, but the fifth changes in the opposite direction to that of the first two. Moreover, the difference in redox potentials of the first oxidation and first reduction for (Pc)M(OOPc)M(OOPc), 0.85−0.98 V, also decreases linearly along with decreasing rare earth ion radius, clearly showing the rare earth ion size effect and indicating enhanced π−π interactions in the triple-deckers connected by smaller lanthanides. This order follows the red-shift seen in the lowest energy band of triple-decker compounds. The electronic differences between the lanthanides and yttrium are more apparent for triple-decker sandwich complexes than for the analogous double-deckers. By comparing triple-decker, double-decker and mononuclear [ZnII] complexes containing the OOPc ligand, the HOMO−LUMO gap has been shown to contract approximately linearly with the number of stacked phthalocyanine ligands.

The ageing of alumina hydrolysates synthesized from sec-butoxyaluminium(III)

A combination of X-ray diffraction, thermal analysis and Raman spectroscopy was employed to characterise the ageing of alumina hydrolysates synthesised from the hydrolysis of anhydrous tri-sec-butoxyaluminium(III). X-Ray diffraction showed that the alumino-oxy(hydroxy) hydrolysates were pseudoboehmite. For boehmite the lamellar spacings are in the b direction and multiple d(020) peaks are observed for the un-aged hydrolysate. After 4 h of ageing, a single d(020) peak is observed at 6.53 Å. Thermal analysis showed five endotherms at 70, 140, 238, 351 and 445°C. These endotherms are attributed to the dehydration and dehydroxylation of the boehmite-like hydrolysate. Raman spectroscopy shows the presence of bands for the washed hydrolysates at 333, 355, 414, 455, 475, 495, 530 and 675 cm–1. These bands are attributed to pseudoboehmite. Ageing of the hydrolysates results in an increase in the crystallite size of the pseudoboehmite.

Comparative Electrochemical Study of Unsubstituted and Substituted Bis(phthalocyaninato) Rare Earth(III) Complexes

The electrochemistry of homoleptic substituted phthalocyaninato rare earth double-decker complexes M(TBPc)2 and M(OOPc)2 [M = Y, La...Lu except Pm; H2TBPc = 3(4),12(13),21(22),30(31)-tetra-tert-butylphthalocyanine, H2OOPc = 3,4,12,13,21,22,30,31-octakis(octyloxy)phthalocyanine] has been comparatively studied by cyclic voltammetry (CV) and differential pulse voltammetry (DPV) in CH2Cl2 containing 0.1 M tetra-n-butylammonium perchlorate (TBAP). Two quasi-reversible one-electron oxidations and three or four quasi-reversible one-electron reductions have been revealed for these neutral double-deckers of two series of substituted complexes, respectively. For comparison, unsubstituted bis(phthalocyaninato) rare earth analogues M(Pc)2 (M = Y, La...Lu except Pm; H2Pc = phthalocyanine) have also been electrochemically investigated. Two quasi-reversible one-electron oxidations and up to five quasi-reversible one-electron reductions have been revealed for these neutral double-decker compounds. The three bis(phthalocyaninato)cerium compounds display one cerium-centered redox wave between the first ligand-based oxidation and reduction. The half-wave potentials of the first and second oxidations and first reduction for double-deckers of the tervalent rare earths depend on the size of the metal center. The difference between the redox potentials of the second and third reductions for MIII(Pc)2, which represents the potential difference between the first oxidation and first reduction of [MIII(Pc)2]−, lies in the range 1.08−1.37 V and also gradually diminishes along with the lanthanide contraction, indicating enhanced π−π interactions in the double-deckers connected by the smaller, lanthanides. This corresponds well with the red-shift of the lowest energy band observed in the electronic absorption spectra of reduced double-decker [MIII(Pc′)2]− (Pc′ = Pc, TBPc, OOPc).

Vibrational spectroscopy of phthalocyanine and naphthalocyanine in sandwich-type (na)phthalocyaninato and porphyrinato rare earth complexes: Part 14. The infrared and Raman characteristics of phthalocyanine in “pinwheel-like” homoleptic bis[1,8,15,22-tetrakis(3-pentyloxy)phthalocyaninato] rare earth(III) double-decker complexes

The infrared (IR) spectroscopic data and Raman spectroscopic properties for a series of 13 “pinwheel-like” homoleptic bis(phthalocyaninato) rare earth complexes M[Pc(α-OC5H11)4]2 [M = Y and Pr–Lu except Pm; H2Pc(α-OC5H11)4 = 1,8,15,22-tetrakis(3-pentyloxy)phthalocyanine] have been collected and comparatively studied. Both the IR and Raman spectra for M[Pc(α-OC5H11)4]2 are more complicated than those of homoleptic bis(phthalocyaninato) rare earth analogues, namely M(Pc)2 and M[Pc(OC8H17)8]2, but resemble (for IR) or are a bit more complicated (for Raman) than those of heteroleptic counterparts M(Pc)[Pc(α-OC5H11)4], revealing the decreased molecular symmetry of these double-decker compounds, namely S8. Except for the obvious splitting of the isoindole breathing band at 1110–1123 cm−1, the IR spectra of M[Pc(α-OC5H11)4]2 are quite similar to those of corresponding M(Pc)[Pc(α-OC5H11)4] and therefore are similarly assigned. With laser excitation at 633 nm, Raman bands derived from isoindole ring and aza stretchings in the range of 1300–1600 cm−1 are selectively intensified. The IR spectra reveal that the frequencies of pyrrole stretching and pyrrole stretching coupled with the symmetrical CH bending of –CH3 groups are sensitive to the rare earth ionic size, while the Raman technique shows that the bands due to the isoindole stretchings and the coupled pyrrole and aza stretchings are similarly affected. Nevertheless, the phthalocyanine monoanion radical Pc′− IR marker band of bis(phthalocyaninato) complexes involving the same rare earth ion is found to shift to lower energy in the order M(Pc)2 > M(Pc)[Pc(α-OC5H11)4] > M[Pc(α-OC5H11)4]2, revealing the weakened π–π interaction between the two phthalocyanine rings in the same order.