Realization of Differentiated Quality of Service for Wideband Code Division Multiple Access Core Network

The development of 3G (the 3rd generation telecommunication) value-added services brings higher requirements of Quality of Service (QoS). Wideband Code Division Multiple Access (WCDMA) is one of three 3G standards, and enhancement of QoS for WCDMA Core Network (CN) becomes more and more important for users and carriers. The dissertation focuses on enhancement of QoS for WCDMA CN. The purpose is to realize the DiffServ (Differentiated Services) model of QoS for WCDMA CN. Based on the parallelism characteristic of Network Processors (NPs), the NP programming model is classified as Pool of Threads (POTs) and Hyper Task Chaining (HTC). In this study, an integrated programming model that combines both of the two models was designed. This model has highly efficient and flexible features, and also solves the problems of sharing conflicts and packet ordering. We used this model as the programming model to realize DiffServ QoS for WCDMA CN. ^ The realization mechanism of the DiffServ model mainly consists of buffer management, packet scheduling and packet classification algorithms based on NPs. First, we proposed an adaptive buffer management algorithm called Packet Adaptive Fair Dropping (PAFD), which takes into consideration of both fairness and throughput, and has smooth service curves. Then, an improved packet scheduling algorithm called Priority-based Weighted Fair Queuing (PWFQ) was introduced to ensure the fairness of packet scheduling and reduce queue time of data packets. At the same time, the delay and jitter are also maintained in a small range. Thirdly, a multi-dimensional packet classification algorithm called Classification Based on Network Processors (CBNPs) was designed. It effectively reduces the memory access and storage space, and provides less time and space complexity. ^ Lastly, an integrated hardware and software system of the DiffServ model of QoS for WCDMA CN was proposed. It was implemented on the NP IXP2400. According to the corresponding experiment results, the proposed system significantly enhanced QoS for WCDMA CN. It extensively improves consistent response time, display distortion and sound image synchronization, and thus increases network efficiency and saves network resource.^

Hybrid approaches to estimating freeway travel times using point traffic detector data

The accurate and reliable estimation of travel time based on point detector data is needed to support Intelligent Transportation System (ITS) applications. It has been found that the quality of travel time estimation is a function of the method used in the estimation and varies for different traffic conditions. In this study, two hybrid on-line travel time estimation models, and their corresponding off-line methods, were developed to achieve better estimation performance under various traffic conditions, including recurrent congestion and incidents. The first model combines the Mid-Point method, which is a speed-based method, with a traffic flow-based method. The second model integrates two speed-based methods: the Mid-Point method and the Minimum Speed method. In both models, the switch between travel time estimation methods is based on the congestion level and queue status automatically identified by clustering analysis. During incident conditions with rapidly changing queue lengths, shock wave analysis-based refinements are applied for on-line estimation to capture the fast queue propagation and recovery. Travel time estimates obtained from existing speed-based methods, traffic flow-based methods, and the models developed were tested using both simulation and real-world data. The results indicate that all tested methods performed at an acceptable level during periods of low congestion. However, their performances vary with an increase in congestion. Comparisons with other estimation methods also show that the developed hybrid models perform well in all cases. Further comparisons between the on-line and off-line travel time estimation methods reveal that off-line methods perform significantly better only during fast-changing congested conditions, such as during incidents. The impacts of major influential factors on the performance of travel time estimation, including data preprocessing procedures, detector errors, detector spacing, frequency of travel time updates to traveler information devices, travel time link length, and posted travel time range, were investigated in this study. The results show that these factors have more significant impacts on the estimation accuracy and reliability under congested conditions than during uncongested conditions. For the incident conditions, the estimation quality improves with the use of a short rolling period for data smoothing, more accurate detector data, and frequent travel time updates.

A computer simulation model for single -lane roundabouts

During the past three decades, the use of roundabouts has increased throughout the world due to their greater benefits in comparison with intersections controlled by traditional means. Roundabouts are often chosen because they are widely associated with low accident rates, lower construction and operating costs, and reasonable capacities and delay. ^ In the planning and design of roundabouts, special attention should be given to the movement of pedestrians and bicycles. As a result, there are several guidelines for the design of pedestrian and bicycle treatments at roundabouts that increase the safety of both pedestrians and bicyclists at existing and proposed roundabout locations. Different design guidelines have differing criteria for handling pedestrians and bicyclists at roundabout locations. Although all of the investigated guidelines provide better safety (depending on the traffic conditions at a specific location), their effects on the performance of the roundabout have not been examined yet. ^ Existing roundabout analysis software packages provide estimates of capacity and performance characteristics. This includes characteristics such as delay, queue lengths, stop rates, effects of heavy vehicles, crash frequencies, and geometric delays, as well as fuel consumption, pollutant emissions and operating costs for roundabouts. None of these software packages, however, are capable of determining the effects of various pedestrian crossing locations, nor the effect of different bicycle treatments on the performance of roundabouts. ^ The objective of this research is to develop simulation models capable of determining the effect of various pedestrian and bicycle treatments at single-lane roundabouts. To achieve this, four models were developed. The first model simulates a single-lane roundabout without bicycle and pedestrian traffic. The second model simulates a single-lane roundabout with a pedestrian crossing and mixed flow bicyclists. The third model simulates a single-lane roundabout with a combined pedestrian and bicycle crossing, while the fourth model simulates a single-lane roundabout with a pedestrian crossing and a bicycle lane at the outer perimeter of the roundabout for the bicycles. Traffic data was collected at a modern roundabout in Boca Raton, Florida. ^ The results of this effort show that installing a pedestrian crossing on the roundabout approach will have a negative impact on the entry flow, while the downstream approach will benefit from the newly created gaps by pedestrians. Also, it was concluded that a bicycle lane configuration is more beneficial for all users of the roundabout instead of the mixed flow or combined crossing. Installing the pedestrian crossing at one-car length is more beneficial for pedestrians than two- and three-car lengths. Finally, it was concluded that the effect of the pedestrian crossing on the vehicle queues diminishes as the distance between the crossing and the roundabout increases. ^

Optimization of waiting time at toll plazas

Toll plazas have several toll payment types such as manual, automatic coin machines, electronic and mixed lanes. In places with high traffic flow, the presence of toll plaza causes a lot of traffic congestion; this creates a bottleneck for the traffic flow, unless the correct mix of payment types is in operation. The objective of this research is to determine the optimal lane configuration for the mix of the methods of payment so that the waiting time in the queue at the toll plaza is minimized. A queuing model representing the toll plaza system and a nonlinear integer program have been developed to determine the optimal mix. The numerical results show that the waiting time can be decreased at the toll plaza by changing the lane configuration. For the case study developed an improvement in the waiting time as high as 96.37 percent was noticed during the morning peak hour.

Variable Speed Limit Strategies to Reduce the Impacts of Traffic Flow Breakdown at Recurrent Freeway Bottlenecks

Variable Speed Limit (VSL) strategies identify and disseminate dynamic speed limits that are determined to be appropriate based on prevailing traffic conditions, road surface conditions, and weather conditions. This dissertation develops and evaluates a shockwave-based VSL system that uses a heuristic switching logic-based controller with specified thresholds of prevailing traffic flow conditions. The system aims to improve operations and mobility at critical bottlenecks. Before traffic breakdown occurrence, the proposed VSL’s goal is to prevent or postpone breakdown by decreasing the inflow and achieving uniform distribution in speed and flow. After breakdown occurrence, the VSL system aims to dampen traffic congestion by reducing the inflow traffic to the congested area and increasing the bottleneck capacity by deactivating the VSL at the head of the congested area. The shockwave-based VSL system pushes the VSL location upstream as the congested area propagates upstream. In addition to testing the system using infrastructure detector-based data, this dissertation investigates the use of Connected Vehicle trajectory data as input to the shockwave-based VSL system performance. Since the field Connected Vehicle data are not available, as part of this research, Vehicle-to-Infrastructure communication is modeled in the microscopic simulation to obtain individual vehicle trajectories. In this system, wavelet transform is used to analyze aggregated individual vehicles’ speed data to determine the locations of congestion. The currently recommended calibration procedures of simulation models are generally based on the capacity, volume and system-performance values and do not specifically examine traffic breakdown characteristics. However, since the proposed VSL strategies are countermeasures to the impacts of breakdown conditions, considering breakdown characteristics in the calibration procedure is important to have a reliable assessment. Several enhancements were proposed in this study to account for the breakdown characteristics at bottleneck locations in the calibration process. In this dissertation, performance of shockwave-based VSL is compared to VSL systems with different fixed VSL message sign locations utilizing the calibrated microscopic model. The results show that shockwave-based VSL outperforms fixed-location VSL systems, and it can considerably decrease the maximum back of queue and duration of breakdown while increasing the average speed during breakdown.

Delay-sensitive service request scheduling for cloud computing

Cloud computing realizes the long-held dream of converting computing capability into a type of utility. It has the potential to fundamentally change the landscape of the IT industry and our way of life. However, as cloud computing expanding substantially in both scale and scope, ensuring its sustainable growth is a critical problem. Service providers have long been suffering from high operational costs. Especially the costs associated with the skyrocketing power consumption of large data centers. In the meantime, while efficient power/energy utilization is indispensable for the sustainable growth of cloud computing, service providers must also satisfy a user's quality of service (QoS) requirements. This problem becomes even more challenging considering the increasingly stringent power/energy and QoS constraints, as well as other factors such as the highly dynamic, heterogeneous, and distributed nature of the computing infrastructures, etc. ^ In this dissertation, we study the problem of delay-sensitive cloud service scheduling for the sustainable development of cloud computing. We first focus our research on the development of scheduling methods for delay-sensitive cloud services on a single server with the goal of maximizing a service provider's profit. We then extend our study to scheduling cloud services in distributed environments. In particular, we develop a queue-based model and derive efficient request dispatching and processing decisions in a multi-electricity-market environment to improve the profits for service providers. We next study a problem of multi-tier service scheduling. By carefully assigning sub deadlines to the service tiers, our approach can significantly improve resource usage efficiencies with statistically guaranteed QoS. Finally, we study the power conscious resource provision problem for service requests with different QoS requirements. By properly sharing computing resources among different requests, our method statistically guarantees all QoS requirements with a minimized number of powered-on servers and thus the power consumptions. The significance of our research is that it is one part of the integrated effort from both industry and academia to ensure the sustainable growth of cloud computing as it continues to evolve and change our society profoundly.^

Integrity-based kernel malware detection

Kernel-level malware is one of the most dangerous threats to the security of users on the Internet, so there is an urgent need for its detection. The most popular detection approach is misuse-based detection. However, it cannot catch up with today's advanced malware that increasingly apply polymorphism and obfuscation. In this thesis, we present our integrity-based detection for kernel-level malware, which does not rely on the specific features of malware. ^ We have developed an integrity analysis system that can derive and monitor integrity properties for commodity operating systems kernels. In our system, we focus on two classes of integrity properties: data invariants and integrity of Kernel Queue (KQ) requests. ^ We adopt static analysis for data invariant detection and overcome several technical challenges: field-sensitivity, array-sensitivity, and pointer analysis. We identify data invariants that are critical to system runtime integrity from Linux kernel 2.4.32 and Windows Research Kernel (WRK) with very low false positive rate and very low false negative rate. We then develop an Invariant Monitor to guard these data invariants against real-world malware. In our experiment, we are able to use Invariant Monitor to detect ten real-world Linux rootkits and nine real-world Windows malware and one synthetic Windows malware. ^ We leverage static and dynamic analysis of kernel and device drivers to learn the legitimate KQ requests. Based on the learned KQ requests, we build KQguard to protect KQs. At runtime, KQguard rejects all the unknown KQ requests that cannot be validated. We apply KQguard on WRK and Linux kernel, and extensive experimental evaluation shows that KQguard is efficient (up to 5.6% overhead) and effective (capable of achieving zero false positives against representative benign workloads after appropriate training and very low false negatives against 125 real-world malware and nine synthetic attacks). ^ In our system, Invariant Monitor and KQguard cooperate together to protect data invariants and KQs in the target kernel. By monitoring these integrity properties, we can detect malware by its violation of these integrity properties during execution.^

Predicting delay reductions from freeway traffic diversion for incident management

Traffic incidents are a major source of traffic congestion on freeways. Freeway traffic diversion using pre-planned alternate routes has been used as a strategy to reduce traffic delays due to major traffic incidents. However, it is not always beneficial to divert traffic when an incident occurs. Route diversion may adversely impact traffic on the alternate routes and may not result in an overall benefit. This dissertation research attempts to apply Artificial Neural Network (ANN) and Support Vector Regression (SVR) techniques to predict the percent of delay reduction from route diversion to help determine whether traffic should be diverted under given conditions. The DYNASMART-P mesoscopic traffic simulation model was applied to generate simulated data that were used to develop the ANN and SVR models. A sample network that comes with the DYNASMART-P package was used as the base simulation network. A combination of different levels of incident duration, capacity lost, percent of drivers diverted, VMS (variable message sign) messaging duration, and network congestion was simulated to represent different incident scenarios. The resulting percent of delay reduction, average speed, and queue length from each scenario were extracted from the simulation output. The ANN and SVR models were then calibrated for percent of delay reduction as a function of all of the simulated input and output variables. The results show that both the calibrated ANN and SVR models, when applied to the same location used to generate the calibration data, were able to predict delay reduction with a relatively high accuracy in terms of mean square error (MSE) and regression correlation. It was also found that the performance of the ANN model was superior to that of the SVR model. Likewise, when the models were applied to a new location, only the ANN model could produce comparatively good delay reduction predictions under high network congestion level.

Delay-Sensitive Service Request Scheduling for Cloud Computing

Cloud computing realizes the long-held dream of converting computing capability into a type of utility. It has the potential to fundamentally change the landscape of the IT industry and our way of life. However, as cloud computing expanding substantially in both scale and scope, ensuring its sustainable growth is a critical problem. Service providers have long been suffering from high operational costs. Especially the costs associated with the skyrocketing power consumption of large data centers. In the meantime, while efficient power/energy utilization is indispensable for the sustainable growth of cloud computing, service providers must also satisfy a user's quality of service (QoS) requirements. This problem becomes even more challenging considering the increasingly stringent power/energy and QoS constraints, as well as other factors such as the highly dynamic, heterogeneous, and distributed nature of the computing infrastructures, etc. In this dissertation, we study the problem of delay-sensitive cloud service scheduling for the sustainable development of cloud computing. We first focus our research on the development of scheduling methods for delay-sensitive cloud services on a single server with the goal of maximizing a service provider's profit. We then extend our study to scheduling cloud services in distributed environments. In particular, we develop a queue-based model and derive efficient request dispatching and processing decisions in a multi-electricity-market environment to improve the profits for service providers. We next study a problem of multi-tier service scheduling. By carefully assigning sub deadlines to the service tiers, our approach can significantly improve resource usage efficiencies with statistically guaranteed QoS. Finally, we study the power conscious resource provision problem for service requests with different QoS requirements. By properly sharing computing resources among different requests, our method statistically guarantees all QoS requirements with a minimized number of powered-on servers and thus the power consumptions. The significance of our research is that it is one part of the integrated effort from both industry and academia to ensure the sustainable growth of cloud computing as it continues to evolve and change our society profoundly.

Integrity-Based Kernel Malware Detection

Kernel-level malware is one of the most dangerous threats to the security of users on the Internet, so there is an urgent need for its detection. The most popular detection approach is misuse-based detection. However, it cannot catch up with today's advanced malware that increasingly apply polymorphism and obfuscation. In this thesis, we present our integrity-based detection for kernel-level malware, which does not rely on the specific features of malware. We have developed an integrity analysis system that can derive and monitor integrity properties for commodity operating systems kernels. In our system, we focus on two classes of integrity properties: data invariants and integrity of Kernel Queue (KQ) requests. We adopt static analysis for data invariant detection and overcome several technical challenges: field-sensitivity, array-sensitivity, and pointer analysis. We identify data invariants that are critical to system runtime integrity from Linux kernel 2.4.32 and Windows Research Kernel (WRK) with very low false positive rate and very low false negative rate. We then develop an Invariant Monitor to guard these data invariants against real-world malware. In our experiment, we are able to use Invariant Monitor to detect ten real-world Linux rootkits and nine real-world Windows malware and one synthetic Windows malware. We leverage static and dynamic analysis of kernel and device drivers to learn the legitimate KQ requests. Based on the learned KQ requests, we build KQguard to protect KQs. At runtime, KQguard rejects all the unknown KQ requests that cannot be validated. We apply KQguard on WRK and Linux kernel, and extensive experimental evaluation shows that KQguard is efficient (up to 5.6% overhead) and effective (capable of achieving zero false positives against representative benign workloads after appropriate training and very low false negatives against 125 real-world malware and nine synthetic attacks). In our system, Invariant Monitor and KQguard cooperate together to protect data invariants and KQs in the target kernel. By monitoring these integrity properties, we can detect malware by its violation of these integrity properties during execution.

The perihelion activity of comet 67P/Churyumov-Gerasimenko as seen by robotic telescopes

Around the time of its perihelion passage the observability of 67P/Churyumov-Gerasimenko from Earth was limited to very short windows each morning from any given site, due to the low solar elongation of the comet. The peak in the comet's activity was therefore difficult to observe with conventionally scheduled telescopes, but was possible where service/queue scheduled mode was possible, and with robotic telescopes. We describe the robotic observations that allowed us to measure the total activity of the comet around perihelion, via photometry (dust) and spectroscopy (gas), and compare these results with the measurements at this time by Rosetta's instruments. The peak of activity occurred approximately two weeks after perihelion. The total brightness (dust) largely followed the predictions from Snodgrass et al. 2013, with no significant change in total activity levels from previous apparitions. The CN gas production rate matched previous orbits near perihelion, but appeared to be relatively low later in the year.

On Queueinginventory Models – product form solution; reservation, Cancellation and common life time

Queueing theory is the mathematical study of ‘queue’ or ‘waiting lines’ where an item from inventory is provided to the customer on completion of service. A typical queueing system consists of a queue and a server. Customers arrive in the system from outside and join the queue in a certain way. The server picks up customers and serves them according to certain service discipline. Customers leave the system immediately after their service is completed. For queueing systems, queue length, waiting time and busy period are of primary interest to applications. The theory permits the derivation and calculation of several performance measures including the average waiting time in the queue or the system, mean queue length, traffic intensity, the expected number waiting or receiving service, mean busy period, distribution of queue length, and the probability of encountering the system in certain states, such as empty, full, having an available server or having to wait a certain time to be served.

Direct catalytic macrolactonization and application of the phase separation strategy in complex molecule synthesis

Les macrolactones sont des squelettes structuraux importants dans de nombreuses sphères de l’industrie chimique, en particulier dans les marchés pharmaceutiques et cosmétiques. Toutefois, la stratégie traditionnelle pour la préparation de macrolactones demeure incommode en requérant notamment l’ajout (super)stœchiométrique d’agents activateurs. Conséquemment, des quantités stœchiométriques de sous-produits sont générées; ils sont souvent toxiques, dommageables pour l’environnement et nécessitent des méthodes de purification fastidieuses afin de les éliminer. La présente thèse décrit le développement d’une macrolactonisation efficace catalysée au hafnium directement à partir de précurseurs portant un acide carboxylique et un alcool primaire, ne générant que de l’eau comme sous-produit et ne nécessitant pas de techniques d’addition lente et/ou azéotropique. Le protocole a également été adapté à la synthèse directe de macrodiolides à partir de mélanges équimolaires de diols et de diacides carboxyliques et à la synthèse de dimères tête-à-queue de seco acides. Des muscs macrocycliques ainsi que des macrolactones pertinentes à la chimie médicinale ont pu être synthétisés avec l’approche développée. Un protocole pour l’estérification directe catalysée au hafnium entre des acides carboxyliques et des alcools primaires a aussi été développé. Différentes méthodes pour la macrolactonisation catalytique directe entre des alcools secondaires et des acides carboxyliques ont été étudiées. En outre, la stratégie de séparation de phase en macrocyclisation en débit continu a été appliquée lors de la synthèse totale formelle de la macrolactone ivorenolide A. Les étapes-clés de la synthèse incluent une macrocyclisation par le couplage d’alcynes de Glaser-Hay et une réaction de métathèse d’alcènes Z-sélective.

Impact de la modulation de l’expression de la protéine sécrétogranine III sur les fonctions analgésiques du récepteur NTS2

Les récepteurs couplés aux protéines G (RCPG) représentent la famille de récepteurs membranaires la plus étendue. Présentement, 30% des médicaments disponibles sur le marché agissent sur plus ou moins 4% des RCPG connus. Cela indique qu’il existe encore une multitude de RCPG à explorer, qui pourraient démontrer un potentiel thérapeutique intéressant pour des pathologies encore sans traitement disponible. La douleur chronique, qui affecte 1 canadien sur 5, fait partie de ces affections pour lesquelles les thérapies sont faiblement efficaces. Dans le but éventuel de pallier à ce problème, le projet de ce mémoire consistait à mieux comprendre les mécanismes régissant l’adressage membranaire du récepteur de la neurotensine de type 2 (NTS2), un RCPG dont l’activation induit des effets analgésiques puissants de type non opioïdergique. Des données du laboratoire ayant révélé une interaction entre la 3e boucle intracellulaire de NTS2 et la sécrétogranine III (SgIII), une protéine résidente de la voie de sécrétion régulée, notre objectif était de caractériser le rôle de SgIII dans la fonctionnalité du récepteur NTS2. Pour ce faire, l’interaction entre les deux protéines a d’abord été vérifiée par co-immunoprécipitation, GST pull down, essais de colocalisation subcellulaire et par FRET. Nous avons également utilisé un outil d’interférence à l’ARN (DsiRNA) pour invalider la protéine SgIII dans un modèle cellulaire neuronal et chez l’animal. Des essais de radioliaison sur le modèle cellulaire ont révélé une diminution de l’adressage de NTS2 à la membrane plasmique lorsque SgIII était supprimée. De la même façon, une invalidation de SgIII chez le rat inhibe les effets analgésiques d’un agoniste NTS2-sélectif (JMV-431), qui sont normalement observés dans le test de retrait de la queue (douleur aiguë). Nous avons cependant identifié que des stimulations au KCl, à la capsaïcine et à la neurotensine induisaient plutôt une insertion du récepteur à la membrane dans les cellules neuronales. Puisque l’interaction entre SgIII et la 3e boucle intracellulaire du récepteur NTS2 est peu probable selon un système de sécrétion classique, un modèle hypothétique de transition dans les corps multi-vésiculaires a été vérifié. Ainsi, la présence de NTS2 et de SgIII a été révélée dans des préparations exosomales de DRG F11, en plus d’une transférabilité du récepteur par le milieu extracellulaire. En somme, ces résultats démontrent la dépendance du récepteur NTS2 envers SgIII et la voie de sécrétion régulée, en plus d’identifier certains facteurs pouvant augmenter son expression à la surface cellulaire. Le projet contribue ainsi à ouvrir la voie vers des approches pour potentialiser les propriétés de NTS2 in vivo¸ en plus d’une piste d’explication concernant le trafic intracellulaire du récepteur.

Using interaction data for improving the offline and online evaluation of search engines

This thesis investigates how web search evaluation can be improved using historical interaction data. Modern search engines combine offline and online evaluation approaches in a sequence of steps that a tested change needs to pass through to be accepted as an improvement and subsequently deployed. We refer to such a sequence of steps as an evaluation pipeline. In this thesis, we consider the evaluation pipeline to contain three sequential steps: an offline evaluation step, an online evaluation scheduling step, and an online evaluation step. In this thesis we show that historical user interaction data can aid in improving the accuracy or efficiency of each of the steps of the web search evaluation pipeline. As a result of these improvements, the overall efficiency of the entire evaluation pipeline is increased. Firstly, we investigate how user interaction data can be used to build accurate offline evaluation methods for query auto-completion mechanisms. We propose a family of offline evaluation metrics for query auto-completion that represents the effort the user has to spend in order to submit their query. The parameters of our proposed metrics are trained against a set of user interactions recorded in the search engine’s query logs. From our experimental study, we observe that our proposed metrics are significantly more correlated with an online user satisfaction indicator than the metrics proposed in the existing literature. Hence, fewer changes will pass the offline evaluation step to be rejected after the online evaluation step. As a result, this would allow us to achieve a higher efficiency of the entire evaluation pipeline. Secondly, we state the problem of the optimised scheduling of online experiments. We tackle this problem by considering a greedy scheduler that prioritises the evaluation queue according to the predicted likelihood of success of a particular experiment. This predictor is trained on a set of online experiments, and uses a diverse set of features to represent an online experiment. Our study demonstrates that a higher number of successful experiments per unit of time can be achieved by deploying such a scheduler on the second step of the evaluation pipeline. Consequently, we argue that the efficiency of the evaluation pipeline can be increased. Next, to improve the efficiency of the online evaluation step, we propose the Generalised Team Draft interleaving framework. Generalised Team Draft considers both the interleaving policy (how often a particular combination of results is shown) and click scoring (how important each click is) as parameters in a data-driven optimisation of the interleaving sensitivity. Further, Generalised Team Draft is applicable beyond domains with a list-based representation of results, i.e. in domains with a grid-based representation, such as image search. Our study using datasets of interleaving experiments performed both in document and image search domains demonstrates that Generalised Team Draft achieves the highest sensitivity. A higher sensitivity indicates that the interleaving experiments can be deployed for a shorter period of time or use a smaller sample of users. Importantly, Generalised Team Draft optimises the interleaving parameters w.r.t. historical interaction data recorded in the interleaving experiments. Finally, we propose to apply the sequential testing methods to reduce the mean deployment time for the interleaving experiments. We adapt two sequential tests for the interleaving experimentation. We demonstrate that one can achieve a significant decrease in experiment duration by using such sequential testing methods. The highest efficiency is achieved by the sequential tests that adjust their stopping thresholds using historical interaction data recorded in diagnostic experiments. Our further experimental study demonstrates that cumulative gains in the online experimentation efficiency can be achieved by combining the interleaving sensitivity optimisation approaches, including Generalised Team Draft, and the sequential testing approaches. Overall, the central contributions of this thesis are the proposed approaches to improve the accuracy or efficiency of the steps of the evaluation pipeline: the offline evaluation frameworks for the query auto-completion, an approach for the optimised scheduling of online experiments, a general framework for the efficient online interleaving evaluation, and a sequential testing approach for the online search evaluation. The experiments in this thesis are based on massive real-life datasets obtained from Yandex, a leading commercial search engine. These experiments demonstrate the potential of the proposed approaches to improve the efficiency of the evaluation pipeline.