Fast rates for estimation error and oracle inequalities for model section

We consider complexity penalization methods for model selection. These methods aim to choose a model to optimally trade off estimation and approximation errors by minimizing the sum of an empirical risk term and a complexity penalty. It is well known that if we use a bound on the maximal deviation between empirical and true risks as a complexity penalty, then the risk of our choice is no more than the approximation error plus twice the complexity penalty. There are many cases, however, where complexity penalties like this give loose upper bounds on the estimation error. In particular, if we choose a function from a suitably simple convex function class with a strictly convex loss function, then the estimation error (the difference between the risk of the empirical risk minimizer and the minimal risk in the class) approaches zero at a faster rate than the maximal deviation between empirical and true risks. In this paper, we address the question of whether it is possible to design a complexity penalized model selection method for these situations. We show that, provided the sequence of models is ordered by inclusion, in these cases we can use tight upper bounds on estimation error as a complexity penalty. Surprisingly, this is the case even in situations when the difference between the empirical risk and true risk (and indeed the error of any estimate of the approximation error) decreases much more slowly than the complexity penalty. We give an oracle inequality showing that the resulting model selection method chooses a function with risk no more than the approximation error plus a constant times the complexity penalty.

Local Rademacher complexities

We propose new bounds on the error of learning algorithms in terms of a data-dependent notion of complexity. The estimates we establish give optimal rates and are based on a local and empirical version of Rademacher averages, in the sense that the Rademacher averages are computed from the data, on a subset of functions with small empirical error. We present some applications to classification and prediction with convex function classes, and with kernel classes in particular.

Classification with a reject option using a hinge loss

We consider the problem of binary classification where the classifier can, for a particular cost, choose not to classify an observation. Just as in the conventional classification problem, minimization of the sample average of the cost is a difficult optimization problem. As an alternative, we propose the optimization of a certain convex loss function φ, analogous to the hinge loss used in support vector machines (SVMs). Its convexity ensures that the sample average of this surrogate loss can be efficiently minimized. We study its statistical properties. We show that minimizing the expected surrogate loss—the φ-risk—also minimizes the risk. We also study the rate at which the φ-risk approaches its minimum value. We show that fast rates are possible when the conditional probability P(Y=1|X) is unlikely to be close to certain critical values.

Exponentiated gradient algorithms for conditional random fields and max-margin Markov Networks

Log-linear and maximum-margin models are two commonly-used methods in supervised machine learning, and are frequently used in structured prediction problems. Efficient learning of parameters in these models is therefore an important problem, and becomes a key factor when learning from very large data sets. This paper describes exponentiated gradient (EG) algorithms for training such models, where EG updates are applied to the convex dual of either the log-linear or max-margin objective function; the dual in both the log-linear and max-margin cases corresponds to minimizing a convex function with simplex constraints. We study both batch and online variants of the algorithm, and provide rates of convergence for both cases. In the max-margin case, O(1/ε) EG updates are required to reach a given accuracy ε in the dual; in contrast, for log-linear models only O(log(1/ε)) updates are required. For both the max-margin and log-linear cases, our bounds suggest that the online EG algorithm requires a factor of n less computation to reach a desired accuracy than the batch EG algorithm, where n is the number of training examples. Our experiments confirm that the online algorithms are much faster than the batch algorithms in practice. We describe how the EG updates factor in a convenient way for structured prediction problems, allowing the algorithms to be efficiently applied to problems such as sequence learning or natural language parsing. We perform extensive evaluation of the algorithms, comparing them to L-BFGS and stochastic gradient descent for log-linear models, and to SVM-Struct for max-margin models. The algorithms are applied to a multi-class problem as well as to a more complex large-scale parsing task. In all these settings, the EG algorithms presented here outperform the other methods.

Sparseness vs estimating conditional probabilities : some asymptotic results

One of the nice properties of kernel classifiers such as SVMs is that they often produce sparse solutions. However, the decision functions of these classifiers cannot always be used to estimate the conditional probability of the class label. We investigate the relationship between these two properties and show that these are intimately related: sparseness does not occur when the conditional probabilities can be unambiguously estimated. We consider a family of convex loss functions and derive sharp asymptotic results for the fraction of data that becomes support vectors. This enables us to characterize the exact trade-off between sparseness and the ability to estimate conditional probabilities for these loss functions.

Adaptive online gradient descent

We study the rates of growth of the regret in online convex optimization. First, we show that a simple extension of the algorithm of Hazan et al eliminates the need for a priori knowledge of the lower bound on the second derivatives of the observed functions. We then provide an algorithm, Adaptive Online Gradient Descent, which interpolates between the results of Zinkevich for linear functions and of Hazan et al for strongly convex functions, achieving intermediate rates between [square root T] and [log T]. Furthermore, we show strong optimality of the algorithm. Finally, we provide an extension of our results to general norms.

Learning the Kernel Matrix with Semi-Definite Programming

Kernel-based learning algorithms work by embedding the data into a Euclidean space, and then searching for linear relations among the embedded data points. The embedding is performed implicitly, by specifying the inner products between each pair of points in the embedding space. This information is contained in the so-called kernel matrix, a symmetric and positive definite matrix that encodes the relative positions of all points. Specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space -- classical model selection problems in machine learning. In this paper we show how the kernel matrix can be learned from data via semi-definite programming (SDP) techniques. When applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm -- using the labelled part of the data one can learn an embedding also for the unlabelled part. The similarity between test points is inferred from training points and their labels. Importantly, these learning problems are convex, so we obtain a method for learning both the model class and the function without local minima. Furthermore, this approach leads directly to a convex method to learn the 2-norm soft margin parameter in support vector machines, solving another important open problem. Finally, the novel approach presented in the paper is supported by positive empirical results.

Natural convection in a triangular enclosure due to non-uniform cooling on top

Natural convection in a triangular enclosure subject to non-uniformly cooling at the inclined surfaces and uniformly heating at the base is investigated numerically. The numerical simulations of the unsteady flows over a range of Rayleigh numbers and aspect ratios are carried out using Finite Volume Method. Since the upper surface is cooled and the bottom surface is heated, the air flow in the enclosure is potentially unstable to Rayleigh Benard instability. It is revealed that the transient flow development in the enclosure can be classified into three distinct stages; an early stage, a transitional stage and a steady stage. It is also found that the flow inside the enclosure strongly depends on the governing parameters, Rayleigh number and aspect ratio. The asymmetric behaviour of the flow about the geometric centre line is discussed in detailed. The heat transfer through the roof and the ceiling as a form of Nusselt number is also reported in this study.

Exercise for breast cancer survivors: bridging the gap between evidence and practice

Evidence supporting the benefits of exercise following the diagnosis of breast cancer is overwhelming and compelling. Exercise reduces the severity and number of treatment-related side effects, optimizes quality of life during and following treatment, and may optimize survival. Yet, exercise does not uniformly form part of the standards of care provided to women following a breast cancer diagnosis. This commentary summarizes the evidence in support of exercise as a form of adjuvant treatment and identifies and discusses potential issues preventing the formal integration of exercise into breast cancer care. Proposed within the commentary is a model of breast cancer care that incorporates exercise prescription as a key component but also integrates the need for surveillance and management for common breast cancer treatment-related morbidities, as well as education. While future research evaluating the potential cost savings through implementation of such amodel is required, a committed, collaborative approach by clinicians, allied health professionals, and researchers will be instrumental in bridging the gap between research and practice.

Graph rigidity for near-coplanar structure from motion

Recent algorithms for monocular motion capture (MoCap) estimate weak-perspective camera matrices between images using a small subset of approximately-rigid points on the human body (i.e. the torso and hip). A problem with this approach, however, is that these points are often close to coplanar, causing canonical linear factorisation algorithms for rigid structure from motion (SFM) to become extremely sensitive to noise. In this paper, we propose an alternative solution to weak-perspective SFM based on a convex relaxation of graph rigidity. We demonstrate the success of our algorithm on both synthetic and real world data, allowing for much improved solutions to marker less MoCap problems on human bodies. Finally, we propose an approach to solve the two-fold ambiguity over bone direction using a k-nearest neighbour kernel density estimator.

Polymer self assembly on carbon nanotubes

This chapter analyses the poly(3-hexylthiophene) self-assembly on carbon nanotubes and the interaction between the two materials forming a new hybrid nanostructure. The chapter starts with a review of the several studies investigating polymers and biomolecules self-assembled on nanotubes. Then conducting polymers and polythiophenes are briefly introduced. Accordingly, carbon nanotube structure and properties are reported in Sect. 3. The experimental section starts with the bulk characterisation of polymer thin films with the inclusion of uniformly distributed carbon nanotubes. By using volume film analysis techniques (AFM, TEM, UV–Vis and Raman), we show how the polymer’s higher degree of order is a direct consequence of interaction with carbon nanotubes. Nevertheless, it is through the use of nanoscale analysis and molecular dynamic simulations that the self-assembly of the polymer on the nanotube surface can be clearly evidenced and characterised. In Sect. 6, the effect of the carbon templating structure on the P3HT organisation on the surface is investigated, showing the chirality-driven polymer assembly on the carbon nanotube surface. The interaction between P3HT and CNTs brings also to charge transfer, with the modification of physical properties for both species. In particular, the alteration of the polymer electronic properties and the modification of the nanotube mechanical structure are a direct consequence of the P3HT p-p stacking on the nanotube surface. Finally, some considerations based on molecular dynamics studies are reported in order to confirm and support the experimental results discussed.

Influence of rainfall and catchment characteristics on urban stormwater quality

This thesis presents the outcomes of a comprehensive research study undertaken to investigate the influence of rainfall and catchment characteristics on urban stormwater quality. The knowledge created is expected to contribute to a greater understanding of urban stormwater quality and thereby enhance the design of stormwater quality treatment systems. The research study was undertaken based on selected urban catchments in Gold Coast, Australia. The research methodology included field investigations, laboratory testing, computer modelling and data analysis. Both univariate and multivariate data analysis techniques were used to investigate the influence of rainfall and catchment characteristics on urban stormwater quality. The rainfall characteristics investigated included average rainfall intensity and rainfall duration whilst catchment characteristics included land use, impervious area percentage, urban form and pervious area location. The catchment scale data for the analysis was obtained from four residential catchments, including rainfall-runoff records, drainage network data, stormwater quality data and land use and land cover data. Pollutants build-up samples were collected from twelve road surfaces in residential, commercial and industrial land use areas. The relationships between rainfall characteristics, catchment characteristics and urban stormwater quality were investigated based on residential catchments and then extended to other land uses. Based on the influence rainfall characteristics exert on urban stormwater quality, rainfall events can be classified into three different types, namely, high average intensity-short duration (Type 1), high average intensity-long duration (Type 2) and low average intensity-long duration (Type 3). This provides an innovative approach to conventional modelling which does not commonly relate stormwater quality to rainfall characteristics. Additionally, it was found that the threshold intensity for pollutant wash-off from urban catchments is much less than for rural catchments. High average intensity-short duration rainfall events are cumulatively responsible for the generation of a major fraction of the annual pollutants load compared to the other rainfall event types. Additionally, rainfall events less than 1 year ARI such as 6- month ARI should be considered for treatment design as they generate a significant fraction of the annual runoff volume and by implication a significant fraction of the pollutants load. This implies that stormwater treatment designs based on larger rainfall events would not be feasible in the context of cost-effectiveness, efficiency in treatment performance and possible savings in land area needed. This also suggests that the simulation of long-term continuous rainfall events for stormwater treatment design may not be needed and that event based simulations would be adequate. The investigations into the relationship between catchment characteristics and urban stormwater quality found that other than conventional catchment characteristics such as land use and impervious area percentage, other catchment characteristics such as urban form and pervious area location also play important roles in influencing urban stormwater quality. These outcomes point to the fact that the conventional modelling approach in the design of stormwater quality treatment systems which is commonly based on land use and impervious area percentage would be inadequate. It was also noted that the small uniformly urbanised areas within a larger mixed catchment produce relatively lower variations in stormwater quality and as expected lower runoff volume with the opposite being the case for large mixed use urbanised catchments. Therefore, a decentralised approach to water quality treatment would be more effective rather than an "end-of-pipe" approach. The investigation of pollutants build-up on different land uses showed that pollutant build-up characteristics vary even within the same land use. Therefore, the conventional approach in stormwater quality modelling, which is based solely on land use, may prove to be inappropriate. Industrial land use has relatively higher variability in maximum pollutant build-up, build-up rate and particle size distribution than the other two land uses. However, commercial and residential land uses had relatively higher variations of nutrients and organic carbon build-up. Additionally, it was found that particle size distribution had a relatively higher variability for all three land uses compared to the other build-up parameters. The high variability in particle size distribution for all land uses illustrate the dissimilarities associated with the fine and coarse particle size fractions even within the same land use and hence the variations in stormwater quality in relation to pollutants adsorbing to different sizes of particles.

Optical and microscopic study of a polymer-nanotube mixture for organic photovoltaic applications

Organic solar cells based on bulk heterojunction between a conductive polymer and a carbon nanostructure offer potential advantages compared to conventional inorganic cells. Low cost, light weight, flexibility and high peak power per unit weight are all features that can be considered a reality for organic photovoltaics. Although polymer/carbon nanotubes solar cells have been proposed, only low power conversion efficiencies have been reached without addressing the mechanisms responsible for this poor performance. The purpose of this work is therefore to investigate the basic interaction between carbon nanotubes and poly(3-hexylthiophene) in order to demonstrate how this interaction affects the performance of photovoltaic devices. The outcomes of this study are the contributions made to the knowledge of the phenomena explaining the behaviour of electronic devices based on carbon nanotubes and poly(3-hexylthiophene). In this PhD, polymer thin films with the inclusion of uniformly distributed carbon nanotubes were deposited from solution and characterised. The bulk properties of the composites were studied with microscopy and spectroscopy techniques to provide evidence of higher degrees of polymer order when interacting with carbon nanotubes. Although bulk investigation techniques provided useful information about the interaction between the polymer and the nanotubes, clear evidence of the phenomena affecting the heterojunction formed between the two species was investigated at nanoscale. Identifying chirality-driven polymer assisted assembly on the carbon nanotube surface was one of the major achievements of this study. Moreover, the analysis of the electrical behaviour of the heterojunction between the polymer and the nanotube highlighted the charge transfer responsible for the low performance of photovoltaic devices. Polymer and carbon nanotube composite-based devices were fabricated and characterised in order to study their electronic properties. The carbon nanotube introduction in the polymer matrix evidenced a strong electrical conductivity enhancement but also a lower photoconductivity response. Moreover, the extension of pristine polymer device characterisation models to composites based devices evidenced the conduction mechanisms related to nanotubes. Finally, the introduction of carbon nanotubes in the polymer matrix was demonstrated to improve the pristine polymer solar cell performance and the spectral response even though the power conversion efficiency is still too low.

Measuring bitrate and quality trade-off in a fast region-of-interest based video coding

Prevailing video adaptation solutions change the quality of the video uniformly throughout the whole frame in the bitrate adjustment process; while region-of-interest (ROI)-based solutions selectively retains the quality in the areas of the frame where the viewers are more likely to pay more attention to. ROI-based coding can improve perceptual quality and viewer satisfaction while trading off some bandwidth. However, there has been no comprehensive study to measure the bitrate vs. perceptual quality trade-off so far. The paper proposes an ROI detection scheme for videos, which is characterized with low computational complexity and robustness, and measures the bitrate vs. quality trade-off for ROI-based encoding using a state-of-the-art H.264/AVC encoder to justify the viability of this type of encoding method. The results from the subjective quality test reveal that ROI-based encoding achieves a significant perceptual quality improvement over the encoding with uniform quality at the cost of slightly more bits. Based on the bitrate measurements and subjective quality assessments, the bitrate and the perceptual quality estimation models for non-scalable ROI-based video coding (AVC) are developed, which are found to be similar to the models for scalable video coding (SVC).

Diversified service provisioning in global business networks

Companies and their services are being increasingly exposed to global business networks and Internet-based ondemand services. Much of the focus is on flexible orchestration and consumption of services, beyond ownership and operational boundaries of services. However, ways in which third-parties in the “global village” can seamlessly self-create new offers out of existing services remains open. This paper proposes a framework for service provisioning in global business networks that allows an open-ended set of techniques for extending services through a rich, multi-tooling environment. The Service Provisioning Management Framework, as such, supports different modeling techniques, through supportive tools, allowing different parts of services to be integrated into new contexts. Integration of service user interfaces, business processes, operational interfaces and business object are supported. The integration specifications that arise from service extensions are uniformly reflected through a kernel technique, the Service Integration Technique. Thus, the framework preserves coherence of service provisioning tasks without constraining the modeling techniques needed for extending different aspects of services.