Convexity, Classification, and Risk Bounds

Many of the classification algorithms developed in the machine learning literature, including the support vector machine and boosting, can be viewed as minimum contrast methods that minimize a convex surrogate of the 0–1 loss function. The convexity makes these algorithms computationally efficient. The use of a surrogate, however, has statistical consequences that must be balanced against the computational virtues of convexity. To study these issues, we provide a general quantitative relationship between the risk as assessed using the 0–1 loss and the risk as assessed using any nonnegative surrogate loss function. We show that this relationship gives nontrivial upper bounds on excess risk under the weakest possible condition on the loss function—that it satisfies a pointwise form of Fisher consistency for classification. The relationship is based on a simple variational transformation of the loss function that is easy to compute in many applications. We also present a refined version of this result in the case of low noise, and show that in this case, strictly convex loss functions lead to faster rates of convergence of the risk than would be implied by standard uniform convergence arguments. Finally, we present applications of our results to the estimation of convergence rates in function classes that are scaled convex hulls of a finite-dimensional base class, with a variety of commonly used loss functions.

Correction to “The Importance of Convexity in Learning With Squared Loss” [Sep 98 1974-1980]

The paper "the importance of convexity in learning with squared loss" gave a lower bound on the sample complexity of learning with quadratic loss using a nonconvex function class. The proof contains an error. We show that the lower bound is true under a stronger condition that holds for many cases of interest.

Total factor productivity growth and convergence in the petroleum industry : empirical analysis testing for convexity

While economic theory acknowledges that some features of technology (e.g., indivisibilities, economies of scale and specialization) can fundamentally violate the traditional convexity assumption, almost all empirical studies accept the convexity property on faith. In this contribution, we apply two alternative flexible production technologies to measure total factor productivity growth and test the significance of the convexity axiom using a nonparametric test of closeness between unknown distributions. Based on unique field level data on the petroleum industry, the empirical results reveal significant differences, indicating that this production technology is most likely non-convex. Furthermore, we also show the impact of convexity on answers to traditional convergence questions in the productivity growth literature.

Comparison of fractional and geodesic anisotropy in diffusion tensor images of 90 monozygotic and dizygotic twins

We used diffusion tensor magnetic resonance imaging (DTI) to reveal the extent of genetic effects on brain fiber microstructure, based on tensor-derived measures, in 22 pairs of monozygotic (MZ) twins and 23 pairs of dizygotic (DZ) twins (90 scans). After Log-Euclidean denoising to remove rank-deficient tensors, DTI volumes were fluidly registered by high-dimensional mapping of co-registered MP-RAGE scans to a geometrically-centered mean neuroanatomical template. After tensor reorientation using the strain of the 3D fluid transformation, we computed two widely used scalar measures of fiber integrity: fractional anisotropy (FA), and geodesic anisotropy (GA), which measures the geodesic distance between tensors in the symmetric positive-definite tensor manifold. Spatial maps of intraclass correlations (r) between MZ and DZ twins were compared to compute maps of Falconer's heritability statistics, i.e. the proportion of population variance explainable by genetic differences among individuals. Cumulative distribution plots (CDF) of effect sizes showed that the manifold measure, GA, comparably the Euclidean measure, FA, in detecting genetic correlations. While maps were relatively noisy, the CDFs showed promise for detecting genetic influences on brain fiber integrity as the current sample expands.

Classification with a reject option using a hinge loss

We consider the problem of binary classification where the classifier can, for a particular cost, choose not to classify an observation. Just as in the conventional classification problem, minimization of the sample average of the cost is a difficult optimization problem. As an alternative, we propose the optimization of a certain convex loss function φ, analogous to the hinge loss used in support vector machines (SVMs). Its convexity ensures that the sample average of this surrogate loss can be efficiently minimized. We study its statistical properties. We show that minimizing the expected surrogate loss—the φ-risk—also minimizes the risk. We also study the rate at which the φ-risk approaches its minimum value. We show that fast rates are possible when the conditional probability P(Y=1|X) is unlikely to be close to certain critical values.

Relics

Relics is a single-channel video derived from a 3D computer animation that combines a range of media including photography, drawing, painting, and pre-shot video. It is constructed around a series of pictorial stills which become interlinked by the more traditionally filmic processes of panning, zooming and crane shots. In keeping with these ideas, the work revolves around a series of static architectural forms within the strangely menacing enclosure of a geodesic dome. These clinical aspects of the work are complemented by a series of elements that evoke fluidity : fireworks, mirrored biomorphic forms and oscillating projections. The visual dimension of the work is complemented by a soundtrack of rainforest bird calls. Through its ambiguous combination of recorded and virtual imagery, Relics explores the indeterminate boundaries between real and virtual space. On the one hand, it represents actual events and spaces drawn from the artist studio and image archive; on the other it represents the highly idealised spaces of drawing and 3D animation. In this work the disembodied wandering virtual eye is met with an uncanny combination of scenes, where scale and the relationships between objects are disrupted and changed. Through this simultaneity between the real and the virtual, the work conveys a disembodied sense of space and time that carries a powerful sense of affect. Relics was among the first international examples of 3D animation technology in contemporary art. It was originally exhibited in the artist’s solo show, ‘Places That Don’t Exist’ (2007, George Petelin Gallery, Gold Coast) and went on to be included in the group shows ‘d/Art 07/Screen: The Post Cinema Experience’ (2007, Chauvel Cinema, Sydney) , ‘Experimenta Utopia Now: International Biennial of Media Art’ (2010, Arts Centre, Melbourne and national touring venues) and ‘Move on Asia’ (2009, Alternative space Loop, Seoul and Para-site Art Space, Hong Kong) and was broadcast on Souvenirs from Earth (Video Art Cable Channel, Germany and France). The work was analysed in catalogue texts for ‘Places That Don’t Exist’ (2007), ‘d/Art 07’ (2007) and ‘Experimenta Utopia Now’ (2010) and the’ Souvenirs from Earth’ website.

Segmental torso masses and coronal plane joint torques in the adolescent scoliotic spine

Introduction. Calculating segmental (vertebral level-by-level) torso masses in Adolescent Idiopathic Scoliosis (AIS) patients allows the gravitational loading on the scoliotic spine during relaxed standing to be determined. This study used CT scans of AIS patients to measure segmental torso masses and explores how joint moments in the coronal plane are affected by changes in the position of the intervertebral joint’s axis of rotation; particularly at the apex of a scoliotic major curve. Methods. Existing low dose CT data from the Paediatric Spine Research Group was used to calculate vertebral level-by-level torso masses and joint torques occurring in the spine for a group of 20 female AIS patients (mean age 15.0 ± 2.7 years, mean Cobb angle 53 ± 7.1°). Image processing software, ImageJ (v1.45 NIH USA) was used to threshold the T1 to L5 CT images and calculate the segmental torso volume and mass corresponding to each vertebral level. Body segment masses for the head, neck and arms were taken from published anthropometric data. Intervertebral (IV) joint torques at each vertebral level were found using principles of static equilibrium together with the segmental body mass data. Summing the torque contributions for each level above the required joint, allowed the cumulative joint torque at a particular level to be found. Since there is some uncertainty in the position of the coronal plane Instantaneous Axis of Rotation (IAR) for scoliosis patients, it was assumed the IAR was located in the centre of the IV disc. A sensitivity analysis was performed to see what effect the IAR had on the joint torques by moving it laterally 10mm in both directions. Results. The magnitude of the torso masses from T1-L5 increased inferiorly, with a 150% increase in mean segmental torso mass from 0.6kg at T1 to 1.5kg at L5. The magnitudes of the calculated coronal plane joint torques during relaxed standing were typically 5-7 Nm at the apex of the curve, with the highest apex joint torque of 7Nm being found in patient 13. Shifting the assumed IAR by 10mm towards the convexity of the spine, increased the joint torque at that level by a mean 9.0%, showing that calculated joint torques were moderately sensitive to the assumed IAR location. When the IAR midline position was moved 10mm away from the convexity of the spine, the joint torque reduced by a mean 8.9%. Conclusion. Coronal plane joint torques as high as 7Nm can occur during relaxed standing in scoliosis patients, which may help to explain the mechanics of AIS progression. This study provides new anthropometric reference data on vertebral level-by-level torso mass in AIS patients which will be useful for biomechanical models of scoliosis progression and treatment. However, the CT scans were performed in supine (no gravitational load on spine) and curve magnitudes are known to be smaller than those measured in standing.

Genetic influences on brain asymmetry: A DTI study of 374 twins and siblings

Brain asymmetry, or the structural and functional specialization of each brain hemisphere, has fascinated neuroscientists for over a century. Even so, genetic and environmental factors that influence brain asymmetry are largely unknown. Diffusion tensor imaging (DTI) now allows asymmetry to be studied at a microscopic scale by examining differences in fiber characteristics across hemispheres rather than differences in structure shapes and volumes. Here we analyzed 4. Tesla DTI scans from 374 healthy adults, including 60 monozygotic twin pairs, 45 same-sex dizygotic pairs, and 164 mixed-sex DZ twins and their siblings; mean age: 24.4 years ± 1.9 SD). All DTI scans were nonlinearly aligned to a geometrically-symmetric, population-based image template. We computed voxel-wise maps of significant asymmetries (left/right differences) for common diffusion measures that reflect fiber integrity (fractional and geodesic anisotropy; FA, GA and mean diffusivity, MD). In quantitative genetic models computed from all same-sex twin pairs (N=210 subjects), genetic factors accounted for 33% of the variance in asymmetry for the inferior fronto-occipital fasciculus, 37% for the anterior thalamic radiation, and 20% for the forceps major and uncinate fasciculus (all L > R). Shared environmental factors accounted for around 15% of the variance in asymmetry for the cortico-spinal tract (R > L) and about 10% for the forceps minor (L > R). Sex differences in asymmetry (men > women) were significant, and were greatest in regions with prominent FA asymmetries. These maps identify heritable DTI-derived features, and may empower genome-wide searches for genetic polymorphisms that influence brain asymmetry.

Reducing structural variation to determine the genetics of white matter integrity across hemispheres - a dti study of 100 twins

Studies of cerebral asymmetry can open doors to understanding the functional specialization of each brain hemisphere, and how this is altered in disease. Here we examined hemispheric asymmetries in fiber architecture using diffusion tensor imaging (DTI) in 100 subjects, using high-dimensional fluid warping to disentangle shape differences from measures sensitive to myelination. Confounding effects of purely structural asymmetries were reduced by using co-registered structural images to fluidly warp 3D maps of fiber characteristics (fractional and geodesic anisotropy) to a structurally symmetric minimal deformation template (MDT). We performed a quantitative genetic analysis on 100 subjects to determine whether the sources of the remaining signal asymmetries were primarily genetic or environmental. A twin design was used to identify the heritable features of fiber asymmetry in various regions of interest, to further assist in the discovery of genes influencing brain micro-architecture and brain lateralization. Genetic influences and left/right asymmetries were detected in the fiber architecture of the frontal lobes, with minor differences depending on the choice of registration template.

The multivariate A/C/E model and the genetics of fiber architecture

We present a new algorithm to compute the voxel-wise genetic contribution to brain fiber microstructure using diffusion tensor imaging (DTI) in a dataset of 25 monozygotic (MZ) twins and 25 dizygotic (DZ) twin pairs (100 subjects total). First, the structural and DT scans were linearly co-registered. Structural MR scans were nonlinearly mapped via a 3D fluid transformation to a geometrically centered mean template, and the deformation fields were applied to the DTI volumes. After tensor re-orientation to realign them to the anatomy, we computed several scalar and multivariate DT-derived measures including the geodesic anisotropy (GA), the tensor eigenvalues and the full diffusion tensors. A covariance-weighted distance was measured between twins in the Log-Euclidean framework [2], and used as input to a maximum-likelihood based algorithm to compute the contributions from genetics (A), common environmental factors (C) and unique environmental ones (E) to fiber architecture. Quanititative genetic studies can take advantage of the full information in the diffusion tensor, using covariance weighted distances and statistics on the tensor manifold.

Multivariate variance-components analysis in DTI

Twin studies are a major research direction in imaging genetics, a new field, which combines algorithms from quantitative genetics and neuroimaging to assess genetic effects on the brain. In twin imaging studies, it is common to estimate the intraclass correlation (ICC), which measures the resemblance between twin pairs for a given phenotype. In this paper, we extend the commonly used Pearson correlation to a more appropriate definition, which uses restricted maximum likelihood methods (REML). We computed proportion of phenotypic variance due to additive (A) genetic factors, common (C) and unique (E) environmental factors using a new definition of the variance components in the diffusion tensor-valued signals. We applied our analysis to a dataset of Diffusion Tensor Images (DTI) from 25 identical and 25 fraternal twin pairs. Differences between the REML and Pearson estimators were plotted for different sample sizes, showing that the REML approach avoids severe biases when samples are smaller. Measures of genetic effects were computed for scalar and multivariate diffusion tensor derived measures including the geodesic anisotropy (tGA) and the full diffusion tensors (DT), revealing voxel-wise genetic contributions to brain fiber microstructure.

Automatic clustering and population analysis of white matter tracts using maximum density paths

We introduce a framework for population analysis of white matter tracts based on diffusion-weighted images of the brain. The framework enables extraction of fibers from high angular resolution diffusion images (HARDI); clustering of the fibers based partly on prior knowledge from an atlas; representation of the fiber bundles compactly using a path following points of highest density (maximum density path; MDP); and registration of these paths together using geodesic curve matching to find local correspondences across a population. We demonstrate our method on 4-Tesla HARDI scans from 565 young adults to compute localized statistics across 50 white matter tracts based on fractional anisotropy (FA). Experimental results show increased sensitivity in the determination of genetic influences on principal fiber tracts compared to the tract-based spatial statistics (TBSS) method. Our results show that the MDP representation reveals important parts of the white matter structure and considerably reduces the dimensionality over comparable fiber matching approaches.

How does angular resolution affect diffusion imaging measures?

A key question in diffusion imaging is how many diffusion-weighted images suffice to provide adequate signal-to-noise ratio (SNR) for studies of fiber integrity. Motion, physiological effects, and scan duration all affect the achievable SNR in real brain images, making theoretical studies and simulations only partially useful. We therefore scanned 50 healthy adults with 105-gradient high-angular resolution diffusion imaging (HARDI) at 4T. From gradient image subsets of varying size (6 ≤ N ≤ 94) that optimized a spherical angular distribution energy, we created SNR plots (versus gradient numbers) for seven common diffusion anisotropy indices: fractional and relative anisotropy (FA, RA), mean diffusivity (MD), volume ratio (VR), geodesic anisotropy (GA), its hyperbolic tangent (tGA), and generalized fractional anisotropy (GFA). SNR, defined in a region of interest in the corpus callosum, was near-maximal with 58, 66, and 62 gradients for MD, FA, and RA, respectively, and with about 55 gradients for GA and tGA. For VR and GFA, SNR increased rapidly with more gradients. SNR was optimized when the ratio of diffusion-sensitized to non-sensitized images was 9.13 for GA and tGA, 10.57 for FA, 9.17 for RA, and 26 for MD and VR. In orientation density functions modeling the HARDI signal as a continuous mixture of tensors, the diffusion profile reconstruction accuracy rose rapidly with additional gradients. These plots may help in making trade-off decisions when designing diffusion imaging protocols.