Noise tolerance under risk minimization

In this paper, we explore noise-tolerant learning of classifiers. We formulate the problem as follows. We assume that there is an unobservable training set that is noise free. The actual training set given to the learning algorithm is obtained from this ideal data set by corrupting the class label of each example. The probability that the class label of an example is corrupted is a function of the feature vector of the example. This would account for most kinds of noisy data one encounters in practice. We say that a learning method is noise tolerant if the classifiers learnt with noise-free data and with noisy data, both have the same classification accuracy on the noise-free data. In this paper, we analyze the noise-tolerance properties of risk minimization (under different loss functions). We show that risk minimization under 0-1 loss function has impressive noise-tolerance properties and that under squared error loss is tolerant only to uniform noise; risk minimization under other loss functions is not noise tolerant. We conclude this paper with some discussion on the implications of these theoretical results.

Natural motion of one-dimensional flexible objects using minimization approaches

For one-dimensional flexible objects such as ropes, chains, hair, the assumption of constant length is realistic for large-scale 3D motion. Moreover, when the motion or disturbance at one end gradually dies down along the curve defining the one-dimensional flexible objects, the motion appears ``natural''. This paper presents a purely geometric and kinematic approach for deriving more natural and length-preserving transformations of planar and spatial curves. Techniques from variational calculus are used to determine analytical conditions and it is shown that the velocity at any point on the curve must be along the tangent at that point for preserving the length and to yield the feature of diminishing motion. It is shown that for the special case of a straight line, the analytical conditions lead to the classical tractrix curve solution. Since analytical solutions exist for a tractrix curve, the motion of a piecewise linear curve can be solved in closed-form and thus can be applied for the resolution of redundancy in hyper-redundant robots. Simulation results for several planar and spatial curves and various input motions of one end are used to illustrate the features of motion damping and eventual alignment with the perturbation vector.

Fast Eigen solution for homogeneous quadratic minimization with at most three constraints

We propose an eigenvalue based technique to solve the Homogeneous Quadratic Constrained Quadratic Programming problem (HQCQP) with at most three constraints which arise in many signal processing problems. Semi-Definite Relaxation (SDR) is the only known approach and is computationally intensive. We study the performance of the proposed fast eigen approach through simulations in the context of MIMO relays and show that the solution converges to the solution obtained using the SDR approach with significant reduction in complexity.

Unconstrained Homooligomeric gamma-Peptides Show High Propensity for C-14 Helix Formation

Monosubstituted gamma(4)-residues (gamma(4)Leu, gamma(4)Ile, and gamma(4)Val) form helices even in short homooligomeric sequences. C-14 helix formation is established by X-ray diffraction in homooligomeric (gamma)(n) tetra-, hexa- and decapeptide sequences demonstrating the high propensity of gamma residues, with proteinogenic side chains, to adopt locally folded conformations.

C-12 Helices in Long Hybrid (alpha gamma)(n) Peptides Composed Entirely of Unconstrained Residues with Proteinogenic Side Chains

Unconstrained gamma(4) amino acid residues derived by homologation of proteinogenic amino acids facilitate helical folding in hybrid (alpha gamma)(n) sequences. The C-12 helical conformation for the decapeptide, Boc-Leu-gamma(4)(R)Val](5)-OMe, is established in crystals by X-ray diffraction. A regular C-12 helix is demonstrated by NMR studies of the 18 residue peptide, Boc-Leu-gamma(4)(AR)Val](9)-OMe, and a designed 16 residue (alpha gamma)(n) peptide, incorporating variable side chains. Unconstrained (alpha gamma)(n) peptides show an unexpectedly high propensity for helical folding in long polypeptide sequences.

Performance evaluation of typical approximation algorithms for nonconvex l(p)-minimization in diffuse optical tomography

The sparse estimation methods that utilize the l(p)-norm, with p being between 0 and 1, have shown better utility in providing optimal solutions to the inverse problem in diffuse optical tomography. These l(p)-norm-based regularizations make the optimization function nonconvex, and algorithms that implement l(p)-norm minimization utilize approximations to the original l(p)-norm function. In this work, three such typical methods for implementing the l(p)-norm were considered, namely, iteratively reweighted l(1)-minimization (IRL1), iteratively reweighted least squares (IRLS), and the iteratively thresholding method (ITM). These methods were deployed for performing diffuse optical tomographic image reconstruction, and a systematic comparison with the help of three numerical and gelatin phantom cases was executed. The results indicate that these three methods in the implementation of l(p)-minimization yields similar results, with IRL1 fairing marginally in cases considered here in terms of shape recovery and quantitative accuracy of the reconstructed diffuse optical tomographic images. (C) 2014 Optical Society of America

Making risk minimization tolerant to label noise

In many applications, the training data, from which one needs to learn a classifier, is corrupted with label noise. Many standard algorithms such as SVM perform poorly in the presence of label noise. In this paper we investigate the robustness of risk minimization to label noise. We prove a sufficient condition on a loss function for the risk minimization under that loss to be tolerant to uniform label noise. We show that the 0-1 loss, sigmoid loss, ramp loss and probit loss satisfy this condition though none of the standard convex loss functions satisfy it. We also prove that, by choosing a sufficiently large value of a parameter in the loss function, the sigmoid loss, ramp loss and probit loss can be made tolerant to nonuniform label noise also if we can assume the classes to be separable under noise-free data distribution. Through extensive empirical studies, we show that risk minimization under the 0-1 loss, the sigmoid loss and the ramp loss has much better robustness to label noise when compared to the SVM algorithm. (C) 2015 Elsevier B.V. All rights reserved.

Minimization Problems Based on Relative alpha-Entropy I: Forward Projection

Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative alpha-entropies (denoted I-alpha), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative alpha-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimizers of these relative alpha-entropies on closed and convex sets are shown to exist. Such minimizations generalize the maximum Renyi or Tsallis entropy principle. The minimizing probability distribution (termed forward I-alpha-projection) for a linear family is shown to obey a power-law. Other results in connection with statistical inference, namely subspace transitivity and iterated projections, are also established. In a companion paper, a related minimization problem of interest in robust statistics that leads to a reverse I-alpha-projection is studied.

Minimization Problems Based on Relative alpha-Entropy II: Reverse Projection

In part I of this two-part work, certain minimization problems based on a parametric family of relative entropies (denoted I-alpha) were studied. Such minimizers were called forward I-alpha-projections. Here, a complementary class of minimization problems leading to the so-called reverse I-alpha-projections are studied. Reverse I-alpha-projections, particularly on log-convex or power-law families, are of interest in robust estimation problems (alpha > 1) and in constrained compression settings (alpha < 1). Orthogonality of the power-law family with an associated linear family is first established and is then exploited to turn a reverse I-alpha-projection into a forward I-alpha-projection. The transformed problem is a simpler quasi-convex minimization subject to linear constraints.

Training-Based Antenna Selection for PER Minimization: A POMDP Approach

This paper considers the problem of receive antenna selection (AS) in a multiple-antenna communication system having a single radio-frequency (RF) chain. The AS decisions are based on noisy channel estimates obtained using known pilot symbols embedded in the data packets. The goal here is to minimize the average packet error rate (PER) by exploiting the known temporal correlation of the channel. As the underlying channels are only partially observed using the pilot symbols, the problem of AS for PER minimization is cast into a partially observable Markov decision process (POMDP) framework. Under mild assumptions, the optimality of a myopic policy is established for the two-state channel case. Moreover, two heuristic AS schemes are proposed based on a weighted combination of the estimated channel states on the different antennas. These schemes utilize the continuous valued received pilot symbols to make the AS decisions, and are shown to offer performance comparable to the POMDP approach, which requires one to quantize the channel and observations to a finite set of states. The performance improvement offered by the POMDP solution and the proposed heuristic solutions relative to existing AS training-based approaches is illustrated using Monte Carlo simulations.

Evaluation and Minimization of Low-Order Harmonic Torque in Low-Switching-Frequency Inverter-Fed Induction Motor Drives

A low-order harmonic pulsating torque is a major concern in high-power drives, high-speed drives, and motor drives operating in an overmodulation region. This paper attempts to minimize the low-order harmonic torques in induction motor drives, operated at a low pulse number (i.e., a low ratio of switching frequency to fundamental frequency), through a frequency domain (FD) approach as well as a synchronous reference frame (SRF) based approach. This paper first investigates FD-based approximate elimination of harmonic torque as suggested by classical works. This is then extended into a procedure for minimization of low-order pulsating torque components in the FD, which is independent of machine parameters and mechanical load. Furthermore, an SRF-based optimal pulse width modulation (PWM) method is proposed to minimize the low-order harmonic torques, considering the motor parameters and load torque. The two optimal methods are evaluated and compared with sine-triangle (ST) PWM and selective harmonic elimination (SHE) PWM through simulations and experimental studies on a 3.7-kW induction motor drive. The SRF-based optimal PWM results in marginally better performance than the FD-based one. However, the selection of optimal switching angle for any modulation index (M) takes much longer in case of SRF than in case of the FD-based approach. The FD-based optimal solutions can be used as good starting solutions and/or to reasonably restrict the search space for optimal solutions in the SRF-based approach. Both of the FD-based and SRF-based optimal PWM methods reduce the low-order pulsating torque significantly, compared to ST PWM and SHE PWM, as shown by the simulation and experimental results.

First-passage failure and its feedback minimization of quasi-partially integrable Hamiltonian systems

An n degree-of-freedom Hamiltonian system with r (1¡r¡n) independent 0rst integrals which are in involution is calledpartially integrable Hamiltonian system. A partially integrable Hamiltonian system subject to light dampings andweak stochastic excitations is called quasi-partially integrable Hamiltonian system. In the present paper, the procedures for studying the 0rst-passage failure and its feedback minimization of quasi-partially integrable Hamiltonian systems are proposed. First, the stochastic averaging methodfor quasi-partially integrable Hamiltonian systems is brie4y reviewed. Then, basedon the averagedIt ˆo equations, a backwardKolmogorov equation governing the conditional reliability function, a set of generalized Pontryagin equations governing the conditional moments of 0rst-passage time and their boundary and initial conditions are established. After that, the dynamical programming equations and their associated boundary and 0nal time conditions for the control problems of maximization of reliability andof maximization of mean 0rst-passage time are formulated. The relationship between the backwardKolmogorov equation andthe dynamical programming equation for reliability maximization, andthat between the Pontryagin equation andthe dynamical programming equation for maximization of mean 0rst-passage time are discussed. Finally, an example is worked out to illustrate the proposed procedures and the e9ectiveness of feedback control in reducing 0rst-passage failure.