A Note on the Generalization Performance of Kernel Classifiers with Margin

We present distribution independent bounds on the generalization misclassification performance of a family of kernel classifiers with margin. Support Vector Machine classifiers (SVM) stem out of this class of machines. The bounds are derived through computations of the $V_gamma$ dimension of a family of loss functions where the SVM one belongs to. Bounds that use functions of margin distributions (i.e. functions of the slack variables of SVM) are derived.

Sparse Correlation Kernel Analysis and Reconstruction

This paper presents a new paradigm for signal reconstruction and superresolution, Correlation Kernel Analysis (CKA), that is based on the selection of a sparse set of bases from a large dictionary of class- specific basis functions. The basis functions that we use are the correlation functions of the class of signals we are analyzing. To choose the appropriate features from this large dictionary, we use Support Vector Machine (SVM) regression and compare this to traditional Principal Component Analysis (PCA) for the tasks of signal reconstruction, superresolution, and compression. The testbed we use in this paper is a set of images of pedestrians. This paper also presents results of experiments in which we use a dictionary of multiscale basis functions and then use Basis Pursuit De-Noising to obtain a sparse, multiscale approximation of a signal. The results are analyzed and we conclude that 1) when used with a sparse representation technique, the correlation function is an effective kernel for image reconstruction and superresolution, 2) for image compression, PCA and SVM have different tradeoffs, depending on the particular metric that is used to evaluate the results, 3) in sparse representation techniques, L_1 is not a good proxy for the true measure of sparsity, L_0, and 4) the L_epsilon norm may be a better error metric for image reconstruction and compression than the L_2 norm, though the exact psychophysical metric should take into account high order structure in images.

A Unified Framework for Regularization Networks and Support Vector Machines

Regularization Networks and Support Vector Machines are techniques for solving certain problems of learning from examples -- in particular the regression problem of approximating a multivariate function from sparse data. We present both formulations in a unified framework, namely in the context of Vapnik's theory of statistical learning which provides a general foundation for the learning problem, combining functional analysis and statistics.

On the V(subscript gamma) Dimension for Regression in Reproducing Kernel Hilbert Spaces

This paper presents a computation of the $V_gamma$ dimension for regression in bounded subspaces of Reproducing Kernel Hilbert Spaces (RKHS) for the Support Vector Machine (SVM) regression $epsilon$-insensitive loss function, and general $L_p$ loss functions. Finiteness of the RV_gamma$ dimension is shown, which also proves uniform convergence in probability for regression machines in RKHS subspaces that use the $L_epsilon$ or general $L_p$ loss functions. This paper presenta a novel proof of this result also for the case that a bias is added to the functions in the RKHS.

Applying the Lessons of “Lean Now” to Transform the US Aerospace Enterprise

Lean is common sense and good business sense. As organizations grow and become more successful, they begin to lose insight into the basic truths of what made them successful. Organizations have to deal with more and more issues that may not have anything to do with directly providing products or services to their customers. Lean is a holistic management approach that brings the focus of the organization back to providing value to the customer. In August 2002, Mrs. Darleen Druyun, the Principal Deputy to the Assistant Secretary of the Air Force for Acquisition and government co-chairperson of the Lean Aerospace Initiative (LAI), decided it was time for Air Force acquisitions to embrace the concepts of lean. At her request, the LAI Executive Board developed a concept and methodology to employ lean into the Air Force’s acquisition culture and processes. This was the birth of the “Lean Now” initiative. An enterprise-wide approach was used, involving Air Force System Program Offices (SPOs), aerospace industry, and several Department of Defense agencies. The aim of Lean Now was to focus on the process interfaces between these “enterprise” stakeholders to eliminate barriers that impede progress. Any best practices developed would be institutionalized throughout the Air Force and the Department of Defense (DoD). The industry members of LAI agreed to help accelerate the government-industry transformation by donating lean Subject Matter Experts (SMEs) to mentor, train, and facilitate the lean events of each enterprise. Currently, the industry SMEs and the Massachusetts Institute of Technology are working together to help the Air Force develop its own lean infrastructure of training courses and Air Force lean SMEs. The first Lean Now programs were the F/A-22, Global Hawk, and F-16. Each program focused on specific acquisition processes. The F/A-22 focused on the Test and Evaluation process; the Global Hawk focused on Evolutionary Acquisitions; and the F-16 focused on improving the Contract Closeout process. Through lean, each enterprise made many significant improvements. The F/A-22 was able to reduce its Operational Flight Plan (OFP) Preparation and Load process time of 2 to 3 months down to 7 hours. The Global Hawk developed a new production plan that increases the annual production of its Integrated Sensor Suite from 3 per year to 6 per year. The F-16 enterprise generated and is working 12 initiatives that could result in a contract closeout cycle time reduction of 3 to 7 years. Each enterprise continues to generate more lean initiatives that focus on other areas and processes within their respective enterprises.