An objective function based on Bayesian likelihoods of necessity and sufficiency for concept learning in the absence of labeled counter-examples

Supervised machine learning techniques generally require that the training set on which learning is based contain sufficient examples representative of the target concept, as well as known counter-examples of the concept; however, in many application domains it is not possible to supply a set of labeled counter-examples. This paper proposes an objective function based on Bayesian likelihoods of necessity and sufficiency. This function can be used to guide search towards the discovery of a concept description given only a set of labeled positive examples of the target concept, and as a corpus of unlabeled examples. Results of experiments performed on several datasets from the VCI repository show that the technique achieves comparable accuracy to conventional supervised learning techniques, despite the fact that the latter require a set of labeled counter-examples to be supplied. The technique can be applied in many domains in which the provision of labeled counter-examples is problematic.

Efficient serial and parallel implementations of the cutting angle global optimisation technique

We examine efficient computer implementation of one method of deterministic global optimisation, the cutting angle method. In this method the objective function is approximated from values below the function with a piecewise linear auxiliary function. The global minimum of the objective function is approximated from the sequence of minima of this auxiliary function. Computing the minima of the auxiliary function is a combinatorial problem, and we show that it can be effectively parallelised. We discuss the improvements made to the serial implementation of the cutting angle method, and ways of distributing computations across multiple processors on parallel and cluster computers.

Challenges of continuous global optimization in molecular structure prediction

The molecular geometry, the three dimensional arrangement of atoms in space, is a major factor determining the properties and reactivity of molecules, biomolecules and macromolecules. Computation of stable molecular conformations can be done by locating minima on the potential energy surface (PES). This is a very challenging global optimization problem because of extremely large numbers of shallow local minima and complicated landscape of PES. This paper illustrates the mathematical and computational challenges on one important instance of the problem, computation of molecular geometry of oligopeptides, and proposes the use of the Extended Cutting Angle Method (ECAM) to solve this problem.

ECAM is a deterministic global optimization technique, which computes tight lower bounds on the values of the objective function and fathoms those part of the domain where the global minimum cannot reside. As with any domain partitioning scheme, its challenge is an extremely large partition of the domain required for accurate lower bounds. We address this challenge by providing an efficient combinatorial algorithm for calculating the lower bounds, and by combining ECAM with a local optimization method, while preserving the deterministic character of ECAM.

Minimizing bounded uncertainty impact on scheduling with earliest start and due-date constraints via interval computation

Bounded uncertainty is a major challenge to real life scheduling as it increases the risk and cost depending on the objective function. Bounded uncertainty provides limited information about its nature. It provides only the upper and the lower bounds without information in between, in contrast to probability distributions and fuzzymembership functions. Bratley algorithm is usually used for scheduling with the constraints of earliest start and due-date. It is formulated as . The proposed research uses interval computation to minimize the impact of bounded uncertainty of processing times on Bratley’s algorithm. It minimizes the uncertainty of the estimate of the objective function. The proposed concept is to do the calculations on the interval values and approximate the end result instead of approximating each interval then doing numerical calculations. This methodology gives a more certain estimate of the objective function.

Cutting angle method - a tool for constrained global optimization

Cutting angle method (CAM) is a deterministic global optimization technique applicable to Lipschitz functions f: Rn → R. The method builds a sequence of piecewise linear lower approximations to the objective function f. The sequence of solutions to these relaxed problems converges to the global minimum of f. This article adapts CAM to the case of linear constraints on the feasible domain. We show how the relaxed problems are modified, and how the numerical efficiency of solving these problems can be preserved. A number of numerical experiments confirms the improved numerical efficiency.

Combining extension matrix and integer programming for optimal concept learning

This paper proposes two integer programming models and their GA-based solutions for optimal concept learning. The models are built to obtain the optimal concept description in the form of propositional logic formulas from examples based on completeness, consistency and simplicity. The simplicity of the propositional rules is selected as the objective function of the integer programming models, and the completeness and consistency of the concept are used as the constraints. Considering the real-world problems that certain level of noise is contained in data set, the constraints in model 11 are slacked by adding slack-variables. To solve the integer programming models, genetic algorithm is employed to search the global solution space. We call our approach IP-AE. Its effectiveness is verified by comparing the experimental results with other well- known concept learning algorithms: AQ15 and C4.5.

Towards optimization of a real-world robotic-sensor system of systems

The problem of threat detection in an unstructured environment is considered. Three systems, comprising of robots and sensors, are proposed to form a system of systems (SoS) to find a solution to the problem. System interactions are defined to provide a framework for formulation as an SoS optimization problem. Different cost and objective functions are introduced for optimization of local criteria. Using different weights, a linear combination of the local cost and objective functions is obtained to propose a global objective function. An algorithm is suggested to find an optimum value for the global objective function leading towards optimization of the SoS.

Extended cutting angle method of global optimization

Methods of Lipschitz optimization allow one to find and confirm the global minimum of multivariate Lipschitz functions using a finite number of function evaluations. This paper extends the Cutting Angle method, in which the optimization problem is solved by building a sequence of piecewise linear underestimates of the objective function. We use a more flexible set of support functions, which yields a better underestimate of a Lipschitz objective function. An efficient algorithm for enumeration of all local minima of the underestimate is presented, along with the results of numerical experiments. One dimensional Pijavski-Shubert method arises as a special case of the proposed approach.

Candidate working set strategy based SMO algorithm in support vector machine

Sequential minimal optimization (SMO) is quite an efficient algorithm for training the support vector machine. The most important step of this algorithm is the selection of the working set, which greatly affects the training speed. The feasible direction strategy for the working set selection can decrease the objective function, however, may augment to the total calculation for selecting the working set in each of the iteration. In this paper, a new candidate working set (CWS) Strategy is presented considering the cost on the working set selection and cache performance. This new strategy can select several greatest violating samples from Cache as the iterative working sets for the next several optimizing steps, which can improve the efficiency of the kernel cache usage and reduce the computational cost related to the working set selection. The results of the theory analysis and experiments demonstrate that the proposed method can reduce the training time, especially on the large-scale datasets.

Bounded lower subdifferentiability optimization techniques : applications

In this article we develop a global optimization algorithm for quasiconvex programming where the objective function is a Lipschitz function which may have "flat parts". We adapt the Extended Cutting Angle method to quasiconvex functions, which reduces significantly the number of iterations and objective function evaluations, and consequently the total computing time. Applications of such an algorithm to mathematical programming problems inwhich the objective function is derived from economic systems and location problems are described. Computational results are presented.

Spectral element model updating for damage identification using clonal selection algorithm

A spectral element model updating procedure is presented to identify damage in a structure using Guided wave propagation results. Two damage spectral elements (DSE1 and DSE2) are developed to model the local (cracks in reinforcement bar) and global (debonding between reinforcement bar and concrete) damage in one-dimensional homogeneous and composite waveguide, respectively. Transfer matrix method is adopted to assemble the stiffness matrix of multiple spectral elements. In order to solve the inverse problem, clonal selection algorithm is used for the optimization calculations. Two displacement-based functions and two frequency-based functions are used as objective functions in this study. Numerical simulations of wave propagation in a bare steel bar and in a reinforcement bar without and with various assumed damage scenarios are carried out. Numerically simulated data are then used to identify local and global damage of the steel rebar and the concrete-steel interface using the proposed method. Results show that local damage is easy to be identified by using any considered objective function with the proposed method while only using the wavelet energy-based objective function gives reliable identification of global damage. The method is then extended to identify multiple damages in a structure. To further verify the proposed method, experiments of wave propagation in a rectangular steel bar before and after damage are conducted. The proposed method is used to update the structural model for damage identification. The results demonstrate the capability of the proposed method in identifying cracks in steel bars based on measured wave propagation data.

Joint learning and dictionary construction for pattern recognition

We propose a joint representation and classification framework that achieves the dual goal of finding the most discriminative sparse overcomplete encoding and optimal classifier parameters. Formulating an optimization problem that combines the objective function of the classification with the representation error of both labeled and unlabeled data, constrained by sparsity, we propose an algorithm that alternates between solving for subsets of parameters, whilst preserving the sparsity. The method is then evaluated over two important classification problems in computer vision: object categorization of natural images using the Caltech 101 database and face recognition using the Extended Yale B face database. The results show that the proposed method is competitive against other recently proposed sparse overcomplete counterparts and considerably outperforms many recently proposed face recognition techniques when the number training samples is small.

Prediction interval construction using interval type-2 fuzzy logic systems

This study proposes a novel non-parametric method for construction of prediction intervals (PIs) using interval type-2 Takagi-Sugeno-Kang fuzzy logic systems (IT2 TSK FLSs). The key idea in the proposed method is to treat the left and right end points of the type-reduced set as the lower and upper bounds of a PI. This allows us to construct PIs without making any special assumption about the data distribution. A new training algorithm is developed to satisfy conditions imposed by the associated confidence level on PIs. Proper adjustment of premise and consequent parameters of IT2 TSK FLSs is performed through the minimization of a PI-based objective function, rather than traditional error-based cost functions. This new cost function covers both validity and informativeness aspects of PIs. A metaheuristic method is applied for minimization of the non-linear non-differentiable cost function. Quantitative measures are applied for assessing the quality of PIs constructed using IT2 TSK FLSs. The demonstrated results for four benchmark case studies with homogenous and heterogeneous noise clearly show the proposed method is capable of generating high quality PIs useful for decision-making.

Adaptive patch size determination for patch-based image completion

Patch-based image completion proceeds by iteratively filling the target (unknown) region by the best matching patches in the source image. In most existing such algorithms, the size of the patches is either fixed and specified by a default number or simply chosen to be inversely proportional to the spatial frequency. However, it is noted that the patch size affects how well the filled patch captures the local characteristics of the source image and thus the final completion accuracy. Thus in this paper we propose a new method to compute appropriate patch sizes for image completion to improve its performance. In particular, we formulate the patch size determination as an optimization problem that minimizes an objective function involving image gradients and distinct and homogenous features. Experimental results show that our method can provide a significant enhancement to patch-based image completion algorithms.

A new approach based on support vector machine for solving stochastic optimization

Making decision usually occurs in the state of being uncertain. These kinds of problems often expresses in a formula as optimization problems. It is desire for decision makers to find a solution for optimization problems. Typically, solving optimization problems in uncertain environment is difficult. This paper proposes a new hybrid intelligent algorithm to solve a kind of stochastic optimization i.e. dependent chance programming (DCP) model. In order to speed up the solution process, we used support vector machine regression (SVM regression) to approximate chance functions which is the probability of a sequence of uncertain event occurs based on the training data generated by the stochastic simulation. The proposed algorithm consists of three steps: (1) generate data to estimate the objective function, (2) utilize SVM regression to reveal a trend hidden in the data (3) apply genetic algorithm (GA) based on SVM regression to obtain an estimation for the chance function. Numerical example is presented to show the ability of algorithm in terms of time-consuming and precision.