The cross-entropy method for network reliability estimation

Consider a network of unreliable links, modelling for example a communication network. Estimating the reliability of the network-expressed as the probability that certain nodes in the network are connected-is a computationally difficult task. In this paper we study how the Cross-Entropy method can be used to obtain more efficient network reliability estimation procedures. Three techniques of estimation are considered: Crude Monte Carlo and the more sophisticated Permutation Monte Carlo and Merge Process. We show that the Cross-Entropy method yields a speed-up over all three techniques.

Application of the cross-entropy method to the buffer allocation problem in a simulation-based environment

The buffer allocation problem (BAP) is a well-known difficult problem in the design of production lines. We present a stochastic algorithm for solving the BAP, based on the cross-entropy method, a new paradigm for stochastic optimization. The algorithm involves the following iterative steps: (a) the generation of buffer allocations according to a certain random mechanism, followed by (b) the modification of this mechanism on the basis of cross-entropy minimization. Through various numerical experiments we demonstrate the efficiency of the proposed algorithm and show that the method can quickly generate (near-)optimal buffer allocations for fairly large production lines.

Heavy tails, importance sampling and cross-entropy

We consider the problem of estimating P(Yi + (...) + Y-n > x) by importance sampling when the Yi are i.i.d. and heavy-tailed. The idea is to exploit the cross-entropy method as a toot for choosing good parameters in the importance sampling distribution; in doing so, we use the asymptotic description that given P(Y-1 + (...) + Y-n > x), n - 1 of the Yi have distribution F and one the conditional distribution of Y given Y > x. We show in some specific parametric examples (Pareto and Weibull) how this leads to precise answers which, as demonstrated numerically, are close to being variance minimal within the parametric class under consideration. Related problems for M/G/l and GI/G/l queues are also discussed.

Homodyne measurement of the average photon number

We describe a scheme for measurement of the mean photon flux at an arbitrary optical sideband frequency using homodyne detection. Experimental implementation of the technique requires an acousto-optic modulator in addition to the homodyne detector, and does not require phase locking. The technique exhibits polarization and frequency and spatial mode selectivity, as well as much improved speed, resolution, and dynamic range when compared to linear photodetectors and avalanche photodiodes, with potential application to quantum-state tomography and information encoding using an optical frequency basis. Experimental data also support a quantum-mechanical description of vacuum noise.

The cross-entropy method for continuous multi-extremal optimization

In recent years, the cross-entropy method has been successfully applied to a wide range of discrete optimization tasks. In this paper we consider the cross-entropy method in the context of continuous optimization. We demonstrate the effectiveness of the cross-entropy method for solving difficult continuous multi-extremal optimization problems, including those with non-linear constraints.

Temperature of a trapped unitary Fermi gas at finite entropy

We present theoretical predictions for the equation of state of a harmonically trapped Fermi gas in the unitary limit. Our calculations compare Monte Carlo results with the equation of state of a uniform gas using three distinct perturbation schemes. We show that in experiments the temperature can be usefully calibrated by making use of the entropy, which is invariant during an adiabatic conversion into the weakly interacting limit of molecular BEC. We predict the entropy dependence of the equation of state.

Effect of the average soft-segment length on the morphology and properties of segmented polyurethane nanocomposites

Two organically modified layered silicates (with small and large diameters) were incorporated into three segmented polyurethanes with various degrees of microphase separation. Microphase separation increased with the molecular weight of the poly(hexamethylene oxide) soft segment. The molecular weight of the soft segment did not influence the amount of polyurethane intercalating the interlayer spacing. Small-angle neutron scattering and differential scanning calorimetry data indicated that the layered silicates did not affect the microphase morphology of any host polymer, regardless of the particle diameter. The stiffness enhancement on filler addition increased as the microphase separation of the polyurethane decreased, presumably because a greater number of urethane linkages were available to interact with the filler. For comparison, the small nanofiller was introduced into a polyurethane with a poly(tetramethylene oxide) soft segment, and a significant increase in the tensile strength and a sharper upturn in the stress-strain curve resulted. No such improvement occurred in the host polymers with poly(hexamethylene oxide) soft segments. It is proposed that the nanocomposite containing the more hydrophilic and mobile poly(tetramethylene oxide) soft segment is capable of greater secondary bonding between the polyurethane chains and the organosilicate surface, resulting in improved stress transfer to the filler and reduced molecular slippage. (c) 2006 Wiley Periodicals, Inc.

Entropy generation for microscale forced convection: effects of different thermal boundary conditions, velocity slip, temperature jump, viscous dissipation, and duct geometry

This work presents closed form solutions for fully developed temperature distribution and entropy generation due to forced convection in microelectromechanical systems (MEMS) in the Slip-flow regime, for which the Knudsen number lies within the range 0.001

A fuzzy approach to texture segmentation

The texture segmentation techniques are diversified by the existence of several approaches. In this paper, we propose fuzzy features for the segmentation of texture image. For this purpose, a membership function is constructed to represent the effect of the neighboring pixels on the current pixel in a window. Using these membership function values, we find a feature by weighted average method for the current pixel. This is repeated for all pixels in the window treating each time one pixel as the current pixel. Using these fuzzy based features, we derive three descriptors such as maximum, entropy, and energy for each window. To segment the texture image, the modified mountain clustering that is unsupervised and fuzzy c-means clustering have been used. The performance of the proposed features is compared with that of fractal features.

Estimating buffer overflows in three stages using cross-entropy

In this paper we propose a fast adaptive Importance Sampling method for the efficient simulation of buffer overflow probabilities in queueing networks. The method comprises three stages. First we estimate the minimum Cross-Entropy tilting parameter for a small buffer level; next, we use this as a starting value for the estimation of the optimal tilting parameter for the actual (large) buffer level; finally, the tilting parameter just found is used to estimate the overflow probability of interest. We recognize three distinct properties of the method which together explain why the method works well; we conjecture that they hold for quite general queueing networks. Numerical results support this conjecture and demonstrate the high efficiency of the proposed algorithm.

Mixture density networks

Minimization of a sum-of-squares or cross-entropy error function leads to network outputs which approximate the conditional averages of the target data, conditioned on the input vector. For classifications problems, with a suitably chosen target coding scheme, these averages represent the posterior probabilities of class membership, and so can be regarded as optimal. For problems involving the prediction of continuous variables, however, the conditional averages provide only a very limited description of the properties of the target variables. This is particularly true for problems in which the mapping to be learned is multi-valued, as often arises in the solution of inverse problems, since the average of several correct target values is not necessarily itself a correct value. In order to obtain a complete description of the data, for the purposes of predicting the outputs corresponding to new input vectors, we must model the conditional probability distribution of the target data, again conditioned on the input vector. In this paper we introduce a new class of network models obtained by combining a conventional neural network with a mixture density model. The complete system is called a Mixture Density Network, and can in principle represent arbitrary conditional probability distributions in the same way that a conventional neural network can represent arbitrary functions. We demonstrate the effectiveness of Mixture Density Networks using both a toy problem and a problem involving robot inverse kinematics.

Modelling the dynamics of genetic algorithms using statistical mechanics

A formalism for modelling the dynamics of Genetic Algorithms (GAs) using methods from statistical mechanics, originally due to Prugel-Bennett and Shapiro, is reviewed, generalized and improved upon. This formalism can be used to predict the averaged trajectory of macroscopic statistics describing the GA's population. These macroscopics are chosen to average well between runs, so that fluctuations from mean behaviour can often be neglected. Where necessary, non-trivial terms are determined by assuming maximum entropy with constraints on known macroscopics. Problems of realistic size are described in compact form and finite population effects are included, often proving to be of fundamental importance. The macroscopics used here are cumulants of an appropriate quantity within the population and the mean correlation (Hamming distance) within the population. Including the correlation as an explicit macroscopic provides a significant improvement over the original formulation. The formalism is applied to a number of simple optimization problems in order to determine its predictive power and to gain insight into GA dynamics. Problems which are most amenable to analysis come from the class where alleles within the genotype contribute additively to the phenotype. This class can be treated with some generality, including problems with inhomogeneous contributions from each site, non-linear or noisy fitness measures, simple diploid representations and temporally varying fitness. The results can also be applied to a simple learning problem, generalization in a binary perceptron, and a limit is identified for which the optimal training batch size can be determined for this problem. The theory is compared to averaged results from a real GA in each case, showing excellent agreement if the maximum entropy principle holds. Some situations where this approximation brakes down are identified. In order to fully test the formalism, an attempt is made on the strong sc np-hard problem of storing random patterns in a binary perceptron. Here, the relationship between the genotype and phenotype (training error) is strongly non-linear. Mutation is modelled under the assumption that perceptron configurations are typical of perceptrons with a given training error. Unfortunately, this assumption does not provide a good approximation in general. It is conjectured that perceptron configurations would have to be constrained by other statistics in order to accurately model mutation for this problem. Issues arising from this study are discussed in conclusion and some possible areas of further research are outlined.

On the annealed VC entropy for margin classifiers: A statistical mechanics study

Using techniques from Statistical Physics, the annealed VC entropy for hyperplanes in high dimensional spaces is calculated as a function of the margin for a spherical Gaussian distribution of inputs.