An advanced regularization methodology for use in watershed model calibration

A calibration methodology based on an efficient and stable mathematical regularization scheme is described. This scheme is a variant of so-called Tikhonov regularization in which the parameter estimation process is formulated as a constrained minimization problem. Use of the methodology eliminates the need for a modeler to formulate a parsimonious inverse problem in which a handful of parameters are designated for estimation prior to initiating the calibration process. Instead, the level of parameter parsimony required to achieve a stable solution to the inverse problem is determined by the inversion algorithm itself. Where parameters, or combinations of parameters, cannot be uniquely estimated, they are provided with values, or assigned relationships with other parameters, that are decreed to be realistic by the modeler. Conversely, where the information content of a calibration dataset is sufficient to allow estimates to be made of the values of many parameters, the making of such estimates is not precluded by preemptive parsimonizing ahead of the calibration process. White Tikhonov schemes are very attractive and hence widely used, problems with numerical stability can sometimes arise because the strength with which regularization constraints are applied throughout the regularized inversion process cannot be guaranteed to exactly complement inadequacies in the information content of a given calibration dataset. A new technique overcomes this problem by allowing relative regularization weights to be estimated as parameters through the calibration process itself. The technique is applied to the simultaneous calibration of five subwatershed models, and it is demonstrated that the new scheme results in a more efficient inversion, and better enforcement of regularization constraints than traditional Tikhonov regularization methodologies. Moreover, it is argued that a joint calibration exercise of this type results in a more meaningful set of parameters than can be achieved by individual subwatershed model calibration. (c) 2005 Elsevier B.V. All rights reserved.

An inverse method for designing loaded RF coils in MRI

Radio-frequency ( RF) coils are designed such that they induce homogeneous magnetic fields within some region of interest within a magnetic resonance imaging ( MRI) scanner. Loading the scanner with a patient disrupts the homogeneity of these fields and can lead to a considerable degradation of the quality of the acquired image. In this paper, an inverse method is presented for designing RF coils, in which the presence of a load ( patient) within the MRI scanner is accounted for in the model. To approximate the finite length of the coil, a Fourier series expansion is considered for the coil current density and for the induced fields. Regularization is used to solve this ill-conditioned inverse problem for the unknown Fourier coefficients. That is, the error between the induced and homogeneous target fields is minimized along with an additional constraint, chosen in this paper to represent the curvature of the coil windings. Smooth winding patterns are obtained for both unloaded and loaded coils. RF fields with a high level of homogeneity are obtained in the unloaded case and a limit to the level of homogeneity attainable is observed in the loaded case.

Gaussian operator bases for correlated fermions

We formulate a general multi-mode Gaussian operator basis for fermions, to enable a positive phase-space representation of correlated Fermi states. The Gaussian basis extends existing bosonic phase-space methods to Fermi systems and thus allows first-principles dynamical or equilibrium calculations in quantum many-body Fermi systems. We prove the completeness of the basis and derive differential forms for products with one- and two-body operators. Because the basis satisfies fermionic superselection rules, the resulting phase space involves only c-numbers, without requiring anticommuting Grassmann variables. Furthermore, because of the overcompleteness of the basis, the phase-space distribution can always be chosen positive. This has important consequences for the sign problem in fermion physics.

Gaussian phase-space representations for fermions

We introduce a positive phase-space representation for fermions, using the most general possible multimode Gaussian operator basis. The representation generalizes previous bosonic quantum phase-space methods to Fermi systems. We derive equivalences between quantum and stochastic moments, as well as operator correspondences that map quantum operator evolution onto stochastic processes in phase space. The representation thus enables first-principles quantum dynamical or equilibrium calculations in many-body Fermi systems. Potential applications are to strongly interacting and correlated Fermi gases, including coherent behavior in open systems and nanostructures described by master equations. Examples of an ideal gas and the Hubbard model are given, as well as a generic open system, in order to illustrate these ideas.

Quantum entanglement in the two-impurity Kondo model

In order to quantify quantum entanglement in two-impurity Kondo systems, we calculate the concurrence, negativity, and von Neumann entropy. The entanglement of the two Kondo impurities is shown to be determined by two competing many-body effects, namely the Kondo effect and the Ruderman-Kittel-Kasuya-Yosida (RKKY) interaction, I. Due to the spin-rotational invariance of the ground state, the concurrence and negativity are uniquely determined by the spin-spin correlation between the impurities. It is found that there exists a critical minimum value of the antiferromagnetic correlation between the impurity spins which is necessary for entanglement of the two impurity spins. The critical value is discussed in relation with the unstable fixed point in the two-impurity Kondo problem. Specifically, at the fixed point there is no entanglement between the impurity spins. Entanglement will only be created [and quantum information processing (QIP) will only be possible] if the RKKY interaction exchange energy, I, is at least several times larger than the Kondo temperature, T-K. Quantitative criteria for QIP are given in terms of the impurity spin-spin correlation.

Estimation of concentration-dependent diffusion coefficient in drying process from the space-averaged concentration versus time with experimental data

The estimation of a concentration-dependent diffusion coefficient in a drying process is known as an inverse coefficient problem. The solution is sought wherein the space-average concentration is known as function of time (mass loss monitoring). The problem is stated as the minimization of a functional and gradient-based algorithms are used to solve it. Many numerical and experimental examples that demonstrate the effectiveness of the proposed approach are presented. Thin slab drying was carried out in an isothermal drying chamber built in our laboratory. The diffusion coefficients of fructose obtained with the present method are compared with existing literature results.

Bayesian invariant measurements of generalisation

The problem of evaluating different learning rules and other statistical estimators is analysed. A new general theory of statistical inference is developed by combining Bayesian decision theory with information geometry. It is coherent and invariant. For each sample a unique ideal estimate exists and is given by an average over the posterior. An optimal estimate within a model is given by a projection of the ideal estimate. The ideal estimate is a sufficient statistic of the posterior, so practical learning rules are functions of the ideal estimator. If the sole purpose of learning is to extract information from the data, the learning rule must also approximate the ideal estimator. This framework is applicable to both Bayesian and non-Bayesian methods, with arbitrary statistical models, and to supervised, unsupervised and reinforcement learning schemes.

On-line learning with adaptive back-propagation in two-layer networks

An adaptive back-propagation algorithm parameterized by an inverse temperature 1/T is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, we analyse these learning algorithms in both the symmetric and the convergence phase for finite learning rates in the case of uncorrelated teachers of similar but arbitrary length T. These analyses show that adaptive back-propagation results generally in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.

Learning in ultrametric committee machines

The problem of learning by examples in ultrametric committee machines (UCMs) is studied within the framework of statistical mechanics. Using the replica formalism we calculate the average generalization error in UCMs with L hidden layers and for a large enough number of units. In most of the regimes studied we find that the generalization error, as a function of the number of examples presented, develops a discontinuous drop at a critical value of the load parameter. We also find that when L>1 a number of teacher networks with the same number of hidden layers and different overlaps induce learning processes with the same critical points.

Interacting nonequilibrium systems with two temperatures

We investigate a simplified model of two fully connected magnetic systems maintained at different temperatures by virtue of being connected to two independent thermal baths while simultaneously being interconnected with each other. Using generating functional analysis, commonly used in statistical mechanics, we find exactly soluble expressions for their individual magnetization that define a two-dimensional nonlinear map, the equations of which have the same form as those obtained for densely connected equilibrium systems. Steady states correspond to the fixed points of this map, separating the parameter space into a rich set of nonequilibrium phases that we analyze in asymptotically high and low (nonequilibrium) temperature limits. The theoretical formalism is shown to revert to the classical nonequilibrium steady state problem for two interacting systems with a nonzero heat transfer between them that catalyzes a phase transition between ambient nonequilibrium states. © 2013 American Physical Society.

On the numerical solution of a Cauchy problem in an elastostatic half-plane with a bounded inclusion

We propose an iterative procedure for the inverse problem of determining the displacement vector on the boundary of a bounded planar inclusion given the displacement and stress fields on an infinite (planar) line-segment. At each iteration step mixed boundary value problems in an elastostatic half-plane containing the bounded inclusion are solved. For efficient numerical implementation of the procedure these mixed problems are reduced to integral equations over the bounded inclusion. Well-posedness and numerical solution of these boundary integral equations are presented, and a proof of convergence of the procedure for the inverse problem to the original solution is given. Numerical investigations are presented both for the direct and inverse problems, and these results show in particular that the displacement vector on the boundary of the inclusion can be found in an accurate and stable way with small computational cost.

Message-passing for inference and optimization of real variables on sparse graphs

The inference and optimization in sparse graphs with real variables is studied using methods of statistical mechanics. Efficient distributed algorithms for the resource allocation problem are devised. Numerical simulations show excellent performance and full agreement with the theoretical results. © Springer-Verlag Berlin Heidelberg 2006.

Typical performance of regular low-density parity-check codes over general symmetric channels

Typical performance of low-density parity-check (LDPC) codes over a general binary-input output-symmetric memoryless channel is investigated using methods of statistical mechanics. Relationship between the free energy in statistical-mechanics approach and the mutual information used in the information-theory literature is established within a general framework; Gallager and MacKay-Neal codes are studied as specific examples of LDPC codes. It is shown that basic properties of these codes known for particular channels, including their potential to saturate Shannon's bound, hold for general symmetric channels. The binary-input additive-white-Gaussian-noise channel and the binary-input Laplace channel are considered as specific channel models.