Globally optimal on-line learning rules for multi-layer neural networks

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

Globally optimal learning rates in multilayer neural networks

A method for calculating the globally optimal learning rate in on-line gradient-descent training of multilayer neural networks is presented. The method is based on a variational approach which maximizes the decrease in generalization error over a given time frame. We demonstrate the method by computing optimal learning rates in typical learning scenarios. The method can also be employed when different learning rates are allowed for different parameter vectors as well as to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule.

Globally optimal on-line learning rules

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

The role of stochasticity in an information-optimal neural population code

In this paper we consider the optimisation of Shannon mutual information (MI) in the context of two model neural systems The first is a stochastic pooling network (population) of McCulloch-Pitts (MP) type neurons (logical threshold units) subject to stochastic forcing; the second is (in a rate coding paradigm) a population of neurons that each displays Poisson statistics (the so called 'Poisson neuron'). The mutual information is optimised as a function of a parameter that characterises the 'noise level'-in the MP array this parameter is the standard deviation of the noise, in the population of Poisson neurons it is the window length used to determine the spike count. In both systems we find that the emergent neural architecture and; hence, code that maximises the MI is strongly influenced by the noise level. Low noise levels leads to a heterogeneous distribution of neural parameters (diversity), whereas, medium to high noise levels result in the clustering of neural parameters into distinct groups that can be interpreted as subpopulations In both cases the number of subpopulations increases with a decrease in noise level. Our results suggest that subpopulations are a generic feature of an information optimal neural population.

Exchange rate economic exposure and hedging:the significance of currency options

This thesis focuses on the theoretical examination of the exchange rate economic (operating) exposure within the context of the theory of the firm, and proposes some hedging solutions using currency options. The examination of economic exposure is based on such parameters as firms' objectives, industry structure and production cost efficiency. In particular, it examines an hypothetical exporting firm with costs in domestic currency, which faces competition from foreign firms in overseas markets and has a market share expansion objective. Within this framework, the hypothesis is established that economic exposure, portrayed in a diagram connecting export prices and real exchange rates, is asymmetric (i.e. the negative effects depreciation are higher than the positive effects of a currency depreciation). In this case, export business can be seen as a real option, given by exporting firms to overseas customer. Different scenarios about the asymmetry hypothesis can be derived for different assumptions about the determinants of economic exposure. Having established the asymmetry hypothesis, the hedging against this exposure is analysed. The hypothesis is established, that a currency call option should be used in hedging against asymmetric economic exposure. Further, some advanced currency options stategies are discussed, and their use in hedging several scenarios of exposure is indicated, establishing the hypothesis that, the optimal options strategy is a function of the determinants of exposure. Some extensions on the theoretical analysis are examined. These include the hedging of multicurrency exposure using options, and the exposure of a purely domestic firm facing import competition. The empirical work addresses two issues: the empirical validity of the asymmetry hypothesis and the examination of the hedging effectiveness of currency options.

Theoretic analysis of IEEE 802.11 rate adaptation algorithm SampleRate

In this paper we investigate rate adaptation algorithm SampleRate, which spends a fixed time on bit-rates other than the currently measured best bit-rate. A simple but effective analytic model is proposed to study the steady-state behavior of the algorithm. Impacts of link condition, channel congestion and multi-rate retry on the algorithm performance are modeled. Simulations validate the model. It is also observed there is still a large performance gap between SampleRate and optimal scheme in case of high frame collision probability.

Susceptibility to pattern glare and the effect of spectral filters on rate of reading and visual search in stroke patients

The present thesis investigates pattern glare susceptibility following stroke and the immediate and prolonged impact of prescribing optimal spectral filters on reading speed, accuracy and visual search performance. Principal observations: A case report has shown that visual stress can occur following stroke. The use of spectral filters and precision tinted lenses proved to be a successful intervention in this case, although the parameters required modification following a further stroke episode. Stroke subjects demonstrate elevated levels of pattern glare compared to normative data values and a control group. Initial use of an optimal spectral filter in a stroke cohort increased reading speed by ~6% and almost halved error scores, findings not replicated in a control group. With the removal of migraine subjects reading speed increased by ~8% with an optimal filter and error scores almost halved. Prolonged use of an optimal spectral filter for stroke subjects, increased reading speed by >9% and error scores more than halved. When the same subjects switched to prolonged use of a grey filter, reading speed reduced by ~4% and error scores increased marginally. When a second group of stroke subjects used a grey filter first, reading speed decreased by ~3% but increased by ~3% with prolonged use of an optimal filter, with error scores almost halving; these findings persisted with migraine subjects excluded. Initial use of an optimal spectral filter improved visual search response time but not error scores in a stroke cohort with migraine subjects excluded. Neither prolonged use of an optimal nor grey filter improved response time or reduced error scores in a stroke group; these findings persisted with the exclusion of migraine subjects.

Adaptive resource allocation for multicast orthogonal frequency division multiple access systems with guaranteed BER and rate

In this paper, we propose a resource allocation scheme to minimize transmit power for multicast orthogonal frequency division multiple access systems. The proposed scheme allows users to have different symbol error rate (SER) across subcarriers and guarantees an average bit error rate and transmission rate for all users. We first provide an algorithm to determine the optimal bits and target SER on subcarriers. Because the worst-case complexity of the optimal algorithm is exponential, we further propose a suboptimal algorithm that separately assigns bit and adjusts SER with a lower complexity. Numerical results show that the proposed algorithm can effectively improve the performance of multicast orthogonal frequency division multiple access systems and that the performance of the suboptimal algorithm is close to that of the optimal one. Copyright © 2012 John Wiley & Sons, Ltd. This paper proposes optimal and suboptimal algorithms for minimizing transmitting power of multicast orthogonal frequency division multiple access systems with guaranteed average bit error rate and data rate requirement. The proposed scheme allows users to have different symbol error rate across subcarriers and guarantees an average bit error rate and transmission rate for all users. Copyright © 2012 John Wiley & Sons, Ltd.

Optimal span length in high-speed transmission systems with hybrid Raman-erbium-doped fiber amplification

We show, using nonlinearity management, that the optimal performance in high-bit-rate dispersion-managed fiber systems with hybrid amplification is achieved for a specific amplifier spacing that is different from the asymptotically vanishing length corresponding to ideally distributed amplification [Opt. Lett. 15, 1064 (1990)]. In particular, we prove the existence of a nontrivial optimal span length for 40-Gbit/s wavelength-division transmission systems with Raman-erbium-doped fiber amplification. Optimal amplifier lengths are obtained for several dispersion maps based on commonly used transmission fibers. © 2005 Optical Society of America.

A hierarchy of phase transitions in optimal neuronal coding:from binary to M-ary discrete optimal codes

We have investigated how optimal coding for neural systems changes with the time available for decoding. Optimization was in terms of maximizing information transmission. We have estimated the parameters for Poisson neurons that optimize Shannon transinformation with the assumption of rate coding. We observed a hierarchy of phase transitions from binary coding, for small decoding times, toward discrete (M-ary) coding with two, three and more quantization levels for larger decoding times. We postulate that the presence of subpopulations with specific neural characteristics could be a signiture of an optimal population coding scheme and we use the mammalian auditory system as an example.

Achievable performance gain of IEEE 802.11 multi-rate link adaptation algorithm with cross-layer design

Link quality-based rate adaptation has been widely used for IEEE 802.11 networks. However, network performance is affected by both link quality and random channel access. Selection of transmit modes for optimal link throughput can cause medium access control (MAC) throughput loss. In this paper, we investigate this issue and propose a generalised cross-layer rate adaptation algorithm. It considers jointly link quality and channel access to optimise network throughput. The objective is to examine the potential benefits by cross-layer design. An efficient analytic model is proposed to evaluate rate adaptation algorithms under dynamic channel and multi-user access environments. The proposed algorithm is compared to link throughput optimisation-based algorithm. It is found rate adaptation by optimising link layer throughput can result in large performance loss, which cannot be compensated by the means of optimising MAC access mechanism alone. Results show cross-layer design can achieve consistent and considerable performance gains of up to 20%. It deserves to be exploited in practical design for IEEE 802.11 networks.