Stochastic optimal control

H. J. Kushner has obtained the differential equation satisfied by the optimal feedback control law for a stochastic control system in which the plant dynamics and observations are perturbed by independent additive Gaussian white noise processes. However, the differentiation includes the first and second functional derivatives and, except for a restricted set of systems, is too complex to solve with present techniques.

This investigation studies the optimal control law for the open loop system and incorporates it in a sub-optimal feedback control law. This suboptimal control law's performance is at least as good as that of the optimal control function and satisfies a differential equation involving only the first functional derivative. The solution of this equation is equivalent to solving two two-point boundary valued integro-partial differential equations. An approximate solution has advantages over the conventional approximate solution of Kushner's equation.

As a result of this study, well known results of deterministic optimal control are deduced from the analysis of optimal open loop control.

The oculomotor system: (1) Vertical-horizontal interaction and signal recognition, (2) Time delays and power spectra

In the first section of this thesis, two-dimensional properties of the human eye movement control system were studied. The vertical - horizontal interaction was investigated by using a two-dimensional target motion consisting of a sinusoid in one of the directions vertical or horizontal, and low-pass filtered Gaussian random motion of variable bandwidth (and hence information content) in the orthogonal direction. It was found that the random motion reduced the efficiency of the sinusoidal tracking. However, the sinusoidal tracking was only slightly dependent on the bandwidth of the random motion. Thus the system should be thought of as consisting of two independent channels with a small amount of mutual cross-talk.

These target motions were then rotated to discover whether or not the system is capable of recognizing the two-component nature of the target motion. That is, the sinusoid was presented along an oblique line (neither vertical nor horizontal) with the random motion orthogonal to it. The system did not simply track the vertical and horizontal components of motion, but rotated its frame of reference so that its two tracking channels coincided with the directions of the two target motion components. This recognition occurred even when the two orthogonal motions were both random, but with different bandwidths.

In the second section, time delays, prediction and power spectra were examined. Time delays were calculated in response to various periodic signals, various bandwidths of narrow-band Gaussian random motions and sinusoids. It was demonstrated that prediction occurred only when the target motion was periodic, and only if the harmonic content was such that the signal was sufficiently narrow-band. It appears as if general periodic motions are split into predictive and non-predictive components.

For unpredictable motions, the relationship between the time delay and the average speed of the retinal image was linear. Based on this I proposed a model explaining the time delays for both random and periodic motions. My experiments did not prove that the system is sampled data, or that it is continuous. However, the model can be interpreted as representative of a sample data system whose sample interval is a function of the target motion.

It was shown that increasing the bandwidth of the low-pass filtered Gaussian random motion resulted in an increase of the eye movement bandwidth. Some properties of the eyeball-muscle dynamics and the extraocular muscle "active state tension" were derived.

Nuclear magnetic resonance studies of protein conformation: I. Ribonuclease S. II. Hemoglobin

Fluorine nuclear magnetic resonance techniques have been used to study conformational processes in two proteins labeled specifically in strategic regions with covalently attached fluorinated molecules. In ribonuclease S, the ϵ-amino groups of lysines 1 and 7 were trifluoroacetylated without diminishing enzymatic activity. As inhibitors bound to the enzyme, changes in orientation of the peptide segment containing the trifluoroacetyl groups were detected in the nuclear magnetic resonance spectrum. pH Titration of one of the histidines in the active site produced a reversal of the conformational process.

Hemoglobin was trifluoroacetonylated at the reactive cysteine 93 of each β chain. The nuclear magnetic resonance spectrum of the fluorine moiety reflected changes in the equilibrium position of the β chain carboxy terminus upon binding of heme ligands and allosteric effectors. The chemical shift positions observed in deoxy- and methemoglobin were pH dependent, undergoing an abnormally steep apparent titration which was not observed in hemoglobin from which histidine β 146 had been removed enzymatically. The abnormal sharpness of these pH dependent processes is probably due to interactions between several ionizing groups.

The carbon monoxide binding process was studied by concurrent observation of the visible and nuclear magnetic resonance spectra of trifluoroacetonylated hemoglobin at fractional ligand saturations throughout the range 0-1.0. Comparison of the ligand binding process observed in these two ways yields evidence for a specific order of ligand binding. The sequence of events is sensitive to the pH and organic phosphate concentration of the medium, demonstrating the delicately balanced control system produced by interactions between the hemoglobin subunits and the effectors.