Explorations of the Practical Issues of Learning Prediction-Control Tasks Using Temporal Difference Learning Methods

There has been recent interest in using temporal difference learning methods to attack problems of prediction and control. While these algorithms have been brought to bear on many problems, they remain poorly understood. It is the purpose of this thesis to further explore these algorithms, presenting a framework for viewing them and raising a number of practical issues and exploring those issues in the context of several case studies. This includes applying the TD(lambda) algorithm to: 1) learning to play tic-tac-toe from the outcome of self-play and of play against a perfectly-playing opponent and 2) learning simple one-dimensional segmentation tasks.

Biological Routes to Gold Nanoplates

Much effort has been devoted to the synthesis of gold nanoparticles with different shapes, including the zero-dimensional nanospheres, one dimensional nanorods, and two-dimensional nanoplates. Compared to zero or one dimensional nanostructures, the synthesis of two-dimensional nanostructures in high yield has always been more involved, often requiring complex and time-consuming steps such as morphology transformation from the nanospheres, or the seeded growth process. Herein we report a high yield method for gold nanoplate synthesis using the extract of unicellular green alga Chlorella vulgaris, which can be carried out under ambient conditions. More than 90% of the total nanoparticle population is of the platelet morphology, surpassing the previously reported value of 45%. The control of the anisotropic growth of different planes; as well as the lateral size, has also been partially optimized.

Effect of Gas Sparging in Mammalian Cell Bioreactors

One of the major problems in the operations of mammalian cell bioreactors is the detrimental effect of gas sparging. Since the most convenient way to oxygenate any bioreactor is by gas sparging, this adverse effect has often been one of the limiting oxygen transport problems in both laboratory and industrial mammalian cell bioreactors. When one examines the literature on the effect of gas sparging on the death of mammalian cells, a great deal of confusions has been reported. It is not clear from the published literature as to the leading cause for gas-sparged related cell death. These confusions prevent the rational design and operations of mammalian cell bioreactors. In our laboratory, we have attempted to address this problem both fundamentally as well as attempt to obtain a general understanding on the adverse effect of gas sparging. Our analyses first examined the fluid shear associated with the various sections that the gas bubbles encounter during entrance, passage through the bioreactor and the final exit of the gas bubbles. Our analyses showed that the major damage of the mammalian cells by gas bubbles is due to the burst of the bubbles when exiting the bioreactor. It was also our hypothesis that the entrained cells in the liquid boundary layer of the gas bubble upon bursting is the major cause for cell death. We have corroborated this hypothesis by correlating the liquid entrainment with the cell death rate using results from our laboratory as well as other studies. Pluonic F-68, a weak surfactant, has routinely been used in laboratory and industrial bioreactors. In the past, the protective effect of Pluronic F-68 has never been shown as to why it is effective. In our research, we have data using microphotography which clearly demonstrated and corroborated our entrainment hypothesis is the major reason for the effectiveness of Pluronic F-68 in protecting the cells from gas-sparged related cell death.