Explorations of the Practical Issues of Learning Prediction-Control Tasks Using Temporal Difference Learning Methods

There has been recent interest in using temporal difference learning methods to attack problems of prediction and control. While these algorithms have been brought to bear on many problems, they remain poorly understood. It is the purpose of this thesis to further explore these algorithms, presenting a framework for viewing them and raising a number of practical issues and exploring those issues in the context of several case studies. This includes applying the TD(lambda) algorithm to: 1) learning to play tic-tac-toe from the outcome of self-play and of play against a perfectly-playing opponent and 2) learning simple one-dimensional segmentation tasks.

Defects in Self Assembled Colloidal Crystals

Colloidal self assembly is an efficient method for making 3-D ordered nanostructures suitable for materials such as photonic crystals and macroscopic solids for catalysis and sensor applications. Colloidal crystals grown by convective methods exhibit defects on two different scales. Macro defects such as cracks and void bands originate from the dynamics of meniscus motion during colloidal crystal growth while micro defects like vacancies, dislocation and stacking faults are indigenous to the colloidal crystalline structure. This paper analyses the crystallography and energetics of the microscopic defects from the point of view of classical thermodynamics and discusses the strategy for the control of the macroscopic defects through optimization of the liquid-vapor interface.