Building Grounded Abstractions for Artificial Intelligence Programming

Most Artificial Intelligence (AI) work can be characterized as either ``high-level'' (e.g., logical, symbolic) or ``low-level'' (e.g., connectionist networks, behavior-based robotics). Each approach suffers from particular drawbacks. High-level AI uses abstractions that often have no relation to the way real, biological brains work. Low-level AI, on the other hand, tends to lack the powerful abstractions that are needed to express complex structures and relationships. I have tried to combine the best features of both approaches, by building a set of programming abstractions defined in terms of simple, biologically plausible components. At the ``ground level'', I define a primitive, perceptron-like computational unit. I then show how more abstract computational units may be implemented in terms of the primitive units, and show the utility of the abstract units in sample networks. The new units make it possible to build networks using concepts such as long-term memories, short-term memories, and frames. As a demonstration of these abstractions, I have implemented a simulator for ``creatures'' controlled by a network of abstract units. The creatures exist in a simple 2D world, and exhibit behaviors such as catching mobile prey and sorting colored blocks into matching boxes. This program demonstrates that it is possible to build systems that can interact effectively with a dynamic physical environment, yet use symbolic representations to control aspects of their behavior.

On the Convergence of Stochastic Iterative Dynamic Programming Algorithms

Recent developments in the area of reinforcement learning have yielded a number of new algorithms for the prediction and control of Markovian environments. These algorithms, including the TD(lambda) algorithm of Sutton (1988) and the Q-learning algorithm of Watkins (1989), can be motivated heuristically as approximations to dynamic programming (DP). In this paper we provide a rigorous proof of convergence of these DP-based learning algorithms by relating them to the powerful techniques of stochastic approximation theory via a new convergence theorem. The theorem establishes a general class of convergent algorithms to which both TD(lambda) and Q-learning belong.

Preconditioning and iterative solution of symmetric indefinite linear systems arising from interior point methods for linear programming

We study the preconditioning of symmetric indefinite linear systems of equations that arise in interior point solution of linear optimization problems. The preconditioning method that we study exploits the block structure of the augmented matrix to design a similar block structure preconditioner to improve the spectral properties of the resulting preconditioned matrix so as to improve the convergence rate of the iterative solution of the system. We also propose a two-phase algorithm that takes advantage of the spectral properties of the transformed matrix to solve for the Newton directions in the interior-point method. Numerical experiments have been performed on some LP test problems in the NETLIB suite to demonstrate the potential of the preconditioning method discussed.

Behavioral Measures and their Correlation with IPM Iteration Counts on Semi-Definite Programming Problems

We study four measures of problem instance behavior that might account for the observed differences in interior-point method (IPM) iterations when these methods are used to solve semidefinite programming (SDP) problem instances: (i) an aggregate geometry measure related to the primal and dual feasible regions (aspect ratios) and norms of the optimal solutions, (ii) the (Renegar-) condition measure C(d) of the data instance, (iii) a measure of the near-absence of strict complementarity of the optimal solution, and (iv) the level of degeneracy of the optimal solution. We compute these measures for the SDPLIB suite problem instances and measure the correlation between these measures and IPM iteration counts (solved using the software SDPT3) when the measures have finite values. Our conclusions are roughly as follows: the aggregate geometry measure is highly correlated with IPM iterations (CORR = 0.896), and is a very good predictor of IPM iterations, particularly for problem instances with solutions of small norm and aspect ratio. The condition measure C(d) is also correlated with IPM iterations, but less so than the aggregate geometry measure (CORR = 0.630). The near-absence of strict complementarity is weakly correlated with IPM iterations (CORR = 0.423). The level of degeneracy of the optimal solution is essentially uncorrelated with IPM iterations.

Lecture 1 - Starting out

What is Programming? A useful definition Object Orientation (and it’s counterparts) Thinking OO Programming Blocks Variables Logic Data Structures Methods

Introduction to JavaScript Programming

These are the resources for an introductory lecture in JavaScript programming. Exercises are provided to practice simple JavaScript programming, including a template for a DHTML implementation of Conway's Game of Life (with encrypted solution).

Mathematical Programming

Exam questions and solutions for a third year mathematical programming course.

Programming Principles: Starting Out

In this lecture we describe the structure of the Programming Principles course at Southampton, look at the definitions and paradigms of programming, and take a look ahead to the key things that we will be covering in the weeks ahead.

Programming Principles: Introduction to Java

In this lecture we look at key concepts in Java: how to write, compile and run Java programs, define a simple class, create a main method, and use if/else structures to define behaviour.

Programming Principles: Variables, Primitives, Objects and Scope

In this session we look more closely at the way that Java deals with variables, and in particular with the differences between primitives (basic types like int and char) and objects. We also take an initial look at the scoping rules in Java, which dictate the visibility of variables in your program

Programming Principles: Computational Thinking

In this session we look at how to think systematically about a problem and create a solution. We look at the definition and characteristics of an algorithm, and see how through modularisation and decomposition we can then choose a set of methods to create. We also compare this somewhat procedural approach, with the way that design works in Object Oriented Systems,

Programming Principles: Methods

In this session we look at how to create more powerful objects through more powerful methods. We look at parameters and call by value vs. call by reference; return types; and overloading.

Programming Principles: Encapsulation and Constructors

In this session we look at the public and protected keywords, and the principle of encapsulation. We also look at how Constructors can help you initialise objects, while maintaining the encapsulation principle.

Programming Principles: Loops and Arrays

In this session we look at the different types of loop in the Java language, and see how they can be used to iterate over Arrays.