Learning Nonlinear Visual Processing from Natural Images

The paradigm of computational vision hypothesizes that any visual function -- such as the recognition of your grandparent -- can be replicated by computational processing of the visual input. What are these computations that the brain performs? What should or could they be? Working on the latter question, this dissertation takes the statistical approach, where the suitable computations are attempted to be learned from the natural visual data itself. In particular, we empirically study the computational processing that emerges from the statistical properties of the visual world and the constraints and objectives specified for the learning process. This thesis consists of an introduction and 7 peer-reviewed publications, where the purpose of the introduction is to illustrate the area of study to a reader who is not familiar with computational vision research. In the scope of the introduction, we will briefly overview the primary challenges to visual processing, as well as recall some of the current opinions on visual processing in the early visual systems of animals. Next, we describe the methodology we have used in our research, and discuss the presented results. We have included some additional remarks, speculations and conclusions to this discussion that were not featured in the original publications. We present the following results in the publications of this thesis. First, we empirically demonstrate that luminance and contrast are strongly dependent in natural images, contradicting previous theories suggesting that luminance and contrast were processed separately in natural systems due to their independence in the visual data. Second, we show that simple cell -like receptive fields of the primary visual cortex can be learned in the nonlinear contrast domain by maximization of independence. Further, we provide first-time reports of the emergence of conjunctive (corner-detecting) and subtractive (opponent orientation) processing due to nonlinear projection pursuit with simple objective functions related to sparseness and response energy optimization. Then, we show that attempting to extract independent components of nonlinear histogram statistics of a biologically plausible representation leads to projection directions that appear to differentiate between visual contexts. Such processing might be applicable for priming, \ie the selection and tuning of later visual processing. We continue by showing that a different kind of thresholded low-frequency priming can be learned and used to make object detection faster with little loss in accuracy. Finally, we show that in a computational object detection setting, nonlinearly gain-controlled visual features of medium complexity can be acquired sequentially as images are encountered and discarded. We present two online algorithms to perform this feature selection, and propose the idea that for artificial systems, some processing mechanisms could be selectable from the environment without optimizing the mechanisms themselves. In summary, this thesis explores learning visual processing on several levels. The learning can be understood as interplay of input data, model structures, learning objectives, and estimation algorithms. The presented work adds to the growing body of evidence showing that statistical methods can be used to acquire intuitively meaningful visual processing mechanisms. The work also presents some predictions and ideas regarding biological visual processing.

Optimisation Problems in Wireless Sensor Networks : Local Algorithms and Local Graphs

This thesis studies optimisation problems related to modern large-scale distributed systems, such as wireless sensor networks and wireless ad-hoc networks. The concrete tasks that we use as motivating examples are the following: (i) maximising the lifetime of a battery-powered wireless sensor network, (ii) maximising the capacity of a wireless communication network, and (iii) minimising the number of sensors in a surveillance application. A sensor node consumes energy both when it is transmitting or forwarding data, and when it is performing measurements. Hence task (i), lifetime maximisation, can be approached from two different perspectives. First, we can seek for optimal data flows that make the most out of the energy resources available in the network; such optimisation problems are examples of so-called max-min linear programs. Second, we can conserve energy by putting redundant sensors into sleep mode; we arrive at the sleep scheduling problem, in which the objective is to find an optimal schedule that determines when each sensor node is asleep and when it is awake. In a wireless network simultaneous radio transmissions may interfere with each other. Task (ii), capacity maximisation, therefore gives rise to another scheduling problem, the activity scheduling problem, in which the objective is to find a minimum-length conflict-free schedule that satisfies the data transmission requirements of all wireless communication links. Task (iii), minimising the number of sensors, is related to the classical graph problem of finding a minimum dominating set. However, if we are not only interested in detecting an intruder but also locating the intruder, it is not sufficient to solve the dominating set problem; formulations such as minimum-size identifying codes and locating dominating codes are more appropriate. This thesis presents approximation algorithms for each of these optimisation problems, i.e., for max-min linear programs, sleep scheduling, activity scheduling, identifying codes, and locating dominating codes. Two complementary approaches are taken. The main focus is on local algorithms, which are constant-time distributed algorithms. The contributions include local approximation algorithms for max-min linear programs, sleep scheduling, and activity scheduling. In the case of max-min linear programs, tight upper and lower bounds are proved for the best possible approximation ratio that can be achieved by any local algorithm. The second approach is the study of centralised polynomial-time algorithms in local graphs these are geometric graphs whose structure exhibits spatial locality. Among other contributions, it is shown that while identifying codes and locating dominating codes are hard to approximate in general graphs, they admit a polynomial-time approximation scheme in local graphs.

Implementation of placement and routing algorithms for computer aided design of printed circuit boards

The Printed Circuit Board (PCB) layout design is one of the most important and time consuming phases during equipment design process in all electronic industries. This paper is concerned with the development and implementation of a computer aided PCB design package. A set of programs which operate on a description of the circuit supplied by the user in the form of a data file and subsequently design the layout of a double-sided PCB has been developed. The algorithms used for the design of the PCB optimise the board area and the length of copper tracks used for the interconnections. The output of the package is the layout drawing of the PCB, drawn on a CALCOMP hard copy plotter and a Tektronix 4012 storage graphics display terminal. The routing density (the board area required for one component) achieved by this package is typically 0.8 sq. inch per IC. The package is implemented on a DEC 1090 system in Pascal and FORTRAN and SIGN(1) graphics package is used for display generation.

Open-pit block sequencing optimization: A mathematical model and solution technique

This study presents a comprehensive mathematical formulation model for a short-term open-pit mine block sequencing problem, which considers nearly all relevant technical aspects in open-pit mining. The proposed model aims to obtain the optimum extraction sequences of the original-size (smallest) blocks over short time intervals and in the presence of real-life constraints, including precedence relationship, machine capacity, grade requirements, processing demands and stockpile management. A hybrid branch-and-bound and simulated annealing algorithm is developed to solve the problem. Computational experiments show that the proposed methodology is a promising way to provide quantitative recommendations for mine planning and scheduling engineers.

Estimation of Execution Times of Process Control Algorithms on Microcomputers

An important question which has to be answered in evaluting the suitability of a microcomputer for a control application is the time it would take to execute the specified control algorithm. In this paper, we present a method of obtaining closed-form formulas to estimate this time. These formulas are applicable to control algorithms in which arithmetic operations and matrix manipulations dominate. The method does not require writing detailed programs for implementing the control algorithm. Using this method, the execution times of a variety of control algorithms on a range of 16-bit mini- and recently announced microcomputers are calculated. The formulas have been verified independently by an analysis program, which computes the execution time bounds of control algorithms coded in Pascal when they are run on a specified micro- or minicomputer.

Integration and optimization of pressure retarded osmosis with reverse osmosis for power generation and high efficiency desalination

Salinity gradient power is proposed as a source of renewable energy when two solutions of different salinity are mixed. In particular, Pressure Retarded Osmosis (PRO) coupled with a Reverse Osmosis process (RO) has been previously suggested for power generation, using RO brine as the draw solution. However, integration of PRO with RO may have further value for increasing the extent of water recovery in a desalination process. Consequently, this study was designed to model the impact of various system parameters to better understand how to design and operate practical PRO-RO units. The impact of feed salinity and recovery rate for the RO process on the concentration of draw solution, feed pressure, and membrane area of the PRO process was evaluated. The PRO system was designed to operate at maximum power density of . Model results showed that the PRO power density generated intensified with increasing seawater salinity and RO recovery rate. For an RO process operating at 52% recovery rate and 35 g/L feed salinity, a maximum power density of 24 W/m2 was achieved using 4.5 M NaCl draw solution. When seawater salinity increased to 45 g/L and the RO recovery rate was 46%, the PRO power density increased to 28 W/m2 using 5 M NaCl draw solution. The PRO system was able to increase the recovery rate of the RO by up to 18% depending on seawater salinity and RO recovery rate. This result suggested a potential advantage of coupling PRO process with RO system to increase the recovery rate of the desalination process and reduce brine discharge.

Reliability Optimization using SLUMT

This paper is concerned with the reliability optimization of a spatially redundant system, subject to various constraints, by using nonlinear programming. The constrained optimization problem is converted into a sequence of unconstrained optimization problems by using a penalty function. The new problem is then solved by the conjugate gradient method. The advantages of this method are highlighted.

Theoretical gain optimization studies in 10.6 µm CO2-N2 gasdynamic lasers. IV. Further results of parametric study

Based on a method proposed by Reddy and Shanmugasundaram, similar solutions have been obtained for the steady inviscid quasi-one-dimensional nonreacting flow in the supersonic nozzle of CO2-N2-H2O and CO2-N2-He gasdynamic laser systems. Instead of using the correlations of a nonsimilar function NS for pure N2 gas, as is done in previous publications, the NS correlations are computed here for the actual gas mixtures used in the gasdynamic lasers. Optimum small-signal optical gain and the corresponding optimum values of the operating parameters like reservoir pressure and temperature and nozzle area ratio are computed using these correlations. The present results are compared with the previous results and the main differences are discussed. Journal of Applied Physics is copyrighted by The American Institute of Physics.

Launch Envelope Optimization of Virtual Sliding Target Guidance Scheme

This paper presents an optimization of the performance of a recently proposed virtual sliding target (VST) guidance scheme in terms of maximization of its launch envelope for three- dimensional (3-D) engagements. The objective is to obtain the launch envelope of the missile using the VST guidance scheme for different lateral launch angles with respect to the line of sight (LOS) and demonstrate its superiority over kinematics-based guidance laws like proportional navigation (PN). The VST scheme uses PN as its basic guidance scheme and exploits the relation between the atmospheric properties, missile aerodynamic characteristics, and the optimal trajectory of the missile. The missile trajectory is shaped by controlling the instantaneous position and the speed of a virtual target which the missile pursues during the midcourse phase. In the proposed method it is shown that an appropriate value of initial position for the virtual target in 3-D, combined with optimized virtual target parameters, can significantly improve the launch envelope performance. The paper presents the formulation of the optimization problem, obtains the approximate models used to make the optimization problem more tractable, and finally presents the optimized performance of the missile in terms of launch envelope and shows significant improvement over kinematic-based guidance laws. The paper also proposes modification to the basic VST scheme. Some simulations using the full-fledged six degrees-of-freedom (6-DOF) models are also presented to validate the models and technique used.

Theoretical gain optimization in CO2-N2-H2 gasdynamic lasers with two-dimensional wedge nozzles

Attention is given to the results of optimization studies with a 16-micron CO2-N2-H2 GDL employing two-dimensional wedge nozzles. The optimum value of the achievable gain reaches 12.7 percent/cm on the P(15) line for a 30:50:20 percent respective apportionment of the aforementioned gases. The corresponding optimum values for reservoir pressure and area ratio are computed as functions of reservoir temperature, and presented graphically.

A multi-criteria approach for hospital capacity analysis

Hospitals are critical elements of health care systems and analysing their capacity to do work is a very important topic. To perform a system wide analysis of public hospital resources and capacity, a multi-objective optimization (MOO) approach has been proposed. This approach identifies the theoretical capacity of the entire hospital and facilitates a sensitivity analysis, for example of the patient case mix. It is necessary because the competition for hospital resources, for example between different entities, is highly influential on what work can be done. The MOO approach has been extensively tested on a real life case study and significant worth is shown. In this MOO approach, the epsilon constraint method has been utilized. However, for solving real life applications, with a large number of competing objectives, it was necessary to devise new and improved algorithms. In addition, to identify the best solution, a separable programming approach was developed. Multiple optimal solutions are also obtained via the iterative refinement and re-solution of the model.

Application of optimization methods to vibration control of stretched strings

In this paper, non-linear programming techniques are applied to the problem of controlling the vibration pattern of a stretched string. First, the problem of finding the magnitudes of two control forces applied at two points l1 and l2 on the string to reduce the energy of vibration over the interval (l1, l2) relative to the energy outside the interval (l1, l2) is considered. For this problem the relative merits of various methods of non-linear programming are compared. The more complicated problem of finding the positions and magnitudes of two control forces to obtain the desired energy pattern is then solved by using the slack unconstrained minimization technique with the Fletcher-Powell search. In the discussion of the results it is shown that the position of the control force is very important in controlling the energy pattern of the string.

Power Minimization for CDMA under Colored Noise

Rate-constrained power minimization (PMIN) over a code division multiple-access (CDMA) channel with correlated noise is studied. PMIN is. shown to be an instance of a separable convex optimization problem subject to linear ascending constraints. PMIN is further reduced to a dual problem of sum-rate maximization (RMAX). The results highlight the underlying unity between PMIN, RMAX, and a problem closely related to PMIN but with linear receiver constraints. Subsequently, conceptually simple sequence design algorithms are proposed to explicitly identify an assignment of sequences and powers that solve PMIN. The algorithms yield an upper bound of 2N - 1 on the number of distinct sequences where N is the processing gain. The sequences generated using the proposed algorithms are in general real-valued. If a rate-splitting and multi-dimensional CDMA approach is allowed, the upper bound reduces to N distinct sequences, in which case the sequences can form an orthogonal set and be binary +/- 1-valued.

A Proof of Convergence of the B-RED and P-RED Algorithms for Random Early Detection

In [8], we recently presented two computationally efficient algorithms named B-RED and P-RED for random early detection. In this letter, we present the mathematical proof of convergence of these algorithms under general conditions to local minima.