Measurement of complex optical flow with use of an augmented generalized gradient scheme

We present a method for measuring the local velocities and first-order variations in velocities in a timevarying image. The scheme is an extension of the generalized gradient model that encompasses the local variation of velocity within a local patch of the image. Motion within a patch is analyzed in parallel by 42 different spatiotemporal filters derived from 6 linearly independent spatiotemporal kernels. No constraints are imposed on the image structure, and there is no need for smoothness constraints on the velocity field. The aperture problem does not arise so long as there is some two-dimensional structure in the patch being analyzed. Among the advantages of the scheme is that there is no requirement to calculate second or higher derivatives of the image function. This makes the scheme robust in the presence of noise. The spatiotemporal kernels are of simple form, involving Gaussian functions, and are biologically plausible receptive fields. The validity of the scheme is demonstrated by application to both synthetic and real video images sequences and by direct comparison with another recently published scheme Biol. Cybern. 63, 185 (1990)] for the measurement of complex optical flow.

Measurement of complex optical flow with use of an augmented generalized gradient scheme

We present a method for measuring the local velocities and first-order variations in velocities in a time-varying image. The scheme is an extension of the generalized gradient model that encompasses the local variation of velocity within a local patch of the image. Motion within a patch is analyzed in parallel by 42 different spatiotemporal filters derived from 6 linearly independent spatiotemporal kernels. No constraints are imposed on the image structure, and there is no need for smoothness constraints on the velocity field. The aperture problem does not arise so long as there is some two-dimensional structure in the patch being analyzed. Among the advantages of the scheme is that there is no requirement to calculate second or higher derivatives of the image function. This makes the scheme robust in the presence of noise. The spatiotemporal kernels are of simple form, involving Gaussian functions, and are biologically plausible receptive fields. The validity of the scheme is demonstrated by application to both synthetic and real video images sequences and by direct comparison with another recently published scheme [Biol. Cybern. 63, 185 (1990)] for the measurement of complex optical flow.

On The Uniqueness of Correspondence Under Orthographic and Perspective Projections

The task of shape recovery from a motion sequence requires the establishment of correspondence between image points. The two processes, the matching process and the shape recovery one, are traditionally viewed as independent. Yet, information obtained during the process of shape recovery can be used to guide the matching process. This paper discusses the mutual relationship between the two processes. The paper is divided into two parts. In the first part we review the constraints imposed on the correspondence by rigid transformations and extend them to objects that undergo general affine (non rigid) transformation (including stretch and shear), as well as to rigid objects with smooth surfaces. In all these cases corresponding points lie along epipolar lines, and these lines can be recovered from a small set of corresponding points. In the second part of the paper we discuss the potential use of epipolar lines in the matching process. We present an algorithm that recovers the correspondence from three contour images. The algorithm was implemented and used to construct object models for recognition. In addition we discuss how epipolar lines can be used to solve the aperture problem.

Temporal Dynamics of Decision-Making during Motion Perception in the Visual Cortex

How does the brain make decisions? Speed and accuracy of perceptual decisions covary with certainty in the input, and correlate with the rate of evidence accumulation in parietal and frontal cortical "decision neurons." A biophysically realistic model of interactions within and between Retina/LGN and cortical areas V1, MT, MST, and LIP, gated by basal ganglia, simulates dynamic properties of decision-making in response to ambiguous visual motion stimuli used by Newsome, Shadlen, and colleagues in their neurophysiological experiments. The model clarifies how brain circuits that solve the aperture problem interact with a recurrent competitive network with self-normalizing choice properties to carry out probablistic decisions in real time. Some scientists claim that perception and decision-making can be described using Bayesian inference or related general statistical ideas, that estimate the optimal interpretation of the stimulus given priors and likelihoods. However, such concepts do not propose the neocortical mechanisms that enable perception, and make decisions. The present model explains behavioral and neurophysiological decision-making data without an appeal to Bayesian concepts and, unlike other existing models of these data, generates perceptual representations and choice dynamics in response to the experimental visual stimuli. Quantitative model simulations include the time course of LIP neuronal dynamics, as well as behavioral accuracy and reaction time properties, during both correct and error trials at different levels of input ambiguity in both fixed duration and reaction time tasks. Model MT/MST interactions compute the global direction of random dot motion stimuli, while model LIP computes the stochastic perceptual decision that leads to a saccadic eye movement.

Neural Models of Motion Integration, Segmentation, and Probablistic Decision-Making

When brain mechanism carry out motion integration and segmentation processes that compute unambiguous global motion percepts from ambiguous local motion signals? Consider, for example, a deer running at variable speeds behind forest cover. The forest cover is an occluder that creates apertures through which fragments of the deer's motion signals are intermittently experienced. The brain coherently groups these fragments into a trackable percept of the deer in its trajectory. Form and motion processes are needed to accomplish this using feedforward and feedback interactions both within and across cortical processing streams. All the cortical areas V1, V2, MT, and MST are involved in these interactions. Figure-ground processes in the form stream through V2, such as the seperation of occluding boundaries of the forest cover from the boundaries of the deer, select the motion signals which determine global object motion percepts in the motion stream through MT. Sparse, but unambiguous, feauture tracking signals are amplified before they propogate across position and are intergrated with far more numerous ambiguous motion signals. Figure-ground and integration processes together determine the global percept. A neural model predicts the processing stages that embody these form and motion interactions. Model concepts and data are summarized about motion grouping across apertures in response to a wide variety of displays, and probabilistic decision making in parietal cortex in response to random dot displays.

Laminar Cortical Dynamics of Visual Form and Motion Interactions During Coherent Object Motion Perception

How do visual form and motion processes cooperate to compute object motion when each process separately is insufficient? Consider, for example, a deer moving behind a bush. Here the partially occluded fragments of motion signals available to an observer must be coherently grouped into the motion of a single object. A 3D FORMOTION model comprises five important functional interactions involving the brain’s form and motion systems that address such situations. Because the model’s stages are analogous to areas of the primate visual system, we refer to the stages by corresponding anatomical names. In one of these functional interactions, 3D boundary representations, in which figures are separated from their backgrounds, are formed in cortical area V2. These depth-selective V2 boundaries select motion signals at the appropriate depths in MT via V2-to-MT signals. In another, motion signals in MT disambiguate locally incomplete or ambiguous boundary signals in V2 via MT-to-V1-to-V2 feedback. The third functional property concerns resolution of the aperture problem along straight moving contours by propagating the influence of unambiguous motion signals generated at contour terminators or corners. Here, sparse “feature tracking signals” from, e.g., line ends, are amplified to overwhelm numerically superior ambiguous motion signals along line segment interiors. In the fourth, a spatially anisotropic motion grouping process takes place across perceptual space via MT-MST feedback to integrate veridical feature-tracking and ambiguous motion signals to determine a global object motion percept. The fifth property uses the MT-MST feedback loop to convey an attentional priming signal from higher brain areas back to V1 and V2. The model's use of mechanisms such as divisive normalization, endstopping, cross-orientation inhibition, and longrange cooperation is described. Simulated data include: the degree of motion coherence of rotating shapes observed through apertures, the coherent vs. element motion percepts separated in depth during the chopsticks illusion, and the rigid vs. non-rigid appearance of rotating ellipses.

Cortical Dynamics of Visual Motion Perception: Short-Range and Long Range Apparent Motion

This article describes further evidence for a new neural network theory of biological motion perception that is called a Motion Boundary Contour System. This theory clarifies why parallel streams Vl-> V2 and Vl-> MT exist for static form and motion form processing among the areas Vl, V2, and MT of visual cortex. The Motion Boundary Contour System consists of several parallel copies, such that each copy is activated by a different range of receptive field sizes. Each copy is further subdivided into two hierarchically organized subsystems: a Motion Oriented Contrast Filter, or MOC Filter, for preprocessing moving images; and a Cooperative-Competitive Feedback Loop, or CC Loop, for generating emergent boundary segmentations of the filtered signals. The present article uses the MOC Filter to explain a variety of classical and recent data about short-range and long-range apparent motion percepts that have not yet been explained by alternative models. These data include split motion; reverse-contrast gamma motion; delta motion; visual inertia; group motion in response to a reverse-contrast Ternus display at short interstimulus intervals; speed-up of motion velocity as interfiash distance increases or flash duration decreases; dependence of the transition from element motion to group motion on stimulus duration and size; various classical dependencies between flash duration, spatial separation, interstimulus interval, and motion threshold known as Korte's Laws; and dependence of motion strength on stimulus orientation and spatial frequency. These results supplement earlier explanations by the model of apparent motion data that other models have not explained; a recent proposed solution of the global aperture problem, including explanations of motion capture and induced motion; an explanation of how parallel cortical systems for static form perception and motion form perception may develop, including a demonstration that these parallel systems are variations on a common cortical design; an explanation of why the geometries of static form and motion form differ, in particular why opposite orientations differ by 90°, whereas opposite directions differ by 180°, and why a cortical stream Vl -> V2 -> MT is needed; and a summary of how the main properties of other motion perception models can be assimilated into different parts of the Motion Boundary Contour System design.

The Perception of Globally Coherent Motion

How do human observers perceive a coherent pattern of motion from a disparate set of local motion measures? Our research has examined how ambiguous motion signals along straight contours are spatially integrated to obtain a globally coherent perception of motion. Observers viewed displays containing a large number of apertures, with each aperture containing one or more contours whose orientations and velocities could be independently specified. The total pattern of the contour trajectories across the individual apertures was manipulated to produce globally coherent motions, such as rotations, expansions, or translations. For displays containing only straight contours extending to the circumferences of the apertures, observers' reports of global motion direction were biased whenever the sampling of contour orientations was asymmetric relative to the direction of motion. Performance was improved by the presence of identifiable features, such as line ends or crossings, whose trajectories could be tracked over time. The reports of our observers were consistent with a pooling process involving a vector average of measures of the component of velocity normal to contour orientation, rather than with the predictions of the intersection-of-constraints analysis in velocity space.

Investigation of the intersection of constraints model of motion perception

The direction and speed of motion of a one-dimensional (1-D) stimulus, such as a grating, presented within a circular aperture is ambiguous. This ambiguity, referred to as the Aperture Problem (Fennema & Thompson, 1979) results from (i) the inability to detect motion parallel to grating orientation, and (ii) the occlusion of border information, such as the ‘ends’ of the grating, by the surface forming the aperture, Adelson and Movshon's (1982) intcrsection-of-constraints (IOC) model of motion perception describes a two-stage method of disambiguating the motion of 1-D moving stimuli (e.g., gratings) to produce unambiguous motion of two-dimensional (2-D) objects (e.g., plaid patterns) made up of several 1-D components. Specifically, in the IOC model ambiguous 1-D motions extracted by Stage 1 component-selective mechanisms are integrated by Stage 2 pattern-selective mechanisms to produce unambiguous 2-D motion signals. ‘Integration’ in the context of the IOC model involves determining the single motion vector (i.e., combination of direction and speed) which is consistent with the I-D components of a 2-D object. Since the IOC model assumes that 2-D objects undergo pure translation (i.e., without distortion, rotation, etc.), the motion vector consistent with all 1-D components describes the motion of the 2-D object itself. Adelson and Movshon (1982) propose that neural implementation of the computation underlying the IOC model is reflected in the perception of coherent 2-D plaid motion reported when two separately-moving ‘component’ gratings are superimposed. Using these plaid patterns the present thesis assesses the IOC model in terms of its ability to account for the perception of 2-D motion in a variety of circumstances. In the first series of experiments it is argued that the unambiguous motion perceived for a single grating presented within a rectangular aperture (i.e., the Barberpole illusion; Wallach, 1976) reflects application of the IOC computation to the moving 1-D grating and the stationary boundary of the aperture. While contrary to the assumption which underlies the IOC model (viz., that integration occurs between moving 1-D stimuli), evidence consistent with the involvement of the IOC computation in mediating the Barberpole illusion (in which there is only one moving stimulus) is obtained by measuring plaid coherence as a function of aperture shape. It is found that rectangular apertures which bias perceived component motions in directions consistent with plaid direction facilitate plaid coherence, while rectangular apertures which bias perceived component motions in directions inconsistent with plaid direction disrupt plaid coherence. In the second series of experiments, perceived directions of motion of type I symmetrical, type I asymmetrical, and type II plaids are measured with the aim of investigating the deviations in plaid directions reported by Ferrera and Wilson (1990) and Yo and Wilson (1992). Perceived directions of both asymmetrical and type II plaids are shown to deviate away from lOC-predicted directions and towards mean component direction. Furthermore, the magnitude of these deviations is being proportional to the difference between lOC-predicted plaid direction and mean component direction. On the basis of these directional deviations, modification to the IOC model is proposed. In the modified IOC model it is argued that plaid perception involves (i) the activity of Stage 2 pattern-selective mechanisms (and the Stage 1 component-selective mechanisms which input into these pattern-selective mechanisms) involved in implementing the IOC computation, and (ii) component-selective mechanisms which influence plaid perception directly, and ‘extraneously’ to the IOC computation. In the third series of experiments the validity of this modified IOC model, as well as the validity of alternative one-stage models of plaid perception are assessed in relation to perceived directions of plaid-induced MAEs as a function of both plaid direction and mean component direction. It is found that plaid-induced MAEs are shifted away from directions opposite to lOC-predicted plaid direction towards the direction opposite to mean component direction. This pattern of results is taken to be consistent with the modified IOC model which predicts the activity, and adaptation both of mechanisms signalling plaid direction (via implementation of the IOC computation), and ‘extraneous-type’ component-selective mechanisms signalling component directions. Alternative one-stage models which predict the adaptation of only mechanisms signalling plaid direction (the feature-tracking model), or the adaptation only of mechanisms signalling component directions (the distribution-of-activity model), cannot account for the directions of plaid-induced MAEs reported. The ability of the modified IOC model to account for the perceived directions of (i) gratings in rectangular apertures, (ii) various types of plaid in circular apertures, and (iii) directions of plaid-induced MAEs, is interpreted as supporting the proposition that human motion perception is based on a parallel and distributed process involving Stage 2 pattern-selective mechanisms (and the Stage 1 component-selective mechanisms which input into these mechanisms) taken to implement the IOC computation, and component-selective mechanisms taken to provide an 'extraneous' direct contribution to motion perception.

Solving the problem of diffraction of an optical-band electromagnetic wave by metal nanostructured aperture arrays with the use of the method of impedance boundary conditions

The problem of diffraction of an optical wave by a 2D periodic metal aperture array with square, circular, and ring apertures is solved with allowance for the finite permittivity of a metal in the optical band. The correctness of the obtained results is verified through comparison with experimental data. It is shown that the transmission coefficient can be substantially greater than the corresponding value reached in the case of diffraction by a grating in a perfectly conducting screen.

Automatic Target Recognition in Synthetic Aperture Radar Imagery: A State-of-the-Art Review

The purpose of this paper is to survey and assess the state-of-the-art in automatic target recognition for synthetic aperture radar imagery (SAR-ATR). The aim is not to develop an exhaustive survey of the voluminous literature, but rather to capture in one place the various approaches for implementing the SAR-ATR system. This paper is meant to be as self-contained as possible, and it approaches the SAR-ATR problem from a holistic end-to-end perspective. A brief overview for the breadth of the SAR-ATR challenges is conducted. This is couched in terms of a single-channel SAR, and it is extendable to multi-channel SAR systems. Stages pertinent to the basic SAR-ATR system structure are defined, and the motivations of the requirements and constraints on the system constituents are addressed. For each stage in the SAR-ATR processing chain, a taxonomization methodology for surveying the numerous methods published in the open literature is proposed. Carefully selected works from the literature are presented under the taxa proposed. Novel comparisons, discussions, and comments are pinpointed throughout this paper. A two-fold benchmarking scheme for evaluating existing SAR-ATR systems and motivating new system designs is proposed. The scheme is applied to the works surveyed in this paper. Finally, a discussion is presented in which various interrelated issues, such as standard operating conditions, extended operating conditions, and target-model design, are addressed. This paper is a contribution toward fulfilling an objective of end-to-end SAR-ATR system design.

A unified mixed-integer programming model for simultaneous fluence weight and aperture optimization in VMAT, tomotherapy, and cyberknife

All rights reserved. In this paper, we propose and study a unified mixed-integer programming model that simultaneously optimizes fluence weights and multi-leaf collimator (MLC) apertures in the treatment planning optimization of VMAT, Tomotherapy, and CyberKnife. The contribution of our model is threefold: (i) Our model optimizes the fluence and MLC apertures simultaneously for a given set of control points. (ii) Our model can incorporate all volume limits or dose upper bounds for organs at risk (OAR) and dose lower bound limits for planning target volumes (PTV) as hard constraints, but it can also relax either of these constraint sets in a Lagrangian fashion and keep the other set as hard constraints. (iii) For faster solutions, we propose several heuristic methods based on the MIP model, as well as a meta-heuristic approach. The meta-heuristic is very efficient in practice, being able to generate dose- and machinery-feasible solutions for problem instances of clinical scale, e.g., obtaining feasible treatment plans to cases with 180 control points, 6750 sample voxels and 18,000 beamlets in 470 seconds, or cases with 72 control points, 8000 sample voxels and 28,800 beamlets in 352 seconds. With discretization and down-sampling of voxels, our method is capable of tackling a treatment field of 8000-64,000cm3, depending on the ratio of critical structure versus unspecified tissues.