Disparity energy model with keypoint disparity validation

A biological disparity energy model can estimate local depth information by using a population of V1 complex cells. Instead of applying an analytical model which explicitly involves cell parameters like spatial frequency, orientation, binocular phase and position difference, we developed a model which only involves the cells’ responses, such that disparity can be extracted from a population code, using only a set of previously trained cells with random-dot stereograms of uniform disparity. Despite good results in smooth regions, the model needs complementary processing, notably at depth transitions. We therefore introduce a new model to extract disparity at keypoints such as edge junctions, line endings and points with large curvature. Responses of end-stopped cells serve to detect keypoints, and those of simple cells are used to detect orientations of their underlying line and edge structures. Annotated keypoints are then used in the leftright matching process, with a hierarchical, multi-scale tree structure and a saliency map to segregate disparity. By combining both models we can (re)define depth transitions and regions where the disparity energy model is less accurate.

A vision fronted with a new disparity model

We are developing a frontend that is based on the image representation in the visual cortex and plausible processing schemes. This frontend consists of multiscale line/edge and keypoint (vertex) detection, using models of simple, complex and end-stopped cells. This frontend is being extended by a new disparity model. Assuming that there is no neural inverse tangent operator, we do not exploit Gabor phase information. Instead, we directly use simple cell (Gabor) responses at positions where lines and edges are detected.

A disparity energy model improved by line, edge and keypoint correspondences

Disparity energy models (DEMs) estimate local depth information on the basis ofVl complex cells. Our recent DEM (Martins et al, 2011 ISSPlT261-266) employs a population code. Once the population's cells have been trained with randorn-dot stereograms, it is applied at all retinotopic positions in the visual field. Despite producing good results in textured regions, the model needs to be made more precise, especially at depth transitions.