A survey of human motion analysis using depth imagery

Analysis of human behaviour through visual information has been a highly active research topic in the computer vision community. This was previously achieved via images from a conventional camera, but recently depth sensors have made a new type of data available. This survey starts by explaining the advantages of depth imagery, then describes the new sensors that are available to obtain it. In particular, the Microsoft Kinect has made high-resolution real-time depth cheaply available. The main published research on the use of depth imagery for analysing human activity is reviewed. Much of the existing work focuses on body part detection and pose estimation. A growing research area addresses the recognition of human actions. The publicly available datasets that include depth imagery are listed, as are the software libraries that can acquire it from a sensor. This survey concludes by summarising the current state of work on this topic, and pointing out promising future research directions.

Measuring motion with kinematically redundant accelerometer arrays: theory, simulation and implementation

This work presents two schemes of measuring the linear and angular kinematics of a rigid body using a kinematically redundant array of triple-axis accelerometers with potential applications in biomechanics. A novel angular velocity estimation algorithm is proposed and evaluated that can compensate for angular velocity errors using measurements of the direction of gravity. Analysis and discussion of optimal sensor array characteristics are provided. A damped 2 axis pendulum was used to excite all 6 DoF of the a suspended accelerometer array through determined complex motion and is the basis of both simulation and experimental studies. The relationship between accuracy and sensor redundancy is investigated for arrays of up to 100 triple axis (300 accelerometer axes) accelerometers in simulation and 10 equivalent sensors (30 accelerometer axes) in the laboratory test rig. The paper also reports on the sensor calibration techniques and hardware implementation.

Modeling image motion in airborne Three-Line-Array (TLA) push-broom cameras

This paper presents an image motion model for airborne three-line-array (TLA) push-broom cameras. Both aircraft velocity and attitude instability are taken into account in modeling image motion. Effects of aircraft pitch, roll, and yaw on image motion are analyzed based on geometric relations in designated coordinate systems. The image motion is mathematically modeled by image motion velocity multiplied by exposure time. Quantitative analysis to image motion velocity is then conducted in simulation experiments. The results have shown that image motion caused by aircraft velocity is space invariant while image motion caused by aircraft attitude instability is more complicated. Pitch,roll and yaw all contribute to image motion to different extents. Pitch dominates the along-track image motion and both roll and yaw greatly contribute to the cross-track image motion. These results provide a valuable base for image motion compensation to ensure high accuracy imagery in aerial photogrammetry.

Motion planning of autonomous vehicles in a non-autonomous vehicle environment without speed lanes

Planning is one of the key problems for autonomous vehicles operating in road scenarios. Present planning algorithms operate with the assumption that traffic is organised in predefined speed lanes, which makes it impossible to allow autonomous vehicles in countries with unorganised traffic. Unorganised traffic is though capable of higher traffic bandwidths when constituting vehicles vary in their speed capabilities and sizes. Diverse vehicles in an unorganised exhibit unique driving behaviours which are analysed in this paper by a simulation study. The aim of the work reported here is to create a planning algorithm for mixed traffic consisting of both autonomous and non-autonomous vehicles without any inter-vehicle communication. The awareness (e.g. vision) of every vehicle is restricted to nearby vehicles only and a straight infinite road is assumed for decision making regarding navigation in the presence of multiple vehicles. Exhibited behaviours include obstacle avoidance, overtaking, giving way for vehicles to overtake from behind, vehicle following, adjusting the lateral lane position and so on. A conflict of plans is a major issue which will almost certainly arise in the absence of inter-vehicle communication. Hence each vehicle needs to continuously track other vehicles and rectify plans whenever a collision seems likely. Further it is observed here that driver aggression plays a vital role in overall traffic dynamics, hence this has also been factored in accordingly. This work is hence a step forward towards achieving autonomous vehicles in unorganised traffic, while similar effort would be required for planning problems such as intersections, mergers, diversions and other modules like localisation.

Eye candy for the blind: re-introducing Lyotard's Acinema into discourses on excess, motion and spectacle in contemporary Hollywood

Jean-François Lyotard's 1973 essay ‘Acinema’ is explicitly concerned with the cinematic medium, but has received scant critical attention. Lyotard's acinema conceives of an experimental, excessive form of film-making that uses stillness and movement to shift away from the orderly process of meaning-making within mainstream cinema. What motivates this present paper is a striking link between Lyotard's writing and contemporary Hollywood production; both are concerned with a sense of excess, especially within moments of motion. Using Charlie's Angels (McG, 2000) as a case study – a film that has been critically dismissed as ‘eye candy for the blind’ – my methodology brings together two different discourses, high culture theory and mainstream film-making, to test out and propose the value of Lyotard's ideas for the study of contemporary film. Combining close textual analysis and engagement with key scholarship on film spectacle, I reflexively engage with the process of film analysis and re-direct attention to a neglected essay by a major theorist, in order to stimulate further engagement with his work.

Default perception of high-speed motion

When human observers are exposed to even slight motion signals followed by brief visual transients—stimuli containing no detectable coherent motion signals—they perceive large and salient illusory jumps. This novel effect, which we call “high phi”, challenges well-entrenched assumptions about the perception of motion, namely the minimal-motion principle and the breakdown of coherent motion perception with steps above an upper limit. Our experiments with transients such as texture randomization or contrast reversal show that the magnitude of the jump depends on spatial frequency and transient duration, but not on the speed of the inducing motion signals, and the direction of the jump depends on the duration of the inducer. Jump magnitude is robust across jump directions and different types of transient. In addition, when a texture is actually displaced by a large step beyond dmax, a breakdown of coherent motion perception is expected, but in the presence of an inducer observers again perceive coherent displacements at or just above dmax. In sum, across a large variety of stimuli, we find that when incoherent motion noise is preceded by a small bias, instead of perceiving little or no motion, as suggested by the minimal-motion principle, observers perceive jumps whose amplitude closely follows their own dmax limits.

Does grammatical aspect affect motion event cognition? A cross-linguistic comparison of English and Swedish speakers

In this article, we explore whether cross-linguistic differences in grammatical aspect encoding may give rise to differences in memory and cognition. We compared native speakers of two languages that encode aspect differently (English and Swedish) in four tasks that examined verbal descriptions of stimuli, online triads matching, and memory-based triads matching with and without verbal interference. Results showed between-group differences in verbal descriptions and in memory-based triads matching. However, no differences were found in online triads matching and in memory-based triads matching with verbal interference. These findings need to be interpreted in the context of the overall pattern of performance, which indicated that both groups based their similarity judgments on common perceptual characteristics of motion events. These results show for the first time a cross-linguistic difference in memory as a function of differences in grammatical aspect encoding, but they also contribute to the emerging view that language fine tunes rather than shapes perceptual processes that are likely to be universal and unchanging.

Mechanism for synchronized motion between two humans in mutual tapping experiments: Transition from alternative mode to synchronization mode

We performed mutual tapping experiments between two humans to investigate the conditions required for synchronized motion. A transition from an alternative mode to a synchronization mode was discovered under the same conditions when a subject changed from a reactive mode to an anticipation mode in single tapping experiments. Experimental results suggest that the cycle time for each tapping motion is tuned by a proportional control that is based on synchronization errors and cycle time errors. As the tapping frequency increases, the mathematical model based on the feedback control in the sensory-motor closed loop predicts a discrete mode transition as the gain factors of the proportional control decease. The conditions of the synchronization were shown as a consequence of the coupled dynamics based on the subsequent feedback loop in the sensory-motor system.

Motion event cognition and grammatical aspect: evidence from Afrikaans

Research on the relationship between grammatical aspect and motion event construal has posited that speakers of non-aspect languages are more prone to encoding event endpoints than are speakers of aspect languages (e.g., von Stutterheim and Carroll 2011). In the present study, we test this hypothesis by extending this line of inquiry to Afrikaans, a non-aspect language which is previously unexplored in this regard. Motion endpoint behavior among Afrikaans speakers was measured by means of a linguistic retelling task and a non-linguistic similarity judgment task, and then compared with the behavior of speakers of a non-aspect language (Swedish) and speakers of an aspect language (English). Results showed the Afrikaans speakers' endpoint patterns aligned with Swedish patterns, but were significantly different from English patterns. It was also found that the variation among the Afrikaans speakers could be partially explained by taking into account their frequency of use of English, such that those who used English more frequently exhibited an endpoint behavior that was more similar to English speakers. The current study thus lends further support to the hypothesis that speakers of different languages attend differently to event endpoints as a function of the grammatical category of aspect.

Estimation of visual motion parameters used for bio-inspired navigation

Visual motion cues play an important role in animal and humans locomotion without the need to extract actual ego-motion information. This paper demonstrates a method for estimating the visual motion parameters, namely the Time-To-Contact (TTC), Focus of Expansion (FOE), and image angular velocities, from a sparse optical flow estimation registered from a downward looking camera. The presented method is capable of estimating the visual motion parameters in a complicated 6 degrees of freedom motion and in real time with suitable accuracy for mobile robots visual navigation.

Biologically-inspired robust motion segmentation using mutual information

This paper presents a neuroscience inspired information theoretic approach to motion segmentation. Robust motion segmentation represents a fundamental first stage in many surveillance tasks. As an alternative to widely adopted individual segmentation approaches, which are challenged in different ways by imagery exhibiting a wide range of environmental variation and irrelevant motion, this paper presents a new biologically-inspired approach which computes the multivariate mutual information between multiple complementary motion segmentation outputs. Performance evaluation across a range of datasets and against competing segmentation methods demonstrates robust performance.

Dynamic distributed lanes: motion planning for multiple autonomous vehicles

Unorganized traffic is a generalized form of travel wherein vehicles do not adhere to any predefined lanes and can travel in-between lanes. Such travel is visible in a number of countries e.g. India, wherein it enables a higher traffic bandwidth, more overtaking and more efficient travel. These advantages are visible when the vehicles vary considerably in size and speed, in the absence of which the predefined lanes are near-optimal. Motion planning for multiple autonomous vehicles in unorganized traffic deals with deciding on the manner in which every vehicle travels, ensuring no collision either with each other or with static obstacles. In this paper the notion of predefined lanes is generalized to model unorganized travel for the purpose of planning vehicles travel. A uniform cost search is used for finding the optimal motion strategy of a vehicle, amidst the known travel plans of the other vehicles. The aim is to maximize the separation between the vehicles and static obstacles. The search is responsible for defining an optimal lane distribution among vehicles in the planning scenario. Clothoid curves are used for maintaining a lane or changing lanes. Experiments are performed by simulation over a set of challenging scenarios with a complex grid of obstacles. Additionally behaviours of overtaking, waiting for a vehicle to cross and following another vehicle are exhibited.

The role of the harmonic vector average in motion integration

The local speeds of object contours vary systematically with the cosine of the angle between the normal component of the local velocity and the global object motion direction. An array of Gabor elements whose speed changes with local spatial orientation in accordance with this pattern can appear to move as a single surface. The apparent direction of motion of plaids and Gabor arrays has variously been proposed to result from feature tracking, vector addition and vector averaging in addition to the geometrically correct global velocity as indicated by the intersection of constraints (IOC) solution. Here a new combination rule, the harmonic vector average (HVA), is introduced, as well as a new algorithm for computing the IOC solution. The vector sum can be discounted as an integration strategy as it increases with the number of elements. The vector average over local vectors that vary in direction always provides an underestimate of the true global speed. The HVA, however, provides the correct global speed and direction for an unbiased sample of local velocities with respect to the global motion direction, as is the case for a simple closed contour. The HVA over biased samples provides an aggregate velocity estimate that can still be combined through an IOC computation to give an accurate estimate of the global velocity, which is not true of the vector average. Psychophysical results for type II Gabor arrays show perceived direction and speed falls close to the IOC direction for Gabor arrays having a wide range of orientations but the IOC prediction fails as the mean orientation shifts away from the global motion direction and the orientation range narrows. In this case perceived velocity generally defaults to the HVA.

Motion of the dayside polar cap boundary during substorm cycles: II. Generation of poleward-moving events and polar cap patches by pulses in the magnetopause reconnection rate

Using data from the EISCAT (European Incoherent Scatter) VHF and CUTLASS (Co-operative UK Twin- Located Auroral Sounding System) HF radars, we study the formation of ionospheric polar cap patches and their relationship to the magnetopause reconnection pulses identified in the companion paper by Lockwood et al. (2005). It is shown that the poleward-moving, high-concentration plasma patches observed in the ionosphere by EISCAT on 23 November 1999, as reported by Davies et al. (2002), were often associated with corresponding reconnection rate pulses. However, not all such pulses generated a patch and only within a limited MLT range (11:00–12:00 MLT) did a patch result from a reconnection pulse. Three proposed mechanisms for the production of patches, and of the concentration minima that separate them, are analysed and evaluated: (1) concentration enhancement within the patches by cusp/cleft precipitation; (2) plasma depletion in the minima between the patches by fast plasma flows; and (3) intermittent injection of photoionisation-enhanced plasma into the polar cap. We devise a test to distinguish between the effects of these mechanisms. Some of the events repeat too frequently to apply the test. Others have sufficiently long repeat periods and mechanism (3) is shown to be the only explanation of three of the longer-lived patches seen on this day. However, effect (2) also appears to contribute to some events. We conclude that plasma concentration gradients on the edges of the larger patches arise mainly from local time variations in the subauroral plasma, via the mechanism proposed by Lockwood et al. (2000).

Motion of the dayside polar cap boundary during substorm cycles: I. Observations of pulses in the magnetopause reconnection rate

Using data from the EISCAT (European Incoherent Scatter) VHF radar and DMSP (Defense Meteorological Satellite Program) spacecraft passes, we study the motion of the dayside open-closed field line boundary during two substorm cycles. The satellite data show that the motions of ion and electron temperature boundaries in EISCAT data, as reported by Moen et al. (2004), are not localised around the radar; rather, they reflect motions of the open-closed field line boundary at all MLT throughout the dayside auroral ionosphere. The boundary is shown to erode equatorward when the IMF points southward, consistent with the effect of magnetopause reconnection. During the substorm expansion and recovery phases, the dayside boundary returns poleward, whether the IMF points northward or southward. However, the poleward retreat was much faster during the substorm for which the IMF had returned to northward than for the substorm for which the IMF remained southward – even though the former substorm is much the weaker of the two. These poleward retreats are consistent with the destruction of open flux at the tail current sheet. Application of a new analysis of the peak ion energies at the equatorward edge of the cleft/cusp/mantle dispersion seen by the DMSP satellites identifies the dayside reconnection merging gap to extend in MLT from about 9.5 to 15.5 h for most of the interval. Analysis of the boundary motion, and of the convection velocities seen near the boundary by EISCAT, allows calculation of the reconnection rate (mapped down to the ionosphere) from the flow component normal to the boundary in its own rest frame. This reconnection rate is not, in general, significantly different from zero before 06:45 UT (MLT<9.5 h) – indicating that the X line footprint expands over the EISCAT field-of-view to earlier MLT only occasionally and briefly. Between 06:45 UT and 12:45UT (9.5