Swimmer localization from a moving camera

At the highest level of competitive sport, nearly all performances of athletes (both training and competitive) are chronicled using video. Video is then often viewed by expert coaches/analysts who then manually label important performance indicators to gauge performance. Stroke-rate and pacing are important performance measures in swimming, and these are previously digitised manually by a human. This is problematic as annotating large volumes of video can be costly, and time-consuming. Further, since it is difficult to accurately estimate the position of the swimmer at each frame, measures such as stroke rate are generally aggregated over an entire swimming lap. Vision-based techniques which can automatically, objectively and reliably track the swimmer and their location can potentially solve these issues and allow for large-scale analysis of a swimmer across many videos. However, the aquatic environment is challenging due to fluctuations in scene from splashes, reflections and because swimmers are frequently submerged at different points in a race. In this paper, we temporally segment races into distinct and sequential states, and propose a multimodal approach which employs individual detectors tuned to each race state. Our approach allows the swimmer to be located and tracked smoothly in each frame despite a diverse range of constraints. We test our approach on a video dataset compiled at the 2012 Australian Short Course Swimming Championships.

Selective combination of visual and thermal imaging for resilient localization in adverse conditions : day and night, smoke and fire

Long-term autonomy in robotics requires perception systems that are resilient to unusual but realistic conditions that will eventually occur during extended missions. For example, unmanned ground vehicles (UGVs) need to be capable of operating safely in adverse and low-visibility conditions, such as at night or in the presence of smoke. The key to a resilient UGV perception system lies in the use of multiple sensor modalities, e.g., operating at different frequencies of the electromagnetic spectrum, to compensate for the limitations of a single sensor type. In this paper, visual and infrared imaging are combined in a Visual-SLAM algorithm to achieve localization. We propose to evaluate the quality of data provided by each sensor modality prior to data combination. This evaluation is used to discard low-quality data, i.e., data most likely to induce large localization errors. In this way, perceptual failures are anticipated and mitigated. An extensive experimental evaluation is conducted on data sets collected with a UGV in a range of environments and adverse conditions, including the presence of smoke (obstructing the visual camera), fire, extreme heat (saturating the infrared camera), low-light conditions (dusk), and at night with sudden variations of artificial light. A total of 240 trajectory estimates are obtained using five different variations of data sources and data combination strategies in the localization method. In particular, the proposed approach for selective data combination is compared to methods using a single sensor type or combining both modalities without preselection. We show that the proposed framework allows for camera-based localization resilient to a large range of low-visibility conditions.

Localization of glucocorticoid receptors at postsynaptic membranes in the lateral amygdala

Glucocorticoids, released in high concentrations from the adrenal cortex during stressful experiences, bind to glucocorticoid receptors in nuclear and peri-nuclear sites in neuronal somata. Their classically known mode of action is to induce gene promoter receptors to alter gene transcription. Nuclear glucocorticoid receptors are particularly dense in brain regions crucial for memory, including memory of stressful experiences, such as the hippocampus and amygdala. While it has been proposed that glucocorticoids may also act via membrane bound receptors, the existence of the latter remains controversial. Using electron microscopy, we found glucocorticoid receptors localized to non-genomic sites in rat lateral amygdala, glia processes, presynaptic terminals, neuronal dendrites, and dendritic spines including spine organelles and postsynaptic membrane densities. The lateral nucleus of the amygdala is a region specifically implicated in the formation of memories for stressful experiences. These newly observed glucocorticoid receptor immunoreactive sites were in addition to glucocorticoid receptor immunoreactive signals observed using electron and confocal microscopy in lateral amygdala principal neuron and GABA neuron soma and nuclei, cellular domains traditionally associated with glucocorticoid immunoreactivity. In lateral amygdala, glucocorticoid receptors are thus also localized to non-nuclear-membrane translocation sites, particularly dendritic spines, where they show an affinity for postsynaptic membrane densities, and may have a specialized role in modulating synaptic transmission plasticity related to fear and emotional memory.

Towards training-free appearance-based localization : probabilistic models for whole-image descriptors

Whole image descriptors have recently been shown to be remarkably robust to perceptual change especially compared to local features. However, whole-image-based localization systems typically rely on heuristic methods for determining appropriate matching thresholds in a particular environment. These environment-specific tuning requirements and the lack of a meaningful interpretation of these arbitrary thresholds limits the general applicability of these systems. In this paper we present a Bayesian model of probability for whole-image descriptors that can be seamlessly integrated into localization systems designed for probabilistic visual input. We demonstrate this method using CAT-Graph, an appearance-based visual localization system originally designed for a FAB-MAP-style probabilistic input. We show that using whole-image descriptors as visual input extends CAT-Graph’s functionality to environments that experience a greater amount of perceptual change. We also present a method of estimating whole-image probability models in an online manner, removing the need for a prior training phase. We show that this online, automated training method can perform comparably to pre-trained, manually tuned local descriptor methods.

Real-time image classification for adaptive mission planning using an Autonomous Underwater Vehicle

Real-time image analysis and classification onboard robotic marine vehicles, such as AUVs, is a key step in the realisation of adaptive mission planning for large-scale habitat mapping in previously unexplored environments. This paper describes a novel technique to train, process, and classify images collected onboard an AUV used in relatively shallow waters with poor visibility and non-uniform lighting. The approach utilises Förstner feature detectors and Laws texture energy masks for image characterisation, and a bag of words approach for feature recognition. To improve classification performance we propose a usefulness gain to learn the importance of each histogram component for each class. Experimental results illustrate the performance of the system in characterisation of a variety of marine habitats and its ability to operate onboard an AUV's main processor suitable for real-time mission planning.

A survey of control allocation methods for underwater vehicles

A control allocation system implements a function that maps the desired control forces generated by the vehicle motion controller into the commands of the different actuators. In this article, a survey of control allocation methods for over-actuated underwater vehicles is presented. The methods are applicable for both surface vessels and underwater vehicles. The paper presents a survey of control allocation methods with focus on mathematical representation and solvability of thruster allocation problems. The paper is useful for university students and engineers who want to get an overview of state-of-the art control allocation methods as well as advance methods to solve more complex problems.

An efficient and effective localization method for networked disjoint top-view cameras

Disjoint top-view networked cameras are among the most commonly utilized networks in many applications. One of the open questions for these cameras' study is the computation of extrinsic parameters (positions and orientations), named extrinsic calibration or localization of cameras. Current approaches either rely on strict assumptions of the object motion for accurate results or fail to provide results of high accuracy without the requirement of the object motion. To address these shortcomings, we present a location-constrained maximum a posteriori (LMAP) approach by applying known locations in the surveillance area, some of which would be passed by the object opportunistically. The LMAP approach formulates the problem as a joint inference of the extrinsic parameters and object trajectory based on the cameras' observations and the known locations. In addition, a new task-oriented evaluation metric, named MABR (the Maximum value of All image points' Back-projected localization errors' L2 norms Relative to the area of field of view), is presented to assess the quality of the calibration results in an indoor object tracking context. Finally, results herein demonstrate the superior performance of the proposed method over the state-of-the-art algorithm based on the presented MABR and classical evaluation metric in simulations and real experiments.

Talk Underwater

Short story

Port-Hamiltonian control of fully actuated underwater vehicles

This chapter presents a novel control strategy for trajectory tracking of underwater marine vehicles that are designed using port-Hamiltonian theory. A model for neutrally buoyant underwater vehicles is formulated as a PHS, and then the tracking controller is designed for the horizontal plane-surge, sway and yaw. The control design is done by formulating the error dynamics as a set-point regulation port-Hamiltonian control problem. The control design is formulated in two steps. In the first step, a static-feedback tracking controller is designed, and the second step integral action is added. The global asymptotic stability of the closed loop system is proved and the performance of the controller is illustrated using a model of an open-frame offshore underwater vehicle.

Control of an underactuated-slender-hull unmanned underwater vehicle using Port-Hamiltonian theory

This paper presents a control design for tracking of attitude and speed of an underactuated slender-hull unmanned underwater vehicle (UUV). The control design is based on Port-Hamiltonian theory. The target dynamics (desired dynamic response) is shaped with particular attention to the target mass matrix so that the influence of the unactuated dynamics on the controlled system is suppressed. This results in achievable dynamics independent of uncontrolled states. Throughout the design, insight of the physical phenomena involved is used to propose the desired target dynamics. The performance of the design is demonstrated through simulation with a high-fidelity model.

Control design for positioning of underwater vehicles with control allocation

In this paper, we address the control design problem of positioning of over-actuated underwater vehicles. The proposed design is based on a control architecture with combined position and velocity loops and a control tuning method based on the decoupled models. We derive analytical tuning rules based on requirements of closed-loop stability, positioning performance, and the vehicle velocity dynamic characteristics. The vehicle modelling is considered from force to motion with appropriate simplifications related to low-speed manoeuvring hydrodynamics and vehicle symmetry. The control design is considered together with a control allocation mapping. This approach makes the control tuning independent of the characteristics of the force actuators and provides the basis for control reconfiguration in the presence of actuator failure. We propose an anti-wind-up implementation of the controller, which ensures that the constraints related to actuation capacity are not violated. This approach simplifies the control allocation problem since the actuator constraints are mapped into generalised force constraints.

Soluble laminin and arginine-glycine-aspartic acid containing peptides differentially regulate type IV collagenase messenger RNA, activation, and localization in testicular cell culture

Rat testicular cells in culture produce several metalloproteinases including type IV collagenases (Sang et al. Biol Reprod 1990; 43:946-955, 956-964). We have now investigated the regulation of testicular cell type IV collagenase and other metalloprotemases in vitro. Soluble laminin stimulated Sertoli cell type IV collagenase mRNA levels. However, three peptides corresponding to different domains of the laminin molecule (CSRAKQAASIKVASADR, FALRGDNP, CLQDGDVRV) did not influence type IV collagenase mENA levels. Zyniographic analysis of medium collected from these cultures revealed that neither soluble laminin nor any of the peptides influenced 72-Wa type IV collagenase protein levels. However, peptide FALRGDNP resulted in both, a selective increase in two higher molecular-weight metalloprotemnases (83 kDa and 110 Wa and in an activation of the 72-Wa rat type IV collagenase. Interleukin-1, phorbol ester, testosterone, and FSH did not affect collagenase activation, lmmunocytochemical studies demonstrated that the addition of soluble laminin resulted in a redistribution of type IV collagenase from intracellular vesicles to the cell-substrate region beneath the cells. Peptide FALRGDNP induced a change from a vesicular to peripheral plasma membrane type of staining pattern. Zymography of plasma membrane preparations demonstrated triton-soluble gelatinases of 76 Wa, 83 Wa, and 110 Wa and a triton-insoluble gelatinase of 225 Wa, These results indicate that testicular cell type IV collagenase mRNA levels, enzyme activation, and distribution are influenced by laminin and RGD-containing peptides.

Localization of fibronectin in the rat testis : effects of serum and dibutyryl cyclic AMP on fibronectin deposition by peritubular myoid cells in culture

The peritubular zone of the rat testis has an extensive extracellular matrix (ECM). Fibronectin (FN) is distributed primarily in the basal lamina of the seminiferous tubule boundary tissue and is synthesized by peritubular myoid cells. Several extracellular changes are mediated by growth factors and these changes occur at the time of hormone mediated testicular development, particularly in the peritubular zone. The effects of serum or dibutyryl cyclic AMP (cAMP) on FN production by the mesenchymal peritubular myoid cells were evaluated. Rats of various ages (10, 15, 20, 40 and 80 days) were employed for immunofluorescent localization of rat testicular FN in frozen sections. In all age groups tested, FN was primarily present in a broad layer around each seminiferous tubule, and blood vessel, and in variable distribution throughout the interstitial stroma. By day 20 there was no clear distinction in FN staining between the peritubular zone and the interstitial tissue. This indicates an involvement of FN in the ECM developments which occur in the peritubular zone of the testis at this time. The peritubular myoid cells were isolated from 20-22 day old rat testis and cultured on glass coverslips. These cells were grown to confluence with 10% fetal calf serum (FCS) in medium until day 4 and then subcultured to have secondary monocultures maintained with or without serum. By means of immunofluorescence and cytochemistry using avidin-biotin peroxidase complex it was observed that peritubular myoid cells were positive for FN and most of the FN was localized in the perinuclear region. Subcultured peritubular myoid cells maintained for 4 days in medium containing FCS developed an extensive interconnecting FN matrix. In the presence of 0.5 mM cAMP in culture, FN became localized along the filamentous process of peritubular myoid cells and more prominently in the areas of triangulated multi-cell aggregates as well as on the surface of the contracted small spherical cells. The addition of cAMP in the presence of FCS, also caused a noticeable change in the staining pattern; FN was detected along the filamentous process developing into a complex network of cells encased in an extensive matrix. It would appear that the translocation of FN in the cytoplasmic extensions of peritubular myoid cells may be a direct consequence of morphological changes associated with metabolic regulation of cAMP. This may also be related to the puberty associated development of in vivo changes in the ECM produced by peritubular myoid cells.

An adaptive appearance-based map for long-term topological localization of mobile robots

"This work considers a mobile service robot which uses an appearance-based representation of its workplace as a map, where the current view and the map are used to estimate the current position in the environment. Due to the nature of real-world environments such as houses and offices, where the appearance keeps changing, the internal representation may become out of date after some time. To solve this problem the robot needs to be able to adapt its internal representation continually to the changes in the environment. This paper presents a method for creating an adaptive map for long-term appearance-based localization of a mobile robot using long-term and short-term memory concepts, with omni-directional vision as the external sensor."--publisher website