Path planning for image based visual servoing

This thesis deals with Visual Servoing and its strictly connected disciplines like projective geometry, image processing, robotics and non-linear control. More specifically the work addresses the problem to control a robotic manipulator through one of the largely used Visual Servoing techniques: the Image Based Visual Servoing (IBVS). In Image Based Visual Servoing the robot is driven by on-line performing a feedback control loop that is closed directly in the 2D space of the camera sensor. The work considers the case of a monocular system with the only camera mounted on the robot end effector (eye in hand configuration). Through IBVS the system can be positioned with respect to a 3D fixed target by minimizing the differences between its initial view and its goal view, corresponding respectively to the initial and the goal system configurations: the robot Cartesian Motion is thus generated only by means of visual informations. However, the execution of a positioning control task by IBVS is not straightforward because singularity problems may occur and local minima may be reached where the reached image is very close to the target one but the 3D positioning task is far from being fulfilled: this happens in particular for large camera displacements, when the the initial and the goal target views are noticeably different. To overcame singularity and local minima drawbacks, maintaining the good properties of IBVS robustness with respect to modeling and camera calibration errors, an opportune image path planning can be exploited. This work deals with the problem of generating opportune image plane trajectories for tracked points of the servoing control scheme (a trajectory is made of a path plus a time law). The generated image plane paths must be feasible i.e. they must be compliant with rigid body motion of the camera with respect to the object so as to avoid image jacobian singularities and local minima problems. In addition, the image planned trajectories must generate camera velocity screws which are smooth and within the allowed bounds of the robot. We will show that a scaled 3D motion planning algorithm can be devised in order to generate feasible image plane trajectories. Since the paths in the image are off-line generated it is also possible to tune the planning parameters so as to maintain the target inside the camera field of view even if, in some unfortunate cases, the feature target points would leave the camera images due to 3D robot motions. To test the validity of the proposed approach some both experiments and simulations results have been reported taking also into account the influence of noise in the path planning strategy. The experiments have been realized with a 6DOF anthropomorphic manipulator with a fire-wire camera installed on its end effector: the results demonstrate the good performances and the feasibility of the proposed approach.

Seizure prediction and control in epilepsy

The first part of my thesis presents an overview of the different approaches used in the past two decades in the attempt to forecast epileptic seizure on the basis of intracranial and scalp EEG. Past research could reveal some value of linear and nonlinear algorithms to detect EEG features changing over different phases of the epileptic cycle. However, their exact value for seizure prediction, in terms of sensitivity and specificity, is still discussed and has to be evaluated. In particular, the monitored EEG features may fluctuate with the vigilance state and lead to false alarms. Recently, such a dependency on vigilance states has been reported for some seizure prediction methods, suggesting a reduced reliability. An additional factor limiting application and validation of most seizure-prediction techniques is their computational load. For the first time, the reliability of permutation entropy [PE] was verified in seizure prediction on scalp EEG data, contemporarily controlling for its dependency on different vigilance states. PE was recently introduced as an extremely fast and robust complexity measure for chaotic time series and thus suitable for online application even in portable systems. The capability of PE to distinguish between preictal and interictal state has been demonstrated using Receiver Operating Characteristics (ROC) analysis. Correlation analysis was used to assess dependency of PE on vigilance states. Scalp EEG-Data from two right temporal epileptic lobe (RTLE) patients and from one patient with right frontal lobe epilepsy were analysed. The last patient was included only in the correlation analysis, since no datasets including seizures have been available for him. The ROC analysis showed a good separability of interictal and preictal phases for both RTLE patients, suggesting that PE could be sensitive to EEG modifications, not visible on visual inspection, that might occur well in advance respect to the EEG and clinical onset of seizures. However, the simultaneous assessment of the changes in vigilance showed that: a) all seizures occurred in association with the transition of vigilance states; b) PE was sensitive in detecting different vigilance states, independently of seizure occurrences. Due to the limitations of the datasets, these results cannot rule out the capability of PE to detect preictal states. However, the good separability between pre- and interictal phases might depend exclusively on the coincidence of epileptic seizure onset with a transition from a state of low vigilance to a state of increased vigilance. The finding of a dependency of PE on vigilance state is an original finding, not reported in literature, and suggesting the possibility to classify vigilance states by means of PE in an authomatic and objectic way. The second part of my thesis provides the description of a novel behavioral task based on motor imagery skills, firstly introduced (Bruzzo et al. 2007), in order to study mental simulation of biological and non-biological movement in paranoid schizophrenics (PS). Immediately after the presentation of a real movement, participants had to imagine or re-enact the very same movement. By key release and key press respectively, participants had to indicate when they started and ended the mental simulation or the re-enactment, making it feasible to measure the duration of the simulated or re-enacted movements. The proportional error between duration of the re-enacted/simulated movement and the template movement were compared between different conditions, as well as between PS and healthy subjects. Results revealed a double dissociation between the mechanisms of mental simulation involved in biological and non-biologial movement simulation. While for PS were found large errors for simulation of biological movements, while being more acurate than healthy subjects during simulation of non-biological movements. Healthy subjects showed the opposite relationship, making errors during simulation of non-biological movements, but being most accurate during simulation of non-biological movements. However, the good timing precision during re-enactment of the movements in all conditions and in both groups of participants suggests that perception, memory and attention, as well as motor control processes were not affected. Based upon a long history of literature reporting the existence of psychotic episodes in epileptic patients, a longitudinal study, using a slightly modified behavioral paradigm, was carried out with two RTLE patients, one patient with idiopathic generalized epilepsy and one patient with extratemporal lobe epilepsy. Results provide strong evidence for a possibility to predict upcoming seizures in RTLE patients behaviorally. In the last part of the thesis it has been validated a behavioural strategy based on neurobiofeedback training, to voluntarily control seizures and to reduce there frequency. Three epileptic patients were included in this study. The biofeedback was based on monitoring of slow cortical potentials (SCPs) extracted online from scalp EEG. Patients were trained to produce positive shifts of SCPs. After a training phase patients were monitored for 6 months in order to validate the ability of the learned strategy to reduce seizure frequency. Two of the three refractory epileptic patients recruited for this study showed improvements in self-management and reduction of ictal episodes, even six months after the last training session.

Biofilms on exposed monumental stones: mechanism of formation and development of new control methods

Within the stone monumental artefacts artistic fountains are extremely favorable to formation of biofilms, giving rise to biodegradation processes related with physical-chemical and visual aspect alterations, because of their particular exposure conditions. Microbial diversity of five fountains (two from Spain and three from Italy) was investigated. It was observed an ample similarity between the biodiversity of monumental stones reported in literature and that one found in studied fountains. Mechanical procedures and toxic chemical products are usually employed to remove such phototrophic patinas. Alternative methods based on natural antifouling substances are recently experimented in the marine sector, due to their very low environmental impact and for the bio settlement prevention on partially immersed structures of ships. In the present work groups of antibiofouling agents (ABAs) were selected from literature for their ability to interfere, at molecular level, with the microbial communication system “quorum sensing”, inhibiting the initial phase of biofilm formation. The efficacy of some natural antibiofoulants agents (ABAs) with terrestrial (Capsaicine - CS, Cinnamaldehyde - CI) and marine origin (Zosteric Acid - ZA, poly-Alkyl Pyridinium Salts – pAPS and Ceramium botryocarpum extract - CBE), incorporated into two commercial coatings (Silres BS OH 100 - S and Wacker Silres BS 290 - W) commonly used in stone conservation procedures were evaluated. The formation of phototrophic biofilms in laboratory conditions (on Carrara marble specimens and Sierra Elvira stone) and on two monumental fountains (Tacca’s Fountain 2 - Florence, Italy and Fountain from Patio de la Lindaraja - Alhambra Palace, Granada, Spain) has been investigated in the presence or absence of these natural antifouling agents. The natural antibiofouling agents, at tested concentrations, demonstrated a certain inhibitory effect. The silane-siloxane based silicone coating (W) mixing with ABAs was more suitable with respect to ethyl silicate coating (S) and proved efficacy against biofilm formation only when incompletely cured. The laboratory results indicated a positive action in inhibiting the patina formation, especially for poly-alkyl pyridinium salts, zosteric acid and cinnamaldehyde, while on site tests revealed a good effect for zosteric acid.

Neural basis of visual deficits recovery: visual residual functions and multisensory integration.

The ability of integrating into a unified percept sensory inputs deriving from different sensory modalities, but related to the same external event, is called multisensory integration and might represent an efficient mechanism of sensory compensation when a sensory modality is damaged by a cortical lesion. This hypothesis has been discussed in the present dissertation. Experiment 1 explored the role of superior colliculus (SC) in multisensory integration, testing patients with collicular lesions, patients with subcortical lesions not involving the SC and healthy control subjects in a multisensory task. The results revealed that patients with collicular lesions, paralleling the evidence of animal studies, demonstrated a loss of multisensory enhancement, in contrast with control subjects, providing the first lesional evidence in humans of the essential role of SC in mediating audio-visual integration. Experiment 2 investigated the role of cortex in mediating multisensory integrative effects, inducing virtual lesions by inhibitory theta-burst stimulation on temporo-parietal cortex, occipital cortex and posterior parietal cortex, demonstrating that only temporo-parietal cortex was causally involved in modulating the integration of audio-visual stimuli at the same spatial location. Given the involvement of the retino-colliculo-extrastriate pathway in mediating audio-visual integration, the functional sparing of this circuit in hemianopic patients is extremely relevant in the perspective of a multisensory-based approach to the recovery of unisensory defects. Experiment 3 demonstrated the spared functional activity of this circuit in a group of hemianopic patients, revealing the presence of implicit recognition of the fearful content of unseen visual stimuli (i.e. affective blindsight), an ability mediated by the retino-colliculo-extrastriate pathway and its connections with amygdala. Finally, Experiment 4 provided evidence that a systematic audio-visual stimulation is effective in inducing long-lasting clinical improvements in patients with visual field defect and revealed that the activity of the spared retino-colliculo-extrastriate pathway is responsible of the observed clinical amelioration, as suggested by the greater improvement observed in patients with cortical lesions limited to the occipital cortex, compared to patients with lesions extending to other cortical areas, found in tasks high demanding in terms of spatial orienting. Overall, the present results indicated that multisensory integration is mediated by the retino-colliculo-extrastriate pathway and that a systematic audio-visual stimulation, activating this spared neural circuit, is able to affect orientation towards the blind field in hemianopic patients and, therefore, might constitute an effective and innovative approach for the rehabilitation of unisensory visual impairments.

Influence of optic flow on postural control

The study of optic flow on postural control may explain how self-motion perception contributes to postural stability in young males and females and how such function changes in the old falls risk population. Study I: The aim was to examine the optic flow effect on postural control in young people (n=24), using stabilometry and surface-electromyography. Subjects viewed expansion and contraction optic flow stimuli which were presented full field, in the foveral or in the peripheral visual field. Results showed that optic flow stimulation causes an asymmetry in postural balance and a different lateralization of postural control in men and women. Gender differences evoked by optic flow were found both in the muscle activity and in the prevalent direction of oscillation. The COP spatial variability was reduced during the view of peripheral stimuli which evoked a clustered prevalent direction of oscillation, while foveal and random stimuli induced non-distributed directions. Study II was aimed at investigating the age-related mechanisms of postural stability during the view of optic flow stimuli in young (n=17) and old (n=19) people, using stabilometry and kinematic. Results showed that old people showed a greater effort to maintain posture during the view of optic flow stimuli than the young. Elderly seems to use the head stabilization on trunk strategy. Visual stimuli evoke an excitatory input on postural muscles, but the stimulus structure produces different postural effects. Peripheral optic flow stabilizes postural sway, while random and foveal stimuli provoke larger sway variability similar to those evoked in baseline. Postural control uses different mechanisms within each leg to produce the appropriate postural response to interact with extrapersonal environment. Ageing reduce the effortlessness to stabilize posture during optic flow, suggesting a neuronal processing decline associated with difficulty integrating multi-sensory information of self-motion perception and increasing risk of falls.

Impaired performance on the rapid visual information processing task (RVIP) could be an endophenotype of schizophrenia

The aim of the present study was to investigate whether healthy first-degree relatives of schizophrenia patients show reduced sensitivity performance, higher intra-individual variability (IIV) in reaction time (RT), and a steeper decline in sensitivity over time in a sustained attention task. Healthy first-degree relatives of schizophrenia patients (n=23) and healthy control subjects (n=46) without a family history of schizophrenia performed a demanding version of the Rapid Visual Information Processing task (RVIP). RTs, hits, false alarms, and the sensitivity index A' were assessed. The relatives were significantly less sensitive, tended to have higher IIV in RT, but sustained the impaired level of sensitivity over time. Impaired performance on the RVIP is a possible endophenotype for schizophrenia. Higher IIV in RT, apparently caused by impaired context representations, might result in fluctuations in control and lead to more frequent attentional lapses.

Development of a novel fMRI compatible visual perception prototype battery to test older people with and without dementia

Background: Visuoperceptual deficits in dementia are common and can reduce quality of life. Testing of visuoperceptual function is often confounded by impairments in other cognitive domains and motor dysfunction. We aimed to develop, pilot, and test a novel visuocognitive prototype test battery which addressed these issues, suitable for both clinical and functional imaging use. Methods: We recruited 23 participants (14 with dementia, 6 of whom had extrapyramidal motor features, and 9 age-matched controls). The novel Newcastle visual perception prototype battery (NEVIP-B-Prototype) included angle, color, face, motion and form perception tasks, and an adapted response system. It allows for individualized task difficulties. Participants were tested outside and inside the 3T functional magnetic resonance imaging (fMRI) scanner. Functional magnetic resonance imaging data were analyzed using SPM8. Results: All participants successfully completed the task inside and outside the scanner. Functional magnetic resonance imaging analysis showed activation regions corresponding well to the regional specializations of the visual association cortex. In both groups, there was significant activity in the ventral occipital-temporal region in the face and color tasks, whereas the motion task activated the V5 region. In the control group, the angle task activated the occipitoparietal cortex. Patients and controls showed similar levels of activation, except on the angle task for which occipitoparietal activation was lower in patients than controls. Conclusion: Distinct visuoperceptual functions can be tested in patients with dementia and extrapyramidal motor features when tests use individualized thresholds, adapted tasks, and specialized response systems.

Visual symptoms in Parkinson's disease and Parkinson's disease dementia

Visual symptoms are common in PD and PD dementia and include difficulty reading, double vision, illusions, feelings of presence and passage, and complex visual hallucinations. Despite the established prognostic implications of complex visual hallucinations, the interaction between cognitive decline, visual impairment, and other visual symptoms remains poorly understood. Our aim was to characterize the spectrum of visual symptomatology in PD and examine clinical predictors for their occurrence. Sixty-four subjects with PD, 26 with PD dementia, and 32 age-matched controls were assessed for visual symptoms, cognitive impairment, and ocular pathology. Complex visual hallucinations were common in PD (17%) and PD dementia (89%). Dementia subjects reported illusions (65%) and presence (62%) more frequently than PD or control subjects, but the frequency of passage hallucinations in PD and PD dementia groups was equivalent (48% versus 69%, respectively; P = 0.102). Visual acuity and contrast sensitivity was impaired in parkinsonian subjects, with disease severity and age emerging as the key predictors. Regression analysis identified a variety of factors independently predictive of complex visual hallucinations (e.g., dementia, visual acuity, and depression), illusions (e.g., excessive daytime somnolence and disease severity), and presence (e.g., rapid eye movement sleep behavior disorder and excessive daytime somnolence). Our results demonstrate that different "hallucinatory" experiences in PD do not necessarily share common disease predictors and may, therefore, be driven by different pathophysiological mechanisms. If confirmed, such a finding will have important implications for future studies of visual symptoms and cognitive decline in PD and PD dementia.

Evidence for a cognitive control network for goal-directed attention in simple sustained attention

The deterioration of performance over time is characteristic for sustained attention tasks. This so-called "performance decrement" is measured by the increase of reaction time (RT) over time. Some behavioural and neurobiological mechanisms of this phenomenon are not yet fully understood. Behaviourally, we examined the increase of RT over time and the inter-individual differences of this performance decrement. On the neurophysiological level, we investigated the task-relevant brain areas where neural activity was modulated by RT and searched for brain areas involved in good performance (i.e. participants with no or moderate performance decrement) as compared to poor performance (i.e. participants with a steep performance decrement). For this purpose, 20 healthy, young subjects performed a carefully designed task for simple sustained attention, namely a low-demanding version of the Rapid Visual Information Processing task. We employed a rapid event-related functional magnetic resonance imaging (fMRI) design. The behavioural results showed a significant increase of RT over time in the whole group, and also revealed that some participants were not as prone to the performance decrement as others. The latter was statistically significant comparing good versus poor performers. Moreover, high BOLD-responses were linked to longer RTs in a task-relevant bilateral fronto-cingulate-insular-parietal network. Among these regions, good performance was associated with significantly higher RT-BOLD correlations in the pre-supplementary motor area (pre-SMA). We concluded that the task-relevant bilateral fronto-cingulate-insular-parietal network was a cognitive control network responsible for goal-directed attention. The pre-SMA in particular might be associated with the performance decrement insofar that good performers could sustain activity in this brain region in order to monitor performance declines and adjust behavioural output.

The neurophysiological time pattern of illusionary visual perceptual transitions: a simultaneous EEG and fMRI study

Several divergent cortical mechanisms generating multistability in visual perception have been suggested. Here, we investigated the neurophysiologic time pattern of multistable perceptual changes by means of a simultaneous recording with electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). Volunteers responded to the subjective perception of a sudden change between stable patterns of illusionary motion (multistable transition) during a stroboscopic paradigm. We found a global deceleration of the EEG frequency prior to a transition and an occipital-accentuated acceleration after a transition, as obtained by low-resolution electromagnetic tomography analysis (LORETA) analysis. A decrease in BOLD response was found in the prefrontal cortex before, and an increase after the transitions was observed in the right anterior insula, the MT/V5 regions and the SMA. The thalamus and left superior temporal gyrus showed a pattern of decrease before and increase after transitions. No such temporal course was found in the control condition. The multimodal approach of data acquisition allows us to argue that the top-down control of illusionary visual perception depends on selective attention, and that a diminution of vigilance reduces selective attention. These are necessary conditions to allow for the occurrence of a perception discontinuity in absence of a physical change of the stimulus.

An fMRI-study of locally oriented perception in autism: altered early visual processing of the block design test

Autism has been associated with enhanced local processing on visual tasks. Originally, this was based on findings that individuals with autism exhibited peak performance on the block design test (BDT) from the Wechsler Intelligence Scales. In autism, the neurofunctional correlates of local bias on this test have not yet been established, although there is evidence of alterations in the early visual cortex. Functional MRI was used to analyze hemodynamic responses in the striate and extrastriate visual cortex during BDT performance and a color counting control task in subjects with autism compared to healthy controls. In autism, BDT processing was accompanied by low blood oxygenation level-dependent signal changes in the right ventral quadrant of V2. Findings indicate that, in autism, locally oriented processing of the BDT is associated with altered responses of angle and grating-selective neurons, that contribute to shape representation, figure-ground, and gestalt organization. The findings favor a low-level explanation of BDT performance in autism.

Functional imbalance of visual pathways indicates alternative face processing strategies in autism

OBJECTIVE: To investigate whether autistic subjects show a different pattern of neural activity than healthy individuals during processing of faces and complex patterns. METHODS: Blood oxygen level-dependent (BOLD) signal changes accompanying visual processing of faces and complex patterns were analyzed in an autistic group (n = 7; 25.3 [6.9] years) and a control group (n = 7; 27.7 [7.8] years). RESULTS: Compared with unaffected subjects, autistic subjects demonstrated lower BOLD signals in the fusiform gyrus, most prominently during face processing, and higher signals in the more object-related medial occipital gyrus. Further signal increases in autistic subjects vs controls were found in regions highly important for visual search: the superior parietal lobule and the medial frontal gyrus, where the frontal eye fields are located. CONCLUSIONS: The cortical activation pattern during face processing indicates deficits in the face-specific regions, with higher activations in regions involved in visual search. These findings reflect different strategies for visual processing, supporting models that propose a predisposition to local rather than global modes of information processing in autism.

Inhibitory control of the human dorsolateral prefrontal cortex during the anti-saccade paradigm--a transcranial magnetic stimulation study

In the anti-saccade paradigm, subjects are instructed not to make a reflexive saccade to an appearing lateral target but to make an intentional saccade to the opposite side instead. The inhibition of reflexive saccade triggering is under the control of the dorsolateral prefrontal cortex (DLPFC). The critical time interval at which this inhibition takes place during the paradigm, however, is not exactly known. In the present study, we used single-pulse transcranial magnetic stimulation (TMS) to interfere with DLPFC function in 15 healthy subjects. TMS was applied over the right DLPFC either 100 ms before the onset of the visual target (i.e. -100 ms), at target onset (i.e. 0 ms) or 100 ms after target onset (i.e. +100 ms). Stimulation 100 ms before target onset significantly increased the percentage of anti-saccade errors to both sides, while stimulation at, or after, target onset had no significant effect. All three stimulation conditions had no significant influence on saccade latency of correct or erroneous anti-saccades. These findings show that the critical time interval at which the DLPFC controls the suppression of a reflexive saccade in the anti-saccade paradigm is before target onset. In addition, the results suggest the view that the triggering of correct anti-saccades is not under direct control of the DLPFC.

A semi-structured interview to assess visual hallucinations in older people

OBJECTIVE: Visual hallucinations are under-reported by patients and are often undiscovered by health professionals. There is no gold standard available to assess hallucinations. Our objective was to develop a reliable, valid, semi-structured interview for identifying and assessing visual hallucinations in older people with eye disease and cognitive impairment. METHODS: We piloted the North-East Visual Hallucinations Interview (NEVHI) in 80 older people with visual and/or cognitive impairment (patient group) and 34 older people without known risks of hallucinations (control group). The informants of 11 patients were interviewed separately. We established face validity, content validity, criterion validity, inter-rater agreement and the internal consistency of the NEVHI, and assessed the factor structure for questions evaluating emotions, cognitions, and behaviours associated with hallucinations. RESULTS: Recurrent visual hallucinations were common in the patient group (68.8%) and absent in controls (0%). The criterion, face and content validities were good and the internal consistency of screening questions for hallucinations was high (Cronbach alpha: 0.71). The inter-rater agreements for simple and complex hallucinations were good (Kappa 0.72 and 0.83, respectively). Four factors associated with experiencing hallucinations (perceived control, pleasantness, distress and awareness) were identified and explained a total variance of 73%. Informants gave more 'don't know answers' than patients throughout the interview (p = 0.008), especially to questions evaluating cognitions and emotions associated with hallucinations (p = 0.02). CONCLUSIONS: NEVHI is a comprehensive assessment tool, helpful to identify the presence of visual hallucinations and to quantify cognitions, emotions and behaviours associated with hallucinations.

A new method to monitor visual field defects caused by photoreceptor degeneration by quantitative optical coherence tomography

PURPOSE: To correlate the dimension of the visual field (VF) tested by Goldman kinetic perimetry with the extent of visibility of the highly reflective layer between inner and outer segments of photoreceptors (IOS) seen in optical coherence tomography (OCT) images in patients with retinitis pigmentosa (RP). METHODS: In a retrospectively designed cross-sectional study, 18 eyes of 18 patients with RP were examined with OCT and Goldmann perimetry using test target I4e and compared with 18 eyes of 18 control subjects. A-scans of raw scan data of Stratus OCT images (Carl Zeiss Meditec, AG, Oberkochen, Germany) were quantitatively analyzed for the presence of the signal generated by the highly reflective layer between the IOS in OCT images. Starting in the fovea, the distance to which this signal was detectable was measured. Visual fields were analyzed by measuring the distance from the center point to isopter I4e. OCT and visual field data were analyzed in a clockwise fashion every 30 degrees , and corresponding measures were correlated. RESULTS: In corresponding alignments, the distance from the center point to isopter I4e and the distance to which the highly reflective signal from the IOS can be detected correlate significantly (r = 0.75, P < 0.0001). The greater the distance in VF, the greater the distance measured in OCT. CONCLUSIONS: The authors hypothesize that the retinal structure from which the highly reflective layer between the IOS emanates is of critical importance for visual and photoreceptor function. Further research is warranted to determine whether this may be useful as an objective marker of progression of retinal degeneration in patients with RP.