Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

Comparison of Humanized IgG and FvFc Anti-CD3 Monoclonal Antibodies Expressed in CHO Cells

Two humanized monoclonal antibody constructs bearing the same variable regions of an anti-CD3 monoclonal antibody, whole IgG and FvFc, were expressed in CHO cells. Random and site-specific integration were used resulting in similar expression levels. The transfectants were selected with appropriate selection agent, and the surviving cells were plated in semi-solid medium for capture with FITC-conjugated anti-human IG antibody and picked with the robotic ClonePix FL. Conditioned media from selected clones were purified by affinity chromatography and characterized by SDS-PAGE, Western-blot, SEC-HPLC, and isoelectric focusing. Binding to the target present in healthy human mononuclear cells was assessed by flow cytometry, as well as by competition between the two constructs and the original murine monoclonal antibody. The humanized constructs were not able to dislodge the murine antibody while the murine anti-CD3 antibody could dislodge around 20% of the FvFc or IgG humanized versions. Further in vitro and in vivo pre-clinical analyses will be carried out to verify the ability of the humanized versions to demonstrate the immunoregulatory profile required for a humanized anti-CD3 monoclonal antibody.

Predictive Factors for Positive Surgical Margins and Their Locations After Robot-Assisted Laparoscopic Radical Prostatectomy

Background: Positive surgical margin (PSM) after radical prostatectomy (RP) has been shown to be an independent predictive factor for cancer recurrence. Several investigations have correlated clinical and histopathologic findings with surgical margin status after open RP. However, few studies have addressed the predictive factors for PSM after robot-assisted laparoscopic RP (RARP). Objective: We sought to identify predictive factors for PSMs and their locations after RARP. Design, setting, and participants: We prospectively analyzed 876 consecutive patients who underwent RARP from January 2008 to May 2009. Intervention: All patients underwent RARP performed by a single surgeon with previous experience of > 1500 cases. Measurements: Stepwise logistic regression was used to identify potential predictive factors for PSM. Three logistic regression models were built: (1) one using preoperative variables only, (2) another using all variables (preoperative, intraoperative, and postoperative) combined, and (3) one created to identify potential predictive factors for PSM location. Preoperative variables entered into the models included age, body mass index (BMI), prostate-specific antigen, clinical stage, number of positive cores, percentage of positive cores, and American Urological Association symptom score. Intra-and postoperative variables analyzed were type of nerve sparing, presence of median lobe, percentage of tumor in the surgical specimen, gland size, histopathologic findings, pathologic stage, and pathologic Gleason grade. Results and limitations: In the multivariable analysis including preoperative variables, clinical stage was the only independent predictive factor for PSM, with a higher PSM rate for T3 versus T1c (odds ratio [OR]: 10.7; 95% confidence interval [CI], 2.6-43.8) and for T2 versus T1c (OR: 2.9; 95% CI, 1.9-4.6). Considering pre-, intra-, and postoperative variables combined, percentage of tumor, pathologic stage, and pathologic Gleason score were associated with increased risk of PSM in the univariable analysis (p < 0.001 for all variables). However, in the multivariable analysis, pathologic stage (pT2 vs pT1; OR: 2.9; 95% CI, 1.9-4.6) and percentage of tumor in the surgical specimen (OR: 8.7; 95% CI, 2.2-34.5; p = 0.0022) were the only independent predictive factors for PSM. Finally, BMI was shown to be an independent predictive factor(OR: 1.1; 95% CI, 1.0-1.3; p = 0.0119) for apical PSMs, with increasing BMI predicting higher incidence of apex location. Because most of our patients were referred from other centers, the biopsy technique and the number of cores were not standardized in our series. Conclusions: Clinical stage was the only preoperative variable independently associated with PSM after RARP. Pathologic stage and percentage of tumor in the surgical specimen were identified as independent predictive factors for PSMs when analyzing pre-, intra-, and postoperative variables combined. BMI was shown to be an independent predictive factor for apical PSMs. (C) 2010 European Association of Urology. Published by Elsevier B. V. All rights reserved.

Gene Expression Profiles Stratified according to Type 1 Diabetes Mellitus Susceptibility Regions

The MHC region (6p21) aggregates the major genes that contribute to susceptibility to type 1 diabetes (T1D). Three additional relevant susceptibility regions mapped on chromosomes 1p13 (PTPN22), 2q33 (CTLA-4), and 11p15 (insulin) have also been described by linkage studies. To evaluate the contribution of these susceptibility regions and the chromosomes that house these regions, we performed a large-scale differential gene expression on lymphomononuclear cells of recently diagnosed T1D patients, pinpointing relevant modulated genes clustered in these regions and their respective chromosomes. A total of 4608 cDNAs from the IMAGE library were spotted onto glass slides using robotic technology. Statistical analysis was carried out using the SAM program, and data regarding gene location and biological function were obtained at the SOURCE, NCBI, and FATIGO programs. Three induced genes were observed spanning around the MHC region (6p21-6p23), and seven modulated genes (5 repressed and 2 repressed) were seen spanning around the 6q21-24 region. Additional modulated genes were observed in and around the 1p13, 2q33, and 11p15 regions. Overall, modulated genes in these regions were primarily associated with cellular metabolism, transcription factors and signaling transduction. The differential gene expression characterization may identify new genes potentially involved with diabetes pathogenesis.

Regional representativity of AERONET observation sites during the biomass burning season in South America determined by correlation studies with MODIS Aerosol Optical Depth

This paper presents an analysis of ground-based Aerosol Optical Depth (AOD) observations by the Aerosol Robotic Network (AERONET) in South America from 2001 to 2007 in comparison with the satellite AOD product of Moderate Resolution Imaging Spectroradiometer (MODIS), aboard TERRA and AQUA satellites. Data of 12 observation sites were used with primary interest in AERONET sites located in or downwind of areas with high biomass burning activity and with measurements available for the full time range. Fires cause the predominant carbonaceous aerosol emission signal during the dry season in South America and are therefore a special focus of this study. Interannual and seasonal behavior of the observed AOD at different sites were investigated, showing clear differences between purely fire and urban influenced sites. An intercomparison of AERONET and MODIS AOD annual correlations revealed that neither an interannual long-term trend may be observed nor that correlations differ significantly owing to different overpass times of TERRA and AQUA. Individual anisotropic representativity areas for each AERONET site were derived by correlating daily AOD of each site for all years with available individual MODIS AOD pixels gridded to 1 degrees x 1 degrees. Results showed that for many sites a good AOD correlation (R(2) > 0.5) persists for large, often strongly anisotropic, areas. The climatological areas of common regional aerosol regimes often extend over several hundreds of kilometers, sometimes far across national boundaries. As a practical application, these strongly inhomogeneous and anisotropic areas of influence are being implemented in the tropospheric aerosol data assimilation system of the Coupled Chemistry-Aerosol-Tracer Transport Model coupled to the Brazilian Regional Atmospheric Modeling System (CCATT-BRAMS) at the Brazilian National Institute for Space Research (INPE). This new information promises an improved exploitation of local site sampling and, thus, chemical weather forecast.

On the development of summer thunderstorms in the city of Sao Paulo: Mean meteorological characteristics and pollution effect

This study investigates how the summer thunderstorms developed over the city of Sao Paulo and if the pollution might affect its development or characteristics during the austral summer (December-January-February-March, DJFM months). A total of 605 days from December 1999 to March 2004 was separated as 241 thunderstorms days (TDs) and 364 non-thunderstorm days (NTDs). The analyses are performed by using hourly measurements of air temperature (T), web-bulb temperature (Tw), surface atmospheric pressure (P), wind velocity and direction, rainfall and thunder and lightning observations collected at the Meteorological Station of the University of Sao Paulo in conjunction with aerosol measurements obtained by AERONET (Aerosol Robotic Network), and the NCEP-DOE (National Centers for Environmental Prediction Department of Energy) reanalysis and radiosondes. The wind diurnal cycle shows that for TDs the morning flow is from the northwest rotating to the southeast after 16: 00 local time (LT) and it remains from the east until the night. For the NTDs, the wind is well characterized by the sea-breeze circulation that in the morning has the wind blowing from the northeast and in the afternoon from the southeast. The TDs show that the air temperature diurnal cycle presents higher amplitude and the maximum temperature of the day is 3.2 degrees C higher than in NTDs. Another important factor found is the difference between moisture that is higher during TDs. In terms of precipitation, the TDs represent 40% of total of days analyzed and those days are responsible for more than 60% of the total rain accumulation during the summer, for instance 50% of the TDs had more than 15.5mm day(-1) while the NTDs had 4 mm day(-1). Moreover, the rainfall distribution shows that TDs have higher rainfall rate intensities and an afternoon precipitation maximum; while in the NTDs there isn`t a defined precipitation diurnal cycle. The wind and temperature fields from NCEP reanalysis concur with the local weather station and radiosonde observations. The NCEP composites show that TDs are controlled by synoptic circulation characterized by a pre-frontal situation, with a baroclinic zone situated at southern part of Sao Paulo. In terms of pollution, this study employed the AERONET data to obtain the main aerosol characteristics in the atmospheric column for both TDs and NTDs. The particle size distribution and particle volume size distribution have similar concentrations for both TDs and NTDs and present a similar fine and coarse mode mean radius. In respect to the atmospheric loading, the aerosol optical depth (AOD) at different frequencies presented closed mean values for both TDs and NTDs that were statistically significant at 95% level. The spectral dependency of those values in conjunction with the Angstrom parameter reveal the higher concentration of the fine mode particles that are more likely to be hygroscopic and from urban areas. In summary, no significant aerosol effect could be found on the development of summer thunderstorms, suggesting the strong synoptic control by the baroclinic forcing for deep convective development. (C) 2010 Published by Elsevier B. V.

Scale architecture of the palmar and plantar epidermis of Polychrus acutirostris Spix, 1825 (Iguania, Polychrotidae) and its relationship to arboreal locomotion

In tetrapod squamates, the diversity of micro-ornamentations of the epidermis of the contact areas of hands and feet is generally associated with constraints and modalities related to locomotion. Polychrus acutirostris is a medium-sized lizard that occurs in open heterogeneous habitats in South America, such as the cerrados, caatingas, and fallow lands. It progresses slowly on branches of various diameters in its arboreal environment. It can also move more rapidly on the ground. The hands and feet are prehensile and may be considered an adaptation for grasping and climbing. Epidermal surfaces from the palmar and plantar areas of the hands and feet of P. acutirostris were prepared for SEM examination, and studied at various magnifications. They show three major levels of complexity: (1) scale types, organized in gradients of size and imbrication, (2) scalar ornamentations, organized by increasing complexity and polarity, and (3) presence of Oberhautchen showing typically iguanian honeycomb micro-ornamentations. The shape and surface structure of the scales with their pattern of micro-ornamental peaks, which improve grip, and the grasping hands and feet indicate that P. acutirostris is morpho-functionally specialized for arboreality. (C) 2009 Elsevier GmbH. All rights reserved.

A real-time polymerase chain reaction assay for the identification and quantification of American Leishmania species on the basis of glucose-6-phosphate dehydrogenase

A real-time polymerase chain reaction (PCR) test was developed on the basis of the Leishmania glucose-6-phosphate dehydrogenase locus that enables identification and quantification of parasites. Using two independent pairs of primers in SYBR-Green assays, the test identified etiologic agents of cutaneous leishmaniasis belonging to both subgenera, Leishmania (Viannia) and Leishmania (Leishmania) in the Americas. Furthermore, use of TaqMan probes enables distinction between L. (V.) braziliensis or L. (V.) peruviania from the other L. (Viannia) species. All assays were negative with DNA of related trypanosomatids, humans, and mice. The parasite burden was estimated by normalizing the number of organisms per total amount of DNA in the sample or per host glyceraldehyde-3-phosphate dehydrogenase copies. The real-time PCR assay for L. (Leishmania) subgenus showed a good linear correlation with quantification on the basis of a limiting dilution assay in experimentally infected mice. The test successfully identifies and quantifies Leishmania in human biopsy specimens and represents a new tool to study leishmaniasis.

Learning of shared attention in sociable robotics

Sociable robots are embodied agents that are part of a heterogeneous society of robots and humans. They Should be able to recognize human beings and each other, and to engage in social, interactions. The use of a robotic architecture may strongly reduce the time and effort required to construct a sociable robot. Such architecture must have structures and mechanisms to allow social interaction. behavior control and learning from environment. Learning processes described oil Science of Behavior Analysis may lead to the development of promising methods and Structures for constructing robots able to behave socially and learn through interactions from the environment by a process of contingency learning. In this paper, we present a robotic architecture inspired from Behavior Analysis. Methods and structures of the proposed architecture, including a hybrid knowledge representation. are presented and discussed. The architecture has been evaluated in the context of a nontrivial real problem: the learning of the shared attention, employing an interactive robotic head. The learning capabilities of this architecture have been analyzed by observing the robot interacting with the human and the environment. The obtained results show that the robotic architecture is able to produce appropriate behavior and to learn from social interaction. (C) 2009 Elsevier Inc. All rights reserved.

A Parallel Hardware Architecture for Scale and Rotation Invariant Feature Detection

This paper proposes a parallel hardware architecture for image feature detection based on the Scale Invariant Feature Transform algorithm and applied to the Simultaneous Localization And Mapping problem. The work also proposes specific hardware optimizations considered fundamental to embed such a robotic control system on-a-chip. The proposed architecture is completely stand-alone; it reads the input data directly from a CMOS image sensor and provides the results via a field-programmable gate array coupled to an embedded processor. The results may either be used directly in an on-chip application or accessed through an Ethernet connection. The system is able to detect features up to 30 frames per second (320 x 240 pixels) and has accuracy similar to a PC-based implementation. The achieved system performance is at least one order of magnitude better than a PC-based solution, a result achieved by investigating the impact of several hardware-orientated optimizations oil performance, area and accuracy.

Semantic mapping using mobile robots

Robotic mapping is the process of automatically constructing an environment representation using mobile robots. We address the problem of semantic mapping, which consists of using mobile robots to create maps that represent not only metric occupancy but also other properties of the environment. Specifically, we develop techniques to build maps that represent activity and navigability of the environment. Our approach to semantic mapping is to combine machine learning techniques with standard mapping algorithms. Supervised learning methods are used to automatically associate properties of space to the desired classification patterns. We present two methods, the first based on hidden Markov models and the second on support vector machines. Both approaches have been tested and experimentally validated in two problem domains: terrain mapping and activity-based mapping.

A Road Following Approach Using Artificial Neural Networks Combinations

Navigation is a broad topic that has been receiving considerable attention from the mobile robotic community over the years. In order to execute autonomous driving in outdoor urban environments it is necessary to identify parts of the terrain that can be traversed and parts that should be avoided. This paper describes an analyses of terrain identification based on different visual information using a MLP artificial neural network and combining responses of many classifiers. Experimental tests using a vehicle and a video camera have been conducted in real scenarios to evaluate the proposed approach.

Characterization of the optical properties of atmospheric aerosols in Amazonia from long-term AERONET monitoring (1993-1995 and 1999-2006)

We present a new climatology of atmospheric aerosols (primarily pyrogenic and biogenic) for the Brazilian tropics on the basis of a high-quality data set of spectral aerosol optical depth and directional sky radiance measurements from Aerosol Robotic Network (AERONET) Cimel Sun-sky radiometers at more than 15 sites distributed across the Amazon basin and adjacent Cerrado region. This network is the only long-term project (with a record including observations from more than 11 years at some locations) ever to have provided ground-based remotely-sensed column aerosol properties for this critical region. Distinctive features of the Amazonian area aerosol are presented by partitioning the region into three aerosol regimes: southern Amazonian forest, Cerrado, and northern Amazonian forest. The monitoring sites generally include measurements from the interval 1999-2006, but some sites have measurement records that date back to the initial days of the AERONET program in 1993. Seasonal time series of aerosol optical depth (AOD), angstrom ngstrom exponent, and columnar-averaged microphysical properties of the aerosol derived from sky radiance inversion techniques (single-scattering albedo, volume size distribution, fine mode fraction of AOD, etc.) are described and contrasted for the defined regions. During the wet season, occurrences of mineral dust penetrating deep into the interior were observed.