Towards vision-based deep reinforcement learning for robotic motion control

This paper introduces a machine learning based system for controlling a robotic manipulator with visual perception only. The capability to autonomously learn robot controllers solely from raw-pixel images and without any prior knowledge of configuration is shown for the first time. We build upon the success of recent deep reinforcement learning and develop a system for learning target reaching with a three-joint robot manipulator using external visual observation. A Deep Q Network (DQN) was demonstrated to perform target reaching after training in simulation. Transferring the network to real hardware and real observation in a naive approach failed, but experiments show that the network works when replacing camera images with synthetic images.

A Novel Opto Electro Mechanical Interrogation System for Measuring Wavelength Shifts in Fiber Bragg Grating (FBG) Sensors

Here, we describe a novel FBG interrogation system in which FBGs are used as both sensing and reference elements. The reference FBGs is bonded to a mechanical flexure system having a linear amplification of 1:3.5, which is actuated using a piezo-actuator by applying a 0-150V ramp. The lengths of the reference gratings decide the maximum strain that can be applied to the reference grating, which in turn decides that strain range which can be interrogated. The main advantages of the present system are the on-line measurement of the wavelength shifts, small size, good sensitivity, multiplexing capability and low cost.

Seismic behaviour of rigid-faced reinforced soil retaining wall models:reinforcement effect

This paper presents the results of shaking table tests on models of rigid-faced reinforced soil retaining walls in which reinforcement materials of different tensile strength were used. The construction of the model retaining walls in a laminar box mounted on a shaking table, the instrumentation and the results from the shaking table tests are described in detail and the effects of the reinforcement parameters on the acceleration response at different elevations of the retaining wall, horizontal soil pressures and face deformations are presented. It was observed from these tests that the horizontal face displacement response of the rigid-faced retaining walls was significantly affected by the inclusion of reinforcement and even low-strength polymer reinforcement was found to be efficient in significantly reducing the deformation of the face. The acceleration amplifications were, however, observed to be less influenced by the reinforcement parameters. The results obtained from this study are helpful in understanding the relative performance of reinforced soil retaining walls under the different test conditions used in the experiments.

Applications of coralline hydroxyapatite with bioabsorbable containment and reinforcement as bone graft substitute : An experimental study

The aim of the present experimental study was to find out if the applications of coralline hydroxyapatite (HA) can be improved by using bioabsorbable containment or binding substance with particulate HA in mandibular contour augmentation and by using bioabsorbable fibre-reinforced HA blocks in filling bone defects and in anterior lumbar interbody fusion. The use of a separate curved polyglycolide (PGA) containment alone or together with a fast resorbing polyglycolide/polylactide (PGA/PLA) binding substance were compared to the conventional non-contained method in ridge augmentation in sheep. The contained methods decreased HA migration, but the augmentations did not differ significantly. The use of the containment caused a risk for wound dehiscence and infection. Histologically there was a rapid connective tissue ingrowth into the HA graft and it was more abundant with the PGA containment compared to the non-contained augmentation and even additionally rich when the HA particles were bound with PGA/PLA copolymer. However, the bone ingrowth was best in the non-contained augmentation exceeding 10-12 % of the total graft area at 24 weeks. Negligible or no bone ingrowth was seen in the cases where the polymer composite was added to the HA particles and, related to that, foreign-body type cells were seen at the interface between the HA and host bone. The PGA and poly-dl/l-lactide (PDLLA) fibre-reinforced coralline HA blocks were studied in the metaphyseal and in the diaphyseal defects in rabbits. A rapid bone ingrowth was seen inside the both types of implants. Both PGA and PDLLA fibres induced an inflammatory fibrous reaction around themselves but it did not hinder the bone ingrowth. The bone ingrowth pattern was directed according to the loading conditions so that the load-carrying cortical ends of the implants as well as the implants sited in the diaphyseal defects were the most ossified. The fibre-reinforced coralline HA implants were further studied as stand-alone grafts in the lumbar anterior interbody implantation in pigs. The strength of the HA implants proved not to be adequate, the implants fractured in six weeks and the disc space was gradually lost similarly to that of the discectomized spaces. Histologically, small quantities of bone ingrowth was seen in some of the PGA and PDLLA reinforced coralline implants while no bone formation was identified in any of the PDLLA reinforced synthetic porous HA implants. While fragmented, the inner structure of the implants was lost, the bone ingrowth was minimal, and the disc was replaced by the fibrous connective tissue. When evaluated radiologically the grade of ossification was assessed as better than histologically, and, when related to the histologic findings, CT was more dependable than the plain films to show ossification of the implanted disc space. Local kyphosis was a frequent finding along with anterior bone bridging and ligament ossification as a consequence of instability of the implanted segment.

Effect of reinforcement form on the bearing capacity of square footings on sand

This paper presents the results of laboratory model loading tests and numerical studies carried out on square footings supported on geosynthetic reinforced sand beds. The relative performance of different forms of geosynthetic reinforcement (i.e. geocell, planar layers and randomly distributed mesh elements) in foundation beds is compared; using same quantity of reinforcement in each test. A biaxial geogrid and a geonet are used for reinforcing the sand beds. Geonet is used in two forms of reinforcement, viz. Planar layers and geocell, while the biaxial geogrid was used in three forms of reinforcement, viz. planar layers, geocell and randomly distributed mesh elements. Laboratory load tests on unreinforced and reinforced footings are simulated in a numerical model and the results are analyzed to understand the distribution of displacements and stresses below the footing better. Both the experimental and numerical studies demonstrated that the geocell is the most advantageous form of soil reinforcement technique of those investigated, provided there is no rupture of the material during loading. Geogrid used in the form of randomly distributed mesh elements is found to be inferior to the other two forms. Some significant observations on the difference in reinforcement mechanism for different forms of reinforcement are presented in this paper.

Compact fiber Bragg grating dynamic strain sensor cum broadband thermometer for thermally unstable ambience

An instrument for simultaneous measurement of dynamic strain and temperature in a thermally unstable ambience has been proposed, based on fiber Bragg grating technology. The instrument can function as a compact and stand-alone broadband thermometer and a dynamic strain gauge. It employs a source wavelength tracking procedure for linear dependence of the output on the measurand, offering high dynamic range. Two schemes have been demonstrated with their relative merits. As a thermometer, the present instrumental configuration can offer a linear response in excess of 500 degrees C that can be easily extended by adding a suitable grating and source without any alteration in the procedure. Temperature sensitivity is about 0.06 degrees C for a bandwidth of 1 Hz. For the current grating, the upper limit of strain measurement is about 150 mu epsilon with a sensitivity of about 80 n epsilon Hz(-1/2). The major source of uncertainty associated with dynamic strain measurement is the laser source intensity noise, which is of broad spectral band. A low noise source device or the use of optical power regulators can offer improved performance. The total harmonic distortion is less than 0.5% up to about 50 mu epsilon, 1.2% at 100 mu epsilon and about 2.3% at 150 mu epsilon. Calibrated results of temperature and strain measurement with the instrument have been presented. Traces of ultrasound signals recorded by the system at 200 kHz, in an ambience of 100-200 degrees C temperature fluctuation, have been included. Also, the vibration spectrum and engine temperature of a running internal combustion engine has been recorded as a realistic application of the system.

Inhibition of Tyrosine Phosphorylation of Sperm Flagellar Proteins, Outer Dense Fiber Protein-2 and Tektin-2, Is Associated With Impaired Motility During Capacitation of Hamster Spermatozoa

In mammals, acquisition of fertilization competence of spermatozoa is dependent on the phenomenon of sperm capacitation. One of the critical molecular events of sperm capacitation is protein tyrosine phosphorylation. In a previous study, we demonstrated that a specific epidermal growth factor receptor (EGFR)-tyrosine kinase inhibitor, tyrphostin-A47, inhibited hamster sperm capacitation, accompanied by a reduced sperm protein tyrosine phosphorylation. Interestingly, a high percentage of tyrphostin-A47-treated spermatozoa exhibited circular motility, which was associated with a distinct hypo-tyrosine phosphorylation of flagellar proteins, predominantly of Mr 45,000-60,000. In this study, we provide evidence on the localization of capacitation-associated tyrosine-phosphorylated proteins to the nonmembranous, structural components of the sperm flagellum. Consistent with this, we show their ultrastructural localization in the outer dense fiber, axoneme, and fibrous sheath of spermatozoa. Among hypo-tyrosine phosphorylated major proteins of tyrphostin-A47-treated spermatozoa, we identified the 45 kDa protein as outer dense fiber protein-2 and the 51 kDa protein as tektin-2, components of the sperm outer dense fiber and axoneme, respectively. This study shows functional association of hypo-tyrosine-phosphorylation status of outer dense fiber protein-2 and tektin-2 with impaired flagellar bending of spermatozoa, following inhibition of EGFR-tyrosine kinase, thereby showing the critical importance of flagellar protein tyrosine phosphorylation during capacitation and hyperactivation of hamster spermatozoa.

Optimal configurations of active fiber composites based on asymptotic torsional analysis - art. no. 641413

Active Fiber Composites (AFC) possess desirable characteristics over a wide range of smart structure applications, such as vibration, shape and flow control as well as structural health monitoring. This type of material, capable of collocated actuation and sensing, call be used in smart structures with self-sensing circuits. This paper proposes four novel applications of AFC structures undergoing torsion: sensors and actuators shaped as strips and tubes; and concludes with a preliminary failure analysis. To enable this, a powerful mathematical technique, the Variational Asymptotic Method (VAM) was used to perform cross-sectional analyses of thin generally anisotropic AFC beams. The resulting closed form expressions have been utilized in the applications presented herein.

Macro fiber composite (MFC) as a delamination sensor in antisymmetric laminates

The change in extension-twist Coupling due to delamination in antisymmetric laminates is experimentally measured. Experimental results are compared with the results from analytical expression existing in literature and finite element analysis. The application of the Macro-Fiber Composite (MFC) developed at the NASA Langley Research Center for sensing the delamination in the laminates is investigated. While many applications have been reported in the literature using the MFC as an actuator, here its use as a twist sensor has been studied. The real-life application envisaged is structural health monitoring of laminated composite flexbeams taking advantage of the symmetry in the structure. Apart from the defect detection under symmetric conditions, other methods of health monitoring for the same structure are reported for further validation. Results show that MFC works well as a sensor.

Simulation of wavelength/time multiple-pulses-per-row fiber-optic CDMA network

Passive wavelength/time fiber-optic code division multiple access (WIT FO-CDMA) network is a viable option for highspeed access networks. Constructions of 2-D codes, suitable for incoherent WIT FO-CDMA, have been proposed to reduce the time spread of the 1-D sequences. The 2-D constructions can be broadly classified as 1) hybrid codes and 2) matrix codes. In our earlier work [141, we had proposed a new family of wavelength/time multiple-pulses-per-row (W/T MPR) matrix codes which have good cardinality, spectral efficiency and at the same time have the lowest off-peak autocorrelation and cross-correlation values equal to unity. In this paper we propose an architecture for a WIT MPR FO-CDAM network designed using the presently available devices and technology. A complete FO-CDMA network of ten users is simulated, for various number of simultaneous users and shown that 0 --> 1 errors can occur only when the number of interfering users is at least equal to the threshold value.

Strain-Temperature Discrimination Using a Single Fiber Bragg Grating

The out-diffusion of germanium from the core of a photosensitive fiber under elevated temperature is exploited to form a Fabry-Perot filter within a single fiber Bragg grating, by subjecting the diffused region to a single exposure using the standard phase-mask technique. A key aspect of our work is the measurement of the out-diffusion through energy dispersive X-ray analysis. Furthermore, we demonstrate the use of the above single-grating filter for discrimination and simultaneous measurement of strain and temperature. The proposed technique provides a significant advantage over other existing methods that require at least two gratings.

The Role of Interface Modification on Thermal Degradation and Crystallization Behavior of Composites from Commingled Polypropylene Fiber and Banana Fiber

The thermal degradation behavior of banana fiber and polypropylene/banana fiber composites has been studied by thermogravimetric analysis. Banana fiber was found to be decomposing in two stages, first one around 320 degrees C and the second one around 450 degrees C. For chemically treated banana fiber, the decomposition process has been at a higher temperature, indicating thermal stability for the treated fiber. Activation energies for thermal degradation were estimated using Coats and Redfern method. Calorific value of the banana fiber was measured using a constant volume isothermal bomb calorimeter. rystallization studies exhibited an increase in the crystallization temperature and crystallinity of polypropylene upon the addition of banana fiber. POLYM. COMPOS., 31:1113-1123, 2010. (C) 2009 Society of Plastics Engineers.

Optimal non-linear reinforcement schemes for stochastic automata

Two optimal non-linear reinforcement schemes—the Reward-Inaction and the Penalty-Inaction—for the two-state automaton functioning in a stationary random environment are considered. Very simple conditions of symmetry of the non-linear function figuring in the reinforcement scheme are shown to be necessary and sufficient for optimality. General expressions for the variance and rate of learning are derived. These schemes are compared with the already existing optimal linear schemes in the light of average variance and average rate of learning.

Dry sliding wear behavior of Al2O3 fiber reinforced aluminum composites

Dry sliding wear behavior of die-cast ADC12 aluminum alloy composites reinforced with short alumina fibers were investigated by using a pin-on-disk wear tester. The Al2O3 fibers were 4 mu m in diameter and were present in volume fractions (T-f)ranging from 0.03 to 0.26, The length of the fiber varied from 40 to 200 mu m. Disks of aluminum-alumina composites were rubbed against a pin of nitrided stainless steel SUS440B with a load of 10 N at a sliding velocity of 0.1 m/s. The unreinforced ADC 12 aluminum alloy and their composites containing low volume fractions of alumina (V-f approximate to 0.05) showed a sliding-distance-dependent transition from severe to mild wear. However, composites containing high volume fractions of alumina ( V-f > 0.05) exhibited only mild wear for all sliding distances. The duration of occurrence of the severe wear regime and the wear rate both decrease with increasing volume fraction. In MMCs the wear rate in the mild wear regime decreases with increase in volume fraction: reaching a minimum value at V-f = 0.09 Beyond V-f = 0.09 the wear rate increasesmarginally. On the other hand, the wear rate of the counterface (steel pin) was found to increase moderately with increase in V-f. From the analysis of wear data and detailed examination of (a) worn surfaces, (b) their cross-sections and (c) wear debris, two modes of wear mechanisms have been identified to be operative, in these materials and these are: (i) adhesive wear in the case of unreinforced matrix material and in MMCs with low Vf and (ii) abrasive wear in the case of MMCs with high V-f. (C) 2000 Elsevier Science Ltd. All rights reserved.

A reinforcement learning based algorithm for finite horizon Markov decision processes

We develop a simulation based algorithm for finite horizon Markov decision processes with finite state and finite action space. Illustrative numerical experiments with the proposed algorithm are shown for problems in flow control of communication networks and capacity switching in semiconductor fabrication.