A reinforcement learning approach for robot control in an unknown environment

In this paper, a control approach based on reinforcement learning is present for a robot to complete a dynamic task in an unknown environment. First, a temporal difference-based reinforcement learning algorithm and its evaluation function are used to make the robot learn with its trials and errors as well as experiences. Second, the simulation are carried out to adjust the parameters of the learning algorithm and determine an optimal policy by using the models of a robot. Last, the effectiveness of the present approach is demonstrated by balancing an inverse pendulum in the unknown environment.

Flowchart software for a modular robot

Flowcharting is a common method of setting out the requirements for a piece of code. It is simple with few rules to follow. Rarely however, is it used as the code itself. This paper describes the outline of a software package that uses the flowchart as the code for a small, autonomous, modular robot, designed for use in High Schools and Universities at an introductory level. By using flowcharting the student is introduced to the concept of structured programming. A flowchart is often the first step in programming. Here it is the only step, easing the student into the art of coding, and simplifying the teachers job.

Flowchart and kernel software for modular robot

Flowcharting is a common method of setting out the requirements for a piece of code. It is simple with few rules to follow. Rarely however, s it used as the code itself. This paper describes the outline of a software package that uses the flowchart as the code for a small, autonomous, modular robot, designed for use in High Schools. It also describe the code used by the robot to complement the flowchart software creating a system that can be used by students and their teachers to design, build and program a robot without previous programming experience.

An autonomous trimming system of large glass fiber reinforced plastics parts using an omni-directional mobile robot and its control

This paper describes an automated trimming system of large glass fiber reinforced plastic (GFRP) using an omni-directional wheeled mobile robot (WMR) and its path control method. In trimming GFRP parts, much glass fiber and plastic powder dust occur and it becomes bad visible in environment. It is necessary to correct dead-reckoning errors of the WMR in order to control its moving path. We have discussed an external correction method of the dead-reckoning errors for the WMR using ultrasonic sensor.

Kinematic and dynamic modeling of a robotic head with linear motors

Our present research focuses on kinematic and dynamic modeling of a 3-DOF robotic cutting head for the next generation of CNC machines. The robotic cutting head is one kind of parallel manipulator of 3-PUU type, which has a high flexibility of motion in three-dimensional space. The parallel manipulator consists of three linear servomotors, which drive three connecting rods independently according to the cutting strategy. Being a parallel manipulator, the robotic cutting head has higher stiffness and position accuracy; consequently, higher velocities and accelerations can be achieved. A very suitable application of this mechanism is as a cutting head of a precision machine tool for three-dimensional cutting problems.

Path planning for sensor data collecting mobile robot

A mobile robot employed for data collection is faced with the problem of travelling from an initial location to a final location while maintaining as close a distance as possible to all the sensors at a given time in the journey. Here we employ optimal control ideas in forming the necessary control commands for such a robot resulting not only the necessary acceleration commands for the underlying robot, but also the resulting trajectory. This approach can also be easily extended for the case of producing the optimal trajectory for an ariel vehicle used for data collection from indiscriminately scattered ad-hoc sensors located on the ground. We demonstrate the implementation of our algorithm using a Pioneer 3-AT robot.

Fuzzy haptic augmentation for telerobotic stair climbing

Teleoperated robotic systems provide a valuable solution for the exploration of hazardous environments. The ability to explore dangerous environments from the safety of a remote location represents an important progression towards the preservation of human safety in the inevitable response to such a threat. While the benefits of removing physical human presence are clear, challenges associated with remote operation of a robotic system need to be addressed. Removing direct human presence from the robot's operating environment introduces telepresence as an important consideration in achieving the desired objective. The introduction of the haptic modality represents one approach towards improving operator performance subject to reduced telepresence. When operating in an urban environment, teleoperative stair climbing is not an uncommon scenario. This work investigates the operation of an articulated track mobile robot designed for ascending stairs under teleoperative control. In order to assist the teleoperator in improved navigational capabilities, a fuzzy expert system is utilised to provide the teleoperator with intelligent haptic augmentation with the aim of improving task performance.

Simultaneous state and input estimation with application to a two-link robotic system

This paper addresses the problem of estimating simultaneously the state and input of a nonlinear system with application to a two link robotic manipulator - the Pendubot. The system nonlinearity comprises a Lipschitz function with respect to the state, and a nonlinear term which is a function of both the state and input. It is shown that under some conditions, an observer can be designed to estimate simultaneously the system’s state and input. Simulation and experimental results, obtained around the inverted equilibrium position, are presented to demonstrate the validity of the approach.

Observer-based decentralized approach to robotic formation control

Control of a group of mobile robots in a formation requires not only environmental sensing but also communication among vehicles. Enlarging the size of the platoon of vehicles causes difficulties due to communications bandwidth limitations. Decentralized control may be an appropriate approach in those cases when the states of all vehicles cannot be obtained in a centralized manner. This paper presents a solution to the problem of decentralized implementation of a global state-feedback controller for N mobile robots in a formation. The proposed solution is based on the design of functional observers to estimate asymptotically the global state-feedback control signals by using the corresponding local output information and some exogenous global functions. The proposed technique is tested through simulation and experiments for the control of groups of Pinoneer-based non-holonomic mobile robots.

Towards optimization of a real-world robotic-sensor system of systems

The problem of threat detection in an unstructured environment is considered. Three systems, comprising of robots and sensors, are proposed to form a system of systems (SoS) to find a solution to the problem. System interactions are defined to provide a framework for formulation as an SoS optimization problem. Different cost and objective functions are introduced for optimization of local criteria. Using different weights, a linear combination of the local cost and objective functions is obtained to propose a global objective function. An algorithm is suggested to find an optimum value for the global objective function leading towards optimization of the SoS.

Simulating autonomous robot teams with Microsoft robotics studio

This paper presents an application of Microsoft Robotics Studio (MSRS) in which a team of six four wheel drive, ground based robots explore and map simulated terrain. The user has the ability to modify the terrain and assign destination objectives to the team while the simulation is running. The terrain is initially generated using a gray scale image, in which the intensity of each pixel in the image gives an altitude datum. The robots start with no knowledge of their surroundings, and map the terrain as they attempt to reach user-defined target objectives. The mapping process simulates the use of common sensory hardware to determine datum points, including provision for field of view, detection range, and measurement accuracy. If traversal of a mapped area is indicated by the users’ targeting commands, path planning heuristics developed for MSRS by the author in earlier work are used to determine an efficient series of waypoints to reach the objective. Mutability of terrain is also explored- the user is able to modify the terrain without stopping the simulation. This forces the robots to adapt to changing environmental conditions, and permits analysis of the robustness of mapping algorithms used when faced with a changing world.

Haptic control methodologies for telerobotic stair traversal

Teleoperated mobile robots provide the ability for a human operator to safely explore and evaluate hazardous environments. This ability represents an important progression towards the preservation of human safety in the inevitable response to situations such as terrorist activities and urban search and rescue. The benefits of removing physical human presence from such environments are obvious, however challenges inhibiting task performance when remotely operating a mobile robotic system need to be addressed. The removal of physical human presence from the target environment introduces telepresence as a vital consideration in achieving the desired objective. Introducing haptic human-robotic interaction represents one approach towards improving operator performance in such a scenario. Teleoperative stair traversal proves to be a challenging task when undertaking threat response in an urban environment. This article investigates the teleoperation of an articulated track mobile robot designed for traversing stairs in a threat response scenario. Utilising a haptic medium for bilateral human-robotic interaction, the haptic cone methodology is introduced with the aim of providing the operator with a vision-independent, intuitive indication of the current commanded robot velocity. The haptic cone methodology operates synergistically with the introduced fuzzy-haptic augmentation for improving teleoperator performance in the stair traversal scenario.

Heavy tools manipulation by low powered direct-drive five-bar parallel robot

This paper presents a simple and available system for manipulation of heavy tools by low powered manipulator for industrial applications. In the heavy manufacturing industries, sometimes, heavy tools are employed for different types of work. But the application of robots with heavy tools is not possible due to the limited torque limits of actuators. Suspended tool systems (STS) have been proposed to manipulate heavy tools by low powered robot-arm for this purpose. A low powered five-bar direct-drive parallel manipulator is designed and constructed to manipulate heavy tools suspended from a spring balancer. The validity, usefulness, and effectiveness of the suspended tool system are shown by experimental results.

Multi-robot hunting in dynamic environments

This paper is concerned with multi-robot hunting in dynamic environments. A BCSLA approach is proposed to allow mobile robots to capture an intelligent evader. During the process of hunting, four states including dispersion-random-search, surrounding, catch and prediction are employed. In order to ensure each robot appropriate movement in each state, a series of strategies are developed in this paper. The dispersion-search strategy enables the robots to find the evader effectively. The leader-adjusting strategy aims to improve the hunting robots’ response to environmental changes and the outflank strategy is proposed for the hunting robots to force the evader to enter a besieging circle. The catch strategy is designed for shrinking the besieging circle to catch the evader. The predict strategy allows the robots to predict the evader’s position when they lose the tracking information about the evader. A novel collision-free motion strategy is also presented in this paper, which is called the direction-optimization strategy. To test the effect of cooperative hunting, the target to be captured owns a safety-motion strategy, which helps it to escape being captured. The computer simulations support the rationality of the approach.