Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Analysis of System Signals Based on Cross Recurrence Method for System Dynamics Characterization

Natural systems are inherently non linear. Recurrent behaviours are typical of natural systems. Recurrence is a fundamental property of non linear dynamical systems which can be exploited to characterize the system behaviour effectively. Cross recurrence based analysis of sensor signals from non linear dynamical system is presented in this thesis. The mutual dependency among relatively independent components of a system is referred as coupling. The analysis is done for a mechanically coupled system specifically designed for conducting experiment. Further, cross recurrence method is extended to the actual machining process in a lathe to characterize the chatter during turning. The result is verified by permutation entropy method. Conventional linear methods or models are incapable of capturing the critical and strange behaviours associated with the dynamical process. Hence any effective feature extraction methodologies should invariably gather information thorough nonlinear time series analysis. The sensor signals from the dynamical system normally contain noise and non stationarity. In an effort to get over these two issues to the maximum possible extent, this work adopts the cross recurrence quantification analysis (CRQA) methodology since it is found to be robust against noise and stationarity in the signals. The study reveals that the CRQA is capable of characterizing even weak coupling among system signals. It also divulges the dependence of certain CRQA variables like percent determinism, percent recurrence and entropy to chatter unambiguously. The surrogate data test shows that the results obtained by CRQA are the true properties of the temporal evolution of the dynamics and contain a degree of deterministic structure. The results are verified using permutation entropy (PE) to detect the onset of chatter from the time series. The present study ascertains that this CRP based methodology is capable of recognizing the transition from regular cutting to the chatter cutting irrespective of the machining parameters or work piece material. The results establish this methodology to be feasible for detection of chatter in metal cutting operation in a lathe.

Management of Public Road Transport System in Trivandrum City

An efficient passenger road transport system is a boon to any city and an inefficient one its bane. Passenger bus transport operation involves various aspects like passenger convenience, profitability of operation and social, technological and environmental factors. The author’s interest in this area was aroused when he conducted a traffic survey of Trivandrum City in 1979. While some studies on the performance of the Kerala State Road Transport Corporation in specific areas like finance, inventory control etc. have already been made, no study has been made from the operational point of view. The study is also the first one of its kind in dealing with the transportation problems for a second order city like Trivandrum. The objective of this research study is to develop a scientific basis for analysing and understanding the various operational aspects of urban bus transport management like assessing travel demand, depot location, fleet allocation, vehicle scheduling, maintenance etc. The operation of public road transportation in Trivandrum City is analysed on the basis of this theoretical background. The studies made have relevance to any medium sized city in India or even abroad. If not properly managed, deterioration of any public utility system is a natural process and it adversely affects the consumers, the economy and the nation. Making any system more efficient requires careful analysis, judicious decision making and proper implementation. It is hoped that this study will throw some light into the various operational aspects of urban passenger road transport management which can be of some help to make it perform more efficiently

Numerical Simulation of Two-Part Underwater Towing System

The motion instability is an important issue that occurs during the operation of towed underwater vehicles (TUV), which considerably affects the accuracy of high precision acoustic instrumentations housed inside the same. Out of the various parameters responsible for this, the disturbances from the tow-ship are the most significant one. The present study focus on the motion dynamics of an underwater towing system with ship induced disturbances as the input. The study focus on an innovative system called two-part towing. The methodology involves numerical modeling of the tow system, which consists of modeling of the tow-cables and vehicles formulation. Previous study in this direction used a segmental approach for the modeling of the cable. Even though, the model was successful in predicting the heave response of the tow-body, instabilities were observed in the numerical solution. The present study devises a simple approach called lumped mass spring model (LMSM) for the cable formulation. In this work, the traditional LMSM has been modified in two ways. First, by implementing advanced time integration procedures and secondly, use of a modified beam model which uses only translational degrees of freedoms for solving beam equation. A number of time integration procedures, such as Euler, Houbolt, Newmark and HHT-α were implemented in the traditional LMSM and the strength and weakness of each scheme were numerically estimated. In most of the previous studies, hydrodynamic forces acting on the tow-system such as drag and lift etc. are approximated as analytical expression of velocities. This approach restricts these models to use simple cylindrical shaped towed bodies and may not be applicable modern tow systems which are diversed in shape and complexity. Hence, this particular study, hydrodynamic parameters such as drag and lift of the tow-system are estimated using CFD techniques. To achieve this, a RANS based CFD code has been developed. Further, a new convection interpolation scheme for CFD simulation, called BNCUS, which is blend of cell based and node based formulation, was proposed in the study and numerically tested. To account for the fact that simulation takes considerable time in solving fluid dynamic equations, a dedicated parallel computing setup has been developed. Two types of computational parallelisms are explored in the current study, viz; the model for shared memory processors and distributed memory processors. In the present study, shared memory model was used for structural dynamic analysis of towing system, distributed memory one was devised in solving fluid dynamic equations.

Nitrification in a packed bed bioreactor integrated into amarine recirculating maturation system under different substrate concentrations and flow rates

BACKGROUND: A packed bed bioreactor (PBBR) activated with an indigenous nitrifying bacterial consortia was developed and commercialized for rapid establishment of nitrification in brackish water and marine hatchery systems in the tropics. The present study evaluated nitrification in PBBR integrated into a Penaeus monodon recirculating maturation system under different substrate concentrations and flow rates. RESULTS:Instantnitrificationwasobservedafter integration ofPBBRinto thematuration system.TANandNO2-Nconcentrations were always maintained below0.5 mg L−1 during operation. The TANandNO2-N removalwas significant (P < 0.001) in all the six reactor compartments of the PBBR having the substrates at initial concentrations of 2, 5 and 10 mg L−1. The average volumetric TAN removal rates increased with flow rates from 43.51 (250 L h−1) to 130.44 (2500 L h−1) gTAN m−3 day−1 (P < 0.05). FISH analysis of the biofilms after 70 days of operation gave positive results with probes NSO 190 ((β ammonia oxidizers), NsV 443 (Nitrosospira spp.) NEU (halophilic Nitrosomonas), Ntspa 712 (Phylum Nitrospira) indicating stability of the consortia. CONCLUSION: The PBBR integrated into the P. monodon maturation system exhibited significant nitrification upon operation for 70 days as well as at different substrate concentrations and flow rates. This system can easily be integrated into marine and brackish water aquaculture systems, to establish instantaneous nitrification

A Realization Of An Fpga Sub System For Reducing Odometric Localization Errors In Wheeled Mobile Robots

This paper introduces a simple and efficient method and its implementation in an FPGA for reducing the odometric localization errors caused by over count readings of an optical encoder based odometric system in a mobile robot due to wheel-slippage and terrain irregularities. The detection and correction is based on redundant encoder measurements. The method suggested relies on the fact that the wheel slippage or terrain irregularities cause more count readings from the encoder than what corresponds to the actual distance travelled by the vehicle. The standard quadrature technique is used to obtain four counts in each encoder period. In this work a three-wheeled mobile robot vehicle with one driving-steering wheel and two-fixed rear wheels in-axis, fitted with incremental optical encoders is considered. The CORDIC algorithm has been used for the computation of sine and cosine terms in the update equations. The results presented demonstrate the effectiveness of the technique

Design and Development of a Remotely Operated Underwater Multi-Robot Manipulator Controller

For the scientific and commercial utilization of Ocean resources, the role of intelligent underwater robotic systems are of great importance. Scientific activities like Marine Bio-technology, Hydrographic mapping, and commercial applications like Marine mining, Ocean energy, fishing, aquaculture, cable laying and pipe lining are a few utilization of ocean resources. As most of the deep undersea exploration are beyond the reachability of divers and also as the use of operator controlled and teleoperated Remotely Operated Vehicles (ROVs) and Diver Transport Vehicles (DTVs) turn out to be highly inefficient, it is essential to have a fully automated system capable providing stable control and communication links for the unstructured undersea environment.

FEM based Virtual Prototyping and Design of Third Harmonic Excitation System for Low Voltage Brushless Alternators

Salient pole brushless alternators coupled to IC engines are extensively used as stand-by power supply units for meeting in- dustrial power demands. Design of such generators demands high power to weight ratio, high e ciency and low cost per KVA out- put. Moreover, the performance characteristics of such machines like voltage regulation and short circuit ratio (SCR) are critical when these machines are put into parallel operation and alterna- tors for critical applications like defence and aerospace demand very low harmonic content in the output voltage. While designing such alternators, accurate prediction of machine characteristics, including total harmonic distortion (THD) is essential to mini- mize development cost and time. Total harmonic distortion in the output voltage of alternators should be as low as possible especially when powering very sophis- ticated and critical applications. The output voltage waveform of a practical AC generator is replica of the space distribution of the ux density in the air gap and several factors such as shape of the rotor pole face, core saturation, slotting and style of coil disposition make the realization of a sinusoidal air gap ux wave impossible. These ux harmonics introduce undesirable e ects on the alternator performance like high neutral current due to triplen harmonics, voltage distortion, noise, vibration, excessive heating and also extra losses resulting in poor e ciency, which in turn necessitate de-rating of the machine especially when connected to non-linear loads. As an important control unit of brushless alternator, the excitation system and its dynamic performance has a direct impact on alternator's stability and reliability. The thesis explores design and implementation of an excitation i system utilizing third harmonic ux in the air gap of brushless al- ternators, using an additional auxiliary winding, wound for 1=3rd pole pitch, embedded into the stator slots and electrically iso- lated from the main winding. In the third harmonic excitation system, the combined e ect of two auxiliary windings, one with 2=3rd pitch and another third harmonic winding with 1=3rd pitch, are used to ensure good voltage regulation without an electronic automatic voltage regulator (AVR) and also reduces the total harmonic content in the output voltage, cost e ectively. The design of the third harmonic winding by analytic methods demands accurate calculation of third harmonic ux density in the air gap of the machine. However, precise estimation of the amplitude of third harmonic ux in the air gap of a machine by conventional design procedures is di cult due to complex geome- try of the machine and non-linear characteristics of the magnetic materials. As such, prediction of the eld parameters by conven- tional design methods is unreliable and hence virtual prototyping of the machine is done to enable accurate design of the third har- monic excitation system. In the design and development cycle of electrical machines, it is recognized that the use of analytical and experimental methods followed by expensive and in exible prototyping is time consum- ing and no longer cost e ective. Due to advancements in com- putational capabilities over recent years, nite element method (FEM) based virtual prototyping has become an attractive al- ternative to well established semi-analytical and empirical design methods as well as to the still popular trial and error approach followed by the costly and time consuming prototyping. Hence, by virtually prototyping the alternator using FEM, the important performance characteristics of the machine are predicted. Design of third harmonic excitation system is done with the help of results obtained from virtual prototype of the machine. Third harmonic excitation (THE) system is implemented in a 45 KVA ii experimental machine and experiments are conducted to validate the simulation results. Simulation and experimental results show that by utilizing third harmonic ux in the air gap of the ma- chine for excitation purposes during loaded conditions, triplen harmonic content in the output phase voltage is signi cantly re- duced. The prototype machine with third harmonic excitation system designed and developed based on FEM analysis proved to be economical due to its simplicity and has the added advan- tage of reduced harmonics in the output phase voltage.

Modelling and simulation of a photovoltaic fuel cell hybrid system

A stand-alone power system is an autonomous system that supplies electricity to the user load without being connected to the electric grid. This kind of decentralized system is frequently located in remote and inaccessible areas. It is essential for about one third of the world population which are living in developed or isolated regions and have no access to an electricity utility grid. The most people live in remote and rural areas, with low population density, lacking even the basic infrastructure. The utility grid extension to these locations is not a cost effective option and sometimes technically not feasible. The purpose of this thesis is the modelling and simulation of a stand-alone hybrid power system, referred to as “hydrogen Photovoltaic-Fuel Cell (PVFC) hybrid system”. It couples a photovoltaic generator (PV), an alkaline water electrolyser, a storage gas tank, a proton exchange membrane fuel cell (PEMFC), and power conditioning units (PCU) to give different system topologies. The system is intended to be an environmentally friendly solution since it tries maximising the use of a renewable energy source. Electricity is produced by a PV generator to meet the requirements of a user load. Whenever there is enough solar radiation, the user load can be powered totally by the PV electricity. During periods of low solar radiation, auxiliary electricity is required. An alkaline high pressure water electrolyser is powered by the excess energy from the PV generator to produce hydrogen and oxygen at a pressure of maximum 30bar. Gases are stored without compression for short- (hourly or daily) and long- (seasonal) term. A proton exchange membrane (PEM) fuel cell is used to keep the system’s reliability at the same level as for the conventional system while decreasing the environmental impact of the whole system. The PEM fuel cell consumes gases which are produced by an electrolyser to meet the user load demand when the PV generator energy is deficient, so that it works as an auxiliary generator. Power conditioning units are appropriate for the conversion and dispatch the energy between the components of the system. No batteries are used in this system since they represent the weakest when used in PV systems due to their need for sophisticated control and their short lifetime. The model library, ISET Alternative Power Library (ISET-APL), is designed by the Institute of Solar Energy supply Technology (ISET) and used for the simulation of the hybrid system. The physical, analytical and/or empirical equations of each component are programmed and implemented separately in this library for the simulation software program Simplorer by C++ language. The model parameters are derived from manufacturer’s performance data sheets or measurements obtained from literature. The identification and validation of the major hydrogen PVFC hybrid system component models are evaluated according to the measured data of the components, from the manufacturer’s data sheet or from actual system operation. Then, the overall system is simulated, at intervals of one hour each, by using solar radiation as the primary energy input and hydrogen as energy storage for one year operation. A comparison between different topologies, such as DC or AC coupled systems, is carried out on the basis of energy point of view at two locations with different geographical latitudes, in Kassel/Germany (Europe) and in Cairo/Egypt (North Africa). The main conclusion in this work is that the simulation method of the system study under different conditions could successfully be used to give good visualization and comparison between those topologies for the overall performance of the system. The operational performance of the system is not only depending on component efficiency but also on system design and consumption behaviour. The worst case of this system is the low efficiency of the storage subsystem made of the electrolyser, the gas storage tank, and the fuel cell as it is around 25-34% at Cairo and 29-37% at Kassel. Therefore, the research for this system should be concentrated in the subsystem components development especially the fuel cell.

A Study of a Comprehensive Career Services System in Chinese Universities

Since 1999, with the adoption of expansion policy in higher education by the Chinese government, enrollment and graduate numbers have been increasing at an unprecedented speed. Accustomed to a system in which university graduates were placed, many students are not trained in “selling themselves”, which exacerbates the situation leading to a skyrocketing unemployment rate among new graduates. The idea of emphasizing career services comes with increasing employment pressure among university graduates in recent years. The 1998 “Higher Education Act” made it a legislative requirement. Thereafter, the Ministry of Education issued a series of documents in order to promote the development of career services. All higher education institutions are required to set up special career service centers and to set a ratio of 1:500 between career staff and the total number of students. Related career management courses, especially career planning classes, are required to be clearly included as specific modules into the teaching plan with a requirement of no less than 38 sessions in one semester at all universities. Developing career services in higher education has thus become a hot issue. One of the more notable trends in higher education in recent years has been the transformation of university career service centers from merely being the coordinators of on-campus placement into full service centers for international career development. The traditional core of career services in higher education had been built around guidance, information and placements (Watts, 1997). This core was still in place, but the role of higher education career services has changed considerably in recent years and the nature of each part is being transformed (Watts, 1997). Most services are undertaking a range of additional activities, and the career guidance issue is emphasized much more than before. Career management courses, especially career planning classes, are given special focus in developing career services in the Chinese case. This links career services clearly and directly with the course provision function. In China, most career service centers are engaging in the transformation period from a “management-oriented” organization to a “service-oriented” organization. Besides guidance services, information services and placement activities, there is a need to blend them together with the new additional teaching function, which follows the general trend as regulated by the government. The role of career services has been expanding and this has brought more challenges to its development in Chinese higher education. Chinese universities still remain in the period of exploration and establishment in developing their own career services. In the face of the new situation, it is very important and meaningful to explore and establish a comprehensive career services system to address student needs in the universities. A key part in developing this system is the introduction of career courses and delivering related career management skills to the students. So there is the need to restructure the career service sectors within the Chinese universities in general. The career service centers will operate as a hub and function as a spoke in the wheel of this model system, providing support and information to staff located in individual teaching departments who are responsible for the delivery of career education, information, advice and guidance. The career service centers will also provide training and career planning classes. The purpose of establishing a comprehensive career services system is to provide a strong base for student career development. The students can prepare themselves well in psychology, ideology and ability before employment with the assistance of effective career services. To conclude, according to the different characteristics and needs of students, there will be appropriate services and guidance in different stages and different ways. In other words, related career services and career guidance activities would be started for newly enrolled freshmen and continue throughout their whole university process. For the operation of a comprehensive services system, there is a need for strong support by the government in the form of macro-control and policy guarantee, but support by the government in the form of macro-control and policy guarantee, but also a need for close cooperation with the academic administration and faculties to be actively involved in career planning and employment programs. As an integral function within the universities, career services must develop and maintain productive relationships with relevant campus offices and key stakeholders both within the universities and externally.

Optimal control theory for a unitary operation under dissipative evolution

We show that optimizing a quantum gate for an open quantum system requires the time evolution of only three states irrespective of the dimension of Hilbert space. This represents a significant reduction in computational resources compared to the complete basis of Liouville space that is commonly believed necessary for this task. The reduction is based on two observations: the target is not a general dynamical map but a unitary operation; and the time evolution of two properly chosen states is sufficient to distinguish any two unitaries. We illustrate gate optimization employing a reduced set of states for a controlled phasegate with trapped atoms as qubit carriers and a iSWAP gate with superconducting qubits.

Technical and scale efficiency of cassava production system in Delta State, Nigeria: an application of Two-Stage DEA approach

The present study examines the level of pure technical and scale efficiencies of cassava production system including its sub-processes (that is production and processing stages) of 278 cassava farmers/processors from three regions of Delta State, Nigeria by applying Two-Stage Data Envelopment Analysis (DEA) approach. Results reveal that pure technical efficiency (PTE) is significantly lower at the production stage 0.41 vs 0.55 for the processing stage, but scale efficiency (SE) is high at both stages (0.84 and 0.87), implying that productivity can be improved substantially by reallocation of resources and adjusting operation size. The socio-economic determinants exert differential impacts on PTE and SE at each stage. Overall, education, experience and main occupation as farmer significantly improve SE while subsistence pressure reduces it. Extension contact significantly improves SE at the processing stage but reduces PTE and SE overall. Inverse size-PTE and size-SE relationships exist in cassava production system. In other words, large/medium farms are technically and scale inefficient. Gender gap exists in performance. Male farmers are technically efficient at processing stage but scale inefficient overall. Farmers in northern region are technically efficient. Investments in education, extension services and infrastructure are suggested as policy options to improve the cassava sector in Nigeria.

Modeling Robot Dynamic Performance for Endpoint Force Control

This research aims to understand the fundamental dynamic behavior of servo-controlled machinery in response to various types of sensory feedback. As an example of such a system, we study robot force control, a scheme which promises to greatly expand the capabilities of industrial robots by allowing manipulators to interact with uncertain and dynamic tasks. Dynamic models are developed which allow the effects of actuator dynamics, structural flexibility, and workpiece interaction to be explored in the frequency and time domains. The models are used first to explain the causes of robot force control instability, and then to find methods of improving this performance.

On Probabilistic Strategies for Robot Tasks

Robots must act purposefully and successfully in an uncertain world. Sensory information is inaccurate or noisy, actions may have a range of effects, and the robot's environment is only partially and imprecisely modeled. This thesis introduces active randomization by a robot, both in selecting actions to execute and in focusing on sensory information to interpret, as a basic tool for overcoming uncertainty. An example of randomization is given by the strategy of shaking a bin containing a part in order to orient the part in a desired stable state with some high probability. Another example consists of first using reliable sensory information to bring two parts close together, then relying on short random motions to actually mate the two parts, once the part motions lie below the available sensing resolution. Further examples include tapping parts that are tightly wedged, twirling gears before trying to mesh them, and vibrating parts to facilitate a mating operation.

MARVEL: A System for Recognizing World Locations with Stereo Vision

To use a world model, a mobile robot must be able to determine its own position in the world. To support truly autonomous navigation, I present MARVEL, a system that builds and maintains its own models of world locations and uses these models to recognize its world position from stereo vision input. MARVEL is designed to be robust with respect to input errors and to respond to a gradually changing world by updating its world location models. I present results from real-world tests of the system that demonstrate its reliability. MARVEL fits into a world modeling system under development.