Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Operational Flexibility: A Model for Effective Project Management

The aim of this study is to investigate the role of operational flexibility for effective project management in the construction industry. The specific objectives are to: a) Identify the determinants of operational flexibility potential in construction project management b) Investigate the contribution of each of the determinants to operational flexibility potential in the construction industry c) Investigate on the moderating factors of operational flexibility potential in a construction project environment d) Investigate whether moderated operational flexibility potential mediates the path between predictors and effective construction project management e) Develop and test a conceptual model of achieving operational flexibility for effective project management The purpose of this study is to findout ways to utilize flexibility inorder to manage uncertain project environment and ultimately achieve effective project management. In what configuration these operational flexibility determinants are demanded by construction project environment in order to achieve project success. This research was conducted in three phases, namely: (i) exploratory phase (ii) questionnaire development phase; and (iii) data collection and analysis phase. The study needs firm level analysis and therefore real estate developers who are members of CREDAI, Kerala Chapter were considered. This study provides a framework on the functioning of operational flexibility, offering guidance to researchers and practitioners for discovering means to gain operational flexibility in construction firms. The findings provide an empirical understanding on kinds of resources and capabilities a construction firm must accumulate to respond flexibly to the changing project environment offering practitioners insights into practices that build firms operational flexibility potential. Firms are dealing with complex, continuous changing and uncertain environments due trends of globalization, technical changes and innovations and changes in the customers’ needs and expectations. To cope with the increasingly uncertain and quickly changing environment firms strive for flexibility. To achieve the level of flexibility that adds value to the customers, firms should look to flexibility from a day to day operational perspective. Each dimension of operational flexibility is derived from competences and capabilities. In this thesis only the influence on customer satisfaction and learning exploitation of flexibility dimensions which directly add value in the customers eyes are studied to answer the followingresearch questions: “What is the impact of operational flexibility on customer satisfaction?.” What are the predictors of operational flexibility in construction industry? .These questions can only be answered after answering the questions like “Why do firms need operational flexibility?” and “how can firms achieve operational flexibility?” in the context of the construction industry. The need for construction firms to be flexible, via the effective utilization of organizational resources and capabilities for improved responsiveness, is important because of the increasing rate of changes in the business environment within which they operate. Achieving operational flexibility is also important because it has a significant correlation with a project effectiveness and hence a firm’s turnover. It is essential for academics and practitioners to recognize that the attainment of operational flexibility involves different types namely: (i) Modification (ii) new product development and (iii) demand management requires different configurations of predictors (i.e., resources, capabilities and strategies). Construction firms should consider these relationships and implement appropriate management practices for developing and configuring the right kind of resources, capabilities and strategies towards achieving different operational flexibility types.