The development of an efficient checkpointing facility exploiting operating systems services of the GENESIS cluster operating system

Recent research efforts of parallel processing on non-dedicated clusters have focused on high execution performance, parallelism management, transparent access to resources, and making clusters easy to use. However, as a collection of independent computers used by multiple users, clusters are susceptible to failure. This paper shows the development of a coordinated checkpointing facility for the GENESIS cluster operating system. This facility was developed by exploiting existing operating system services. High performance and low overheads are achieved by allowing the processes of a parallel application to continue executing during the creation of checkpoints, while maintaining low demands on cluster resources by using coordinated checkpointing.

A cluster operating system supporting parallel computing

The single factor limiting the harnessing of the enormous computing power of clusters for parallel computing is the lack of appropriate software. Present cluster operating systems are not built to support parallel computing – they do not provide services to manage parallelism. The cluster operating environments that are used to assist the execution of parallel applications do not provide support for both Message Passing (MP) or Distributed Shared Memory (DSM) paradigms. They are only offered as separate components implemented at the user level as library and independent servers. Due to poor operating systems users must deal with computers of a cluster rather than to see this cluster as a single powerful computer. A Single System Image of the cluster is not offered to users. There is a need for an operating system for clusters. We claim and demonstrate that it is possible to develop a cluster operating system that is
able to efficiently manage parallelism, support Message Passing and DSM and offer the Single System Image. In order to substantiate the claim the first version of a cluster operating system, called GENESIS, that manages parallelism and offers the Single System Image has been developed.

GENESIS: an efficient, transparent and easy to use cluster operating system

Present operating systems are not built to support parallel computing––they do not provide services to manage parallelism, i.e., to globally manage parallel processes and computational resources. The cluster operating environments that are used to assist the execution of parallel applications do not provide support for both programming paradigms, message passing (MP) or distributed shared memory (DSM)––they are mainly offered as separate components implemented at the user level as library and independent server processes. Due to poor operating systems users must deal with clusters as a set of independent computers rather than to see this cluster as a single powerful computer. A single system image (SSI) of the cluster is not offered to users. There is a need for an operating system for clusters. We claim and demonstrate in this paper that it is possible to develop a cluster operating system that is able to efficiently manage parallelism; use cluster resources efficiently; support MP in the form of standard MP and PVM, and DSM; offer SSI; and make it easy to use. We show that to achieve these aims this operating system should inherit many features of a distributed operating system and provide new services which address the needs of parallel processes, cluster's resources, and application developers. In order to substantiate the claim the first version of a cluster operating system managing parallelism and offering SSI, called GENESIS, has been developed.

Toward an operating system that supports parallel processing on nondedicated clusters

Present operating systems are not built to support parallel computing on clusters - they do not provide services to manage parallelism, i.e., to manage parallel processes and cluster resources. They do not provide support for both programming paradigms, Message Passing (MP) or Distributed Shared Memory (DSM). Due to poor operating systems, users must deal with computers of a cluster rather than to see this cluster as a single powerful computer. There is a need for cluster operating systems. We claim that it is possible to develop a cluster operating system that is able to efficiently manage parallelism, support MP and DSM and offer transparency. To substantiate this claim the first version of a cluster operating system managing parallelism and offering transparency, called GENESIS, has been developed.

Cluster operating system support for parallel autonomic computing

The aim of this paper is to show a general design of autonomic elements and initial implementation of a cluster operating system that moves parallel processing on clusters to the computing mainstream using the autonomic computing vision. The significance of this solution is as follows. Autonomic Computing was identified by IBM as one of computing's Grand Challenges. The human body was used to illustrate an Autonomic Computing system that possesses self-knowledge, self-configuration, self optimization, self-healing, and self-protection, knowledge of its environment and user friendliness properties. One of the areas that could benefit from the comprehensive approach created by the autonomic computing vision is parallel processing on non-dedicated clusters. Many researchers and research groups have responded positively to the challenge by initiating research around one or two of the characteristics identified by IBM as the requirements for autonomic computing. We demonstrate here that it is possible to satisfy all Autonomic Computing characteristics.

Programmer friendly and efficient distributed shared memory integrated into a distributed operating system

Distributed Shared Memory (DSM) provides programmers with a shared memory environment in systems where memory is not physically shared. Clusters of Workstations (COWs), an often untapped source of computing power, are characterised by a very low cost/performance ratio. The combination of Clusters of Workstations (COWs) with DSM provides an environment in which the programmer can use the well known approaches and methods of programming for physically shared memory systems and parallel processing can be carried out to make full use of the computing power and cost advantages of the COW. The aim of this research is to synthesise and develop a distributed shared memory system as an integral part of an operating system in order to provide application programmers with a convenient environment in which the development and execution of parallel applications can be done easily and efficiently, and which does this in a transparent manner. Furthermore, in order to satisfy our challenging design requirements we want to demonstrate that the operating system into which the DSM system is integrated should be a distributed operating system. In this thesis a study into the synthesis of a DSM system within a microkernel and client-server based distributed operating system which uses both strict and weak consistency models, with a write-invalidate and write-update based approach for consistency maintenance is reported. Furthermore a unique automatic initialisation system which allows the programmer to start the parallel execution of a group of processes with a single library call is reported. The number and location of these processes are determined by the operating system based on system load information. The DSM system proposed has a novel approach in that it provides programmers with a complete programming environment in which they are easily able to develop and run their code or indeed run existing shared memory code. A set of demanding DSM system design requirements are presented and the incentives for the placement of the DSM system with a distributed operating system and in particular in the memory management server have been reported. The new DSM system concentrated on an event-driven set of cooperating and distributed entities, and a detailed description of the events and reactions to these events that make up the operation of the DSM system is then presented. This is followed by a pseudocode form of the detailed design of the main modules and activities of the primitives used in the proposed DSM system. Quantitative results of performance tests and qualitative results showing the ease of programming and use of the RHODOS DSM system are reported. A study of five different application is given and the results of tests carried out on these applications together with a discussion of the results are given. A discussion of how RHODOS’ DSM allows programmers to write shared memory code in an easy to use and familiar environment and a comparative evaluation of RHODOS DSM with other DSM systems is presented. In particular, the ease of use and transparency of the DSM system have been demonstrated through the description of the ease with which a moderately inexperienced undergraduate programmer was able to convert, write and run applications for the testing of the DSM system. Furthermore, the description of the tests performed using physically shared memory shows that the latter is indistinguishable from distributed shared memory; this is further evidence that the DSM system is fully transparent. This study clearly demonstrates that the aim of the research has been achieved; it is possible to develop a programmer friendly and efficient DSM system fully integrated within a distributed operating system. It is clear from this research that client-server and microkernel based distributed operating system integrated DSM makes shared memory operations transparent and almost completely removes the involvement of the programmer beyond classical activities needed to deal with shared memory. The conclusion can be drawn that DSM, when implemented within a client-server and microkernel based distributed operating system, is one of the most encouraging approaches to parallel processing since it guarantees performance improvements with minimal programmer involvement.

Monitoring the performance of a microkernel based operating system and supported applications

The message passing microkernel based operating system is a class of operating system in which the policies are implemented using servers whose co-operation is supported by using a small hardware dependent layer called a microkernel. This thesis addresses the issue of reducing the performance deficiency by documenting the synthesis, development, implementation and assessment of a methodology and mechanisms for monitoring the performance of a message passing microkernel based operating system and supported processes. The methodology has been extensively evaluated to ensure that it can be used successfully to monitor performance.

Toolkit for constructing service replication systems

Distributed replication is the key to providing high availability, fault-tolerance, and enhanced performance. The thesis focuses on providing a toolkit to support the automatic construction of reliable distributed service replication systems. The toolkit frees programmers from dealing with network communications and replication control protocols.

Experiences gained from building a services-based distributed operating system

The goal of this paper is to present the experiences gained over 15 years of research into the design and development of a services-based distributed operating system. The lessons learnt over this period, we hope, will be of value to researchers involved in the design and development of operating systems that wish to harness the collective resources of ever-expanding distributed systems.

A performance model of speculative prefetching in distributed information systems

Previous studies in speculative prefetching focus on building and evaluating access models for the purpose of access prediction. This paper investigates a complementary area which has been largely ignored, that of performance modelling. We use improvement in access time as the performance metric, for which we derive a formula in terms of resource parameters (time available and time required for prefetching) and speculative parameters (probabilities for next access). The performance maximization problem is expressed as a stretch knapsack problem. We develop an algorithm to maximize the improvement in access time by solving the stretch knapsack problem, using theoretically proven apparatus to reduce the search space. Integration between speculative prefetching and caching is also investigated, albeit under the assumption of equal item sizes.

PGSW-OS: a novel approach for resource management in a semantic web operating system based on a P2P grid architecture

A web operating system is an operating system that users can access from any hardware at any location. A peer-to-peer (P2P) grid uses P2P communication for resource management and communication between nodes in a grid and manages resources locally in each cluster, and this provides a proper architecture for a web operating system. Use of semantic technology in web operating systems is an emerging field that improves the management and discovery of resources and services. In this paper, we propose PGSW-OS (P2P grid semantic Web OS), a model based on a P2P grid architecture and semantic technology to improve resource management in a web operating system through resource discovery with the aid of semantic features. Our approach integrates distributed hash tables (DHTs) and semantic overlay networks to enable semantic-based resource management by advertising resources in the DHT based upon their annotations to enable semantic-based resource matchmaking. Our model includes ontologies and virtual organizations. Our technique decreases the computational complexity of searching in a web operating system environment. We perform a simulation study using the Gridsim simulator, and our experiments show that our model provides enhanced utilization of resources, better search expressiveness, scalability, and precision. © 2014 Springer Science+Business Media New York.

Toward highly available, self-healing, adaptable, grid-like and user friendly nondedicated clusters

The human body was used to illustrate an Autonomic Computing system that possesses self-knowledge, self-configuration, self-optimization, self-healing, and self-protection, knowledge of its environment and user friendliness properties. Autonomic Computing was identified by IBM as one of the Grand Challenges. Many researchers and research groups have responded positively to the challenge by initiating research around one or two of the characteristics
identified by IBM as the requirements for Autonomic Computing. One of the areas that could benefit from the comprehensive approach created by the Autonomic Computing vision is parallel processing on nondedicated clusters. This paper shows a general design of services and initial implementation of a system that moves parallel processing on clusters to the computing mainstream using the Autonomic Computing vision.

Extracting fingerprint features using textures

Personal identification of individuals is becoming increasingly adopted in society today. Due to the large number of electronic systems that require human identification, faster and more secure identification systems are pursued. Biometrics is based upon the physical characteristics of individuals; of these the fingerprint is the most common as used within law enforcement. Fingerprint-based systems have been introduced into the society but have not been well received due to relatively high rejection rates and false acceptance rates. This limited acceptance of fingerprint identification systems requires new techniques to be investigated to improve this identification method and the acceptance of the technology within society. Electronic fingerprint identification provides a method of identifying an individual within seconds quickly and easily. The fingerprint must be captured instantly to allow the system to identify the individual without any technical user interaction to simplify system operation. The performance of the entire system relies heavily on the quality of the original fingerprint image that is captured digitally. A single fingerprint scan for verification makes it easier for users accessing the system as it replaces the need to remember passwords or authorisation codes. The identification system comprises of several components to perform this function, which includes a fingerprint sensor, processor, feature extraction and verification algorithms. A compact texture feature extraction method will be implemented within an embedded microprocessor-based system for security, performance and cost effective production over currently available commercial fingerprint identification systems. To perform these functions various software packages are available for developing programs for windows-based operating systems but must not constrain to a graphical user interface alone. MATLAB was the software package chosen for this thesis due to its strong mathematical library, data analysis and image analysis libraries and capability. MATLAB enables the complete fingerprint identification system to be developed and implemented within a PC environment and also to be exported at a later date directly to an embedded processing environment. The nucleus of the fingerprint identification system is the feature extraction approach presented in this thesis that uses global texture information unlike traditional local information in minutiae-based identification methods. Commercial solid-state sensors such as the type selected for use in this thesis have a limited contact area with the fingertip and therefore only sample a limited portion of the fingerprint. This limits the number of minutiae that can be extracted from the fingerprint and as such limits the number of common singular points between two impressions of the same fingerprint. The application of texture feature extraction will be tested using variety of fingerprint images to determine the most appropriate format for use within the embedded system. This thesis has focused on designing a fingerprint-based identification system that is highly expandable using the MATLAB environment. The main components that are defined within this thesis are the hardware design, image capture, image processing and feature extraction methods. Selection of the final system components for this electronic fingerprint identification system was determined by using specific criteria to yield the highest performance from an embedded processing environment. These platforms are very cost effective and will allow fingerprint-based identification technology to be implemented in more commercial products that can benefit from the security and simplicity of a fingerprint identification system.

Multiple strategy process migration

The future of computing lies with distributed systems, i.e. a network of workstations controlled by a modern distributed operating system. By supporting load balancing and parallel execution, the overall performance of a distributed system can be improved dramatically. Process migration, the act of moving a running process from a highly loaded machine to a lightly loaded machine, could be used to support load balancing, parallel execution, reliability etc. This thesis identifies the problems past process migration facilities have had and determines the possible differing strategies that can be used to resolve these problems. The result of this analysis has led to a new design philosophy. This philosophy requires the design of a process migration facility and the design of an operating system to be conducted in parallel. Modern distributed operating systems follow the microkernel and client/server paradigms. Applying these design paradigms, in conjunction with the requirements of both process migration and a distributed operating system, results in a system where each resource is controlled by a separate server process. However, a process is a complex resource composed of simple resources such as data structures, an address space and communication state. For this reason, a process migration facility does not directly migrate the resources of a process. Instead, it requests the appropriate servers to transfer the resources. This novel solution yields a modular, high performance facility that is easy to create, debug and maintain. Furthermore, the design easily incorporates providing multiple migration strategies. In order to verify the validity of this design, a process migration facility was developed and tested within RHODOS (ResearcH Oriented Distributed Operating System). RHODOS is a modern microkernel and client/server based distributed operating system. In RHODOS, a process is composed of at least three separate resources: process state - maintained by a process manager, address space - maintained by a memory manager and communication state - maintained by an InterProcess Communication Manager (IPCM). The RHODOS multiple strategy migration manager utilises the services of the process, memory and IPC Managers to migrate the resources of a process. Performance testing of this facility indicates that this design is as fast or better than existing systems which use faster hardware. Furthermore, by studying the results of the performance test ing, the conditions under which a particular strategy should be employed have been identified. This thesis also addresses heterogeneous process migration. The current trend is to have islands of homogeneous workstations amid a sea of heterogeneity. From this situation and the current literature on the topic, heterogeneous process migration can be seen as too inefficient for general use. Instead, only homogeneous workstations should be used for process migration. This implies a need to locate homogeneous workstations. Entities called traders, which store and disseminate knowledge about the resources of several workstations, should be used to provide resource discovery. Resource discovery will enable the detection of homogeneous workstations to which processes can be migrated.