A Systems-level approach to building sustainable assessment cultures : moderation, quality task design and dependability of judgement

This paper considers the conditions that are necessary at system and local levels for teacher assessment to be valid, reliable and rigorous. With sustainable assessment cultures as a goal, the paper examines how education systems can support local level efforts for quality learning and dependable teacher assessment. This is achieved through discussion of relevant research and consideration of a case study involving an evaluation of a cross-sectoral approach to promoting confidence in school-based assessment in Queensland, Australia. Building on the reported case study, essential characteristics for developing sustainable assessment cultures are presented, including: leadership in learning; alignment of curriculum, pedagogy and assessment; the design of quality assessment tasks and accompanying standards, and evidence-based judgement and moderation. Taken together, these elements constitute a new framework for building assessment capabilities and promoting quality assurance.

Embedding human expert cognition into autonomous UAS trajectory planning

This paper presents a new approach for the inclusion of human expert cognition into autonomous trajectory planning for unmanned aerial systems (UASs) operating in low-altitude environments. During typical UAS operations, multiple objectives may exist; therefore, the use of multicriteria decision aid techniques can potentially allow for convergence to trajectory solutions which better reflect overall mission requirements. In that context, additive multiattribute value theory has been applied to optimize trajectories with respect to multiple objectives. A graphical user interface was developed to allow for knowledge capture from a human decision maker (HDM) through simulated decision scenarios. The expert decision data gathered are converted into value functions and corresponding criteria weightings using utility additive theory. The inclusion of preferences elicited from HDM data within an automated decision system allows for the generation of trajectories which more closely represent the candidate HDM decision preferences. This approach has been demonstrated in this paper through simulation using a fixed-wing UAS operating in low-altitude environments.

A multi-layered approach for site detection in UAS emergency landing scenarios using geometry-based image segmentation

This paper presents an alternative approach to image segmentation by using the spatial distribution of edge pixels as opposed to pixel intensities. The segmentation is achieved by a multi-layered approach and is intended to find suitable landing areas for an aircraft emergency landing. We combine standard techniques (edge detectors) with novel developed algorithms (line expansion and geometry test) to design an original segmentation algorithm. Our approach removes the dependency on environmental factors that traditionally influence lighting conditions, which in turn have negative impact on pixel-based segmentation techniques. We present test outcomes on realistic visual data collected from an aircraft, reporting on preliminary feedback about the performance of the detection. We demonstrate consistent performances over 97% detection rate.

Nonlinear actuator fault detection for small-scale UAS

This paper presents a recursive strategy for online detection of actuator faults on a unmanned aerial system (UAS) subjected to accidental actuator faults. The proposed detection algorithm aims to provide a UAS with the capability of identifying and determining characteristics of actuator faults, offering necessary flight information for the design of fault-tolerant mechanism to compensate for the resultant side-effect when faults occur. The proposed fault detection strategy consists of a bank of unscented Kalman filters (UKFs) with each one detecting a specific type of actuator faults and estimating corresponding velocity and attitude information. Performance of the proposed method is evaluated using a typical nonlinear UAS model and it is demonstrated in simulations that our method is able to detect representative faults with a sufficient accuracy and acceptable time delay, and can be applied to the design of fault-tolerant flight control systems of UASs.

Risk-management of UAS robust autonomy for its integration into civil aviation safety frameworks

This paper discusses a model of the civil aviation reg- ulation framework and shows how the current assess- ment of reliability and risk for piloted aircraft has limited applicability for Unmanned Aircraft Systems (UAS) with high levels of autonomous decision mak- ing. Then, a new framework for risk management of robust autonomy is proposed, which arises from combining quantified measures of risk with normative decision making. The term Robust Autonomy de- scribes the ability of an autonomous system to either continue or abort its operation whilst not breaching a minimum level of acceptable safety in the presence of anomalous conditions. The decision making associ- ated with risk management requires quantifying prob- abilities associated with the measures of risk and also consequences of outcomes related to the behaviour of autonomy. The probabilities are computed from an assessment under both nominal and anomalous sce- narios described by faults, which can be associated with the aircraft’s actuators, sensors, communication link, changes in dynamics, and the presence of other aircraft in the operational space. The consequences of outcomes are characterised by a loss function which rewards the certification decision

Evaluation of robust autonomy and implications on UAS certification and design

As the number of potential applications of Unmanned Aircraft Systems (UAS) grows in civilian operations and national security, National Airworthiness Authorities are under increasing pressure to provide a path for certification and allow UAS integration into the national airspace. The success of this integration depends on developments in improved UAS reliability and safety, regulations for certification, and technologies for operational performance and safety assessment. This paper focusses on the latter and describes the use of a framework for evaluating robust autonomy of UAS, namely, the autonomous system’s ability to either continue operation in the presence of faults or safely shut down. The paper draws parallels between the proposed evaluation framework and the evaluation of pilots during the licensing process. It also discusses how the data from the proposed evaluation can be uses as an aid for decision making in certification and UAS designs.

Computational aspects of probabilistic assessment of UAS robust autonomy

As the level of autonomy in Unmanned Aircraft Systems (UAS) increases, there is an imperative need for developing methods to assess robust autonomy. This paper focuses on the computations that lead to a set of measures of robust autonomy. These measures are the probabilities that selected performance indices related to the mission requirements and airframe capabilities remain within regions of acceptable performance.

A framework for testing robust autonomy of UAS during design and certification

As the number of Uninhabited Airborne Systems (UAS) proliferates in civil applications, industry is increasingly putting pressure on regulation authorities to provide a path for certification and allow UAS integration into regulated airspace. The success of this integration depends on developments in improved UAS reliability and safety, regulations for certification, and technologies for operational performance and safety assessment. This paper focusses on the last topic and describes a framework for quantifying robust autonomy of UAS, which quantifies the system's ability to either continue operating in the presence of faults or safely shut down. Two figures of merit are used to evaluate vehicle performance relative to mission requirements and the consequences of autonomous decision making in motion control and guidance systems. These figures of merit are interpreted within a probabilistic framework, which extends previous work in the literature. The valuation of the figures of merit can be done using stochastic simulation scenarios during both vehicle development and certification stages with different degrees of integration of hardware-in-the-loop simulation technology. The objective of the proposed framework is to aid in decision making about the suitability of a vehicle with respect to safety and reliability relative to mission requirements.

Joint motion control and control allocation design for UAS flight control systems

As Unmanned Aircraft Systems (UAS) grow in complexity, and their level of autonomy increases|moving away from the concept of a remotely piloted systems and more towards autonomous systems|there is a need to further improve reliability and tolerance to faults. The traditional way to accommodate actuator faults is by using standard control allocation techniques as part of the flight control system. The allocation problem in the presence of faults often requires adding constraints that quantify the maximum capacity of the actuators. This in turn requires on-line numerical optimisation. In this paper, we propose a framework for joint allocation and constrained control scheme via vector input scaling. The actuator configuration is used to map actuator constraints into the space of the aircraft generalised forces, which are the magnitudes demanded by the light controller. Then by constraining the output of controller, we ensure that the allocation function always receive feasible demands. With the proposed framework, the allocation problem does not require numerical optimisation, and since the controller handles the constraints, there is not need to implement heuristics to inform the controller about actuator saturation.

Building the safety case for UAS operations in support of natural disaster response

A framework supporting the systematic development of safety cases for Unmanned Aircraft System (UAS) operations in a broad range of civil and commercial applications is presented. The case study application is the use of UAS for disaster response. In those States where regulations do not preclude UAS operations altogether, approvals for UAS operations can be granted on a case-by-case basis contingent on the provision of a safety case acceptable to the relevant National Airworthiness Authority (NAA). A safety case for UAS operations must show how the risks associated with the hazards have been managed to an acceptable level. The foundational components necessary for structuring and assessing these safety cases have not yet been proposed. Barrier-bow-tie models are used in this paper to structure the safety case for the two primary hazards of 1) a ground impact, and 2) a Mid-Air Collision (MAC). The models establish the set of Risk Control Variables (RCVs) available to reduce the risk. For the ground-impact risk model, seven RCVs are identified which in combination govern the probability of an accident. Similarly, ten RCVs are identified within the MAC model. The effectiveness of the RCVs and how they can implemented in terms of processes, policies, devices, practices, or other actions for each of the case-study applications are discussed. The framework presented can provide for the more systematic and consistent regulation of UAS through a "safety target" approach.

Experiments and experiences with dependability for a large autonomous industrial vehicle

This paper describes the experiences gained performing multiple experiments while developing a large autonomous industrial vehicle. Hot Metal Carriers (HMCs) are large forklift-type vehicles used in the light metals industry to move molten or hot metal around a smelter. Autonomous vehicles of this type must be dependable as they are large and potentially hazardous to infrastructure and people. This paper will talk about four aspects of dependability, that of safety, reliability, availability and security and how they have been addressed on our experimental autonomous HMC.

A Survey Of Software Dependability

This paper presents on overview of the issues in precisely defining, specifying and evaluating the dependability of software, particularly in the context of computer controlled process systems. Dependability is intended to be a generic term embodying various quality factors and is useful for both software and hardware. While the developments in quality assurance and reliability theories have proceeded mostly in independent directions for hardware and software systems, we present here the case for developing a unified framework of dependability—a facet of operational effectiveness of modern technological systems, and develop a hierarchical systems model helpful in clarifying this view. In the second half of the paper, we survey the models and methods available for measuring and improving software reliability. The nature of software “bugs”, the failure history of the software system in the various phases of its lifecycle, the reliability growth in the development phase, estimation of the number of errors remaining in the operational phase, and the complexity of the debugging process have all been considered to varying degrees of detail. We also discuss the notion of software fault-tolerance, methods of achieving the same, and the status of other measures of software dependability such as maintainability, availability and safety.

Reliability, Availability, Dependability and Performability: A User-centered View

Reliability and availability have long been considered twin system properties that could be enhanced by distribution. Paradoxically, the traditional definitions of these properties do not recognize the positive impact of recovery as distinct from simple repair and restart on reliability, nor the negative effect of recovery, and of internetworking of clients and servers, on availability. As a result of employing the standard definitions, reliability would tend to be underestimated, and availability overestimated. We offer revised definitions of these two critical metrics, which we call service reliability and service availability, that improve the match between their formal expression, and intuitive meaning. A fortuitous advantage of our approach is that the product of our two metrics yields a highly meaningful figure of merit for the overall dependability of a system. But techniques that enhance system dependability exact a performance cost, so we conclude with a cohesive definition of performability that rewards the system for performance that is delivered to its client applications, after discounting the following consequences of failure: service denial and interruption, lost work, and recovery cost.

Improving MTM-UAS to predetermine automotive maintenance times

The preventive knowledge of serviceability times is a critical factor for the quantification of after-sales services costs of a vehicle. Predetermined motion time system are frequently used to set labor rates in industry by quantifying the amount of time required to perform specific tasks. The first such system is known as Methods-time measurement (MTM). Several variants of MTM have been developed differing from each other on their level of focus. Among them MTM-UAS is suitable for processes that average around 1-3 min. However experimental tests carried out by the authors in Elasis (Research Center of FIAT Group) demonstrate that MTM-UAS is not the optimal approach to measure serviceability times. The reason is that it doesn't take into account ergonomic factors. In the present paper the authors propose to correct the MTM-UAS method including in the task analysis the study of human postures and efforts. The proposed approach allows to estimate with an "acceptable" error the time needed to perform maintenance tasks since the first phases of product design, by working on Digital Mock-up and human models in virtual environment. As a byproduct of that analysis, it is possible to obtain a list of maintenance times in order to preventively set after-sales service costs. © 2012 Springer-Verlag.

Quantifying Submerged Fluvial Topography Using Hyperspatial Resolution UAS Imagery and Structure From Motion Photogrammetry.

Quantifying the topography of rivers and their associated bedforms has been a fundamental concern of fluvial geomorphology for decades. Such data, acquired at high temporal and spatial resolutions, are increasingly in demand for process-oriented investigations of flow hydraulics, sediment dynamics and in-stream habitat. In these riverine environments, the most challenging region for topographic measurement is the wetted, submerged channel. Generally, dry bed topography and submerged bathymetry are measured using different methods and technology. This adds to the costs, logistical challenges and data processing requirements of comprehensive river surveys. However, some technologies are capable of measuring the submerged topography. Through-water photogrammetry and bathymetric LiDAR are capable of reasonably accurate measurements of channel beds in clear water. Whilst the cost of bathymetric LiDAR remains high and its resolution relatively coarse, the recent developments in photogrammetry using Structure from Motion (SfM) algorithms promise a fundamental shift in the accessibility of topographic data for a wide range of settings. Here we present results demonstrating the potential of so called SfM-photogrammetry for quantifying both exposed and submerged fluvial topography at the mesohabitat scale. We show that imagery acquired from a rotary-winged Unmanned Aerial System (UAS) can be processed in order to produce digital elevation models (DEMs) with hyperspatial resolutions (c. 0.02 m) for two different river systems over channel lengths of 50-100 m. Errors in submerged areas range from 0.016 m to 0.089 m, which can be reduced to between 0.008 m and 0.053 m with the application of a simple refraction correction. This work therefore demonstrates the potential of UAS platforms and SfM-photogrammetry as a single technique for surveying fluvial topography at the mesoscale (defined as lengths of channel from c.10 m to a few hundred metres). This article is protected by copyright. All rights reserved.