Evaluation of design-builder qualifications through the analysis of Requests for Qualifications (RFQs)

The two-phrase best-value process has been widely used by public agencies for Design and Build (DB) procurement, with a key issue in the first phase of pre-qualification being the determination of evaluation criteria. This study identified a set of general qualification criteria for design-builders and compares their relative importance by a thorough content analysis of 97 Requests for Qualification (RFQ) for public DB projects advertised between 2000 and 2011 in various regions of the USA. The thirty-nine qualification criteria found are summarized and classified into eight categories comprising: experience; project understanding and approach; organizational structure and capacity; past performance record; professional qualifications; responsiveness to RFQs, office location and familiarity with local environment; and legal status in descending order of their relative importance. A comparative analysis of different types of projects shows that the relative weightings of the qualification criteria vary according to different characteristics of the DB projects involved.

The design science research roadmap : in progress evaluation

Although Design Science Research (DSR) is now an accepted approach to research in the Information Systems (IS) discipline, consensus on the methodology of DSR has yet to be achieved. Lack of a comprehensive and detailed methodology for Design Science Research (DSR) in the Information System (IS) discipline is a main issue. Prior research (the parent-study) aimed to remedy this situation and resulted in the DSR-Roadmap (Alturki et al., 2011a). Continuing empirical validation and revision of the DSR-Roadmap strives towards a methodology with appropriate levels of detail, integration, and completeness for novice researchers to efficiently and effectively conduct and report DSR in IS. The sub-study reported herein contributes to this larger, ongoing effort. This paper reports results from a formative evaluation effort of the DSR-Roadmap conducted using focus group analysis. Generally, participants endorsed the utility and intuitiveness of the DSR-Roadmap, while also suggesting valuable refinements. Both parent-study and sub-study make methodological contributions. The parent-study is the first attempt of utilizing DSR to develop a research methodology showing an example of how to use DSR in research methodology construction. The sub-study demonstrates the value of the focus group method in DSR for formative product evaluation.

Design, implementation and multisite evaluation of a system suitability protocol for the quantitative assessment of instrument performance in liquid chromatography-multiple reaction monitoring-MS (LC-MRM-MS)

Multiple reaction monitoring (MRM) mass spectrometry coupled with stable isotope dilution (SID) and liquid chromatography (LC) is increasingly used in biological and clinical studies for precise and reproducible quantification of peptides and proteins in complex sample matrices. Robust LC-SID-MRM-MS-based assays that can be replicated across laboratories and ultimately in clinical laboratory settings require standardized protocols to demonstrate that the analysis platforms are performing adequately. We developed a system suitability protocol (SSP), which employs a predigested mixture of six proteins, to facilitate performance evaluation of LC-SID-MRM-MS instrument platforms, configured with nanoflow-LC systems interfaced to triple quadrupole mass spectrometers. The SSP was designed for use with low multiplex analyses as well as high multiplex approaches when software-driven scheduling of data acquisition is required. Performance was assessed by monitoring of a range of chromatographic and mass spectrometric metrics including peak width, chromatographic resolution, peak capacity, and the variability in peak area and analyte retention time (RT) stability. The SSP, which was evaluated in 11 laboratories on a total of 15 different instruments, enabled early diagnoses of LC and MS anomalies that indicated suboptimal LC-MRM-MS performance. The observed range in variation of each of the metrics scrutinized serves to define the criteria for optimized LC-SID-MRM-MS platforms for routine use, with pass/fail criteria for system suitability performance measures defined as peak area coefficient of variation <0.15, peak width coefficient of variation <0.15, standard deviation of RT <0.15 min (9 s), and the RT drift <0.5min (30 s). The deleterious effect of a marginally performing LC-SID-MRM-MS system on the limit of quantification (LOQ) in targeted quantitative assays illustrates the use and need for a SSP to establish robust and reliable system performance. Use of a SSP helps to ensure that analyte quantification measurements can be replicated with good precision within and across multiple laboratories and should facilitate more widespread use of MRM-MS technology by the basic biomedical and clinical laboratory research communities.

Evaluation of robust autonomy and implications on UAS certification and design

As the number of potential applications of Unmanned Aircraft Systems (UAS) grows in civilian operations and national security, National Airworthiness Authorities are under increasing pressure to provide a path for certification and allow UAS integration into the national airspace. The success of this integration depends on developments in improved UAS reliability and safety, regulations for certification, and technologies for operational performance and safety assessment. This paper focusses on the latter and describes the use of a framework for evaluating robust autonomy of UAS, namely, the autonomous system’s ability to either continue operation in the presence of faults or safely shut down. The paper draws parallels between the proposed evaluation framework and the evaluation of pilots during the licensing process. It also discusses how the data from the proposed evaluation can be uses as an aid for decision making in certification and UAS designs.

The capture and retention evaluation of a stormwater gross pollutant trap design

Gross pollutant traps (GPT) are designed to capture and retain visible street waste, such as anthropogenic litter and organic matter. Blocked screens, low/high downstream tidal waters and flows operating above/below the intended design limits can hamper the operations of a stormwater GPT. Under these adverse operational conditions, a recently developed GPT was evaluated. Capture and retention experiments were conducted on a 50% scale model with partially and fully blocked screens, placed inside a hydraulic flume. Flows were established through the model via an upstream channel-inlet configuration. Floatable, partially buoyant, neutrally buoyant and sinkable spheres were released into the GPT and monitored at the outlet. These experiments were repeated with a pipe-inlet configured GPT. The key findings from the experiments were of practical significance to the design, operation and maintenance of GPTs. These involved an optimum range of screen blockages and a potentially improved inlet design for efficient gross pollutant capture/retention operations. For example, the outlet data showed that the capture and retention efficiency deteriorated rapidly when the screens were fully blocked. The low pressure drop across the retaining screens and the reduced inlet flow velocities were either insufficient to mobilise the gross pollutants, or the GPT became congested.

Design, construction and evaluation of systems to predict risk in obstetrics

We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy. We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy.

The Lymphedema Evaluation in Gynecological cancer Study (LEGS): Design of a prospective, longitudinal, cohort study

Background The Lymphoedema Evaluation in Gynecological cancer Study (LEGS) was a longitudinal, observational, cohort study prospectively evaluating the incidence and risk factors of lower-limb lymphedema after treatment for gynecological cancer. Here we describe the study protocol and characteristics of the sample. Methods Women with a newly diagnosed gynecological cancer between June 1, 2008 and February 28, 2011, aged 18 years or older, and treated at one of six hospitals in Queensland, Australia, were eligible. Lymphedema was assessed by circumference measurements, bioimpedance spectroscopy, and self-reported swelling. LEGS incorporated a cohort of patients requiring surgery for benign gynecological conditions for comparison purposes. Data were collected prior to surgery and at regular intervals thereafter up to 2-years post-diagnosis. Results 546 women participated (408 cancer, 138 benign), with a 24-month retention rate of 78%. Clinical and treatment characteristics of participants were similar to the Queensland gynecological cancer population, except for a higher proportion of early-stage cervical cancers recruited to LEGS compared with Queensland proportions (89% versus 55%, respectively). Discussion Few imbalances were observed between participants with complete and incomplete follow-up data. The prospective design and collection of objective and patient-reported outcome data will allow comprehensive assessment of incidence and risk factors of lower-limb lymphedema.

Design and evaluation of intentionally enriched awareness

In this chapter we introduce and explore the notion of “intentionally enriched awareness”. Intentional enrichment refers to the process of actively engaging users in the awareness process by enabling them to express intentions. We initially look at the phenomenon if sharing intentional information in related collaborative systems. We then explore the concept of intentional enrichment through designing and evaluating the AnyBiff system which allows users to freely create, share and use a variety of biff applications. Biffs are simple representation of pre-defined activities. Users can select biffs to indicate that they are engaged in an activity. We summarise the results of a trial which allowed us to gain insights into the potential of the AnyBiff prototype and the underlying biff concept to implement intentionally enriched awareness. Our findings show that intentional disclosure mechanisms in the form of biffs were successfully used in a variety of contexts. Users actively engaged in the design of a large variety of biffs and explored many different uses of the concept. The study revealed a whole host of issues with regard to intentionally enriched awareness which give valuable insight into the conception and design of future applications in this area.

Design and evaluation of decision and control strategies for autonomous vision-based see and avoid systems

This paper details the design and performance assessment of a unique collision avoidance decision and control strategy for autonomous vision-based See and Avoid systems. The general approach revolves around re-positioning a collision object in the image using image-based visual servoing, without estimating range or time to collision. The decision strategy thus involves determining where to move the collision object, to induce a safe avoidance manuever, and when to cease the avoidance behaviour. These tasks are accomplished by exploiting human navigation models, spiral motion properties, expected image feature uncertainty and the rules of the air. The result is a simple threshold based system that can be tuned and statistically evaluated by extending performance assessment techniques derived for alerting systems. Our results demonstrate how autonomous vision-only See and Avoid systems may be designed under realistic problem constraints, and then evaluated in a manner consistent to aviation expectations.

The Step approach to Message Design and Testing (SatMDT): A conceptual framework to guide the development and evaluation of persuasive health messages

This paper provides an important and timely overview of a conceptual framework designed to assist with the development of message content, as well as the evaluation, of persuasive health messages. While an earlier version of this framework was presented in a prior publication by the authors in 2009, important refinements to the framework have seen it evolve in recent years, warranting the need for an updated review. This paper outlines the Step approach to Message Design and Testing (or SatMDT) in accordance with the theoretical evidence which underpins, as well as empirical evidence which demonstrates the relevance and feasibility, of each of the framework’s steps. The development and testing of the framework have thus far been based exclusively within the road safety advertising context; however, the view expressed herein is that the framework may have broader appeal and application to the health persuasion context.

Oxovanadium(IV)-based near-IR PDT agents: design to biological evaluation

An oxovanadium(IV) complex of dipyridophenazine, as a potent metal-based PDT agent, shows efficient DNA photocleavage activity at near-IR region and high photocytotoxicity in both UV-A and visible light in HeLa cells.

Reflecting on Reggio: An evaluation of design intent in an early childhood learning environment

Reggio Emilia is an educational philosophy that encourages teachers, students and their parents to collaborate and actively engage with the environment. This study investigates how the Reggio Emilia design approach was translated architecturally for a kindergarten in an Australian context, and provides insights into the operation of this Reggio kindergarten and the impact that it is now having on the occupants. It evaluates the original architectural design intent of the Reggio Emilia early childhood learning environment against its spatial provision. The relationship that the Reggio Emilia approach facilitates between students and the environment, and the contribution that this approach has on their learning, are also explored. Several key themes emerging from the Reggio values were identified in the literature. These were then used to inform an exploration of the kindergarten spaces and places.. Architects, teachers and a sustainability manager of the kindergarten were interviewed with their experiences constituting the primary data of this study. Using a Grounded Theory methodology, systematic data coding and analysis were then conducted. Themes and concepts that emerged from this process include: differing interpretations of the Reggio Emilia philosophy; motivations for neglect of traditional external structures and play equipment; the impact of education for sustainability; and the positive effects that Reggio Emilia is having on the rest of the institution’s development.