Improving the performance of model-order selection criteria by partial-model selection search

The traditional searching method for model-order selection in linear regression is a nested full-parameters-set searching procedure over the desired orders, which we call full-model order selection. On the other hand, a method for model-selection searches for the best sub-model within each order. In this paper, we propose using the model-selection searching method for model-order selection, which we call partial-model order selection. We show by simulations that the proposed searching method gives better accuracies than the traditional one, especially for low signal-to-noise ratios over a wide range of model-order selection criteria (both information theoretic based and bootstrap-based). Also, we show that for some models the performance of the bootstrap-based criterion improves significantly by using the proposed partial-model selection searching method. Index Terms— Model order estimation, model selection, information theoretic criteria, bootstrap 1. INTRODUCTION Several model-order selection criteria can be applied to find the optimal order. Some of the more commonly used information theoretic-based procedures include Akaike’s information criterion (AIC) [1], corrected Akaike (AICc) [2], minimum description length (MDL) [3], normalized maximum likelihood (NML) [4], Hannan-Quinn criterion (HQC) [5], conditional model-order estimation (CME) [6], and the efficient detection criterion (EDC) [7]. From a practical point of view, it is difficult to decide which model order selection criterion to use. Many of them perform reasonably well when the signal-to-noise ratio (SNR) is high. The discrepancies in their performance, however, become more evident when the SNR is low. In those situations, the performance of the given technique is not only determined by the model structure (say a polynomial trend versus a Fourier series) but, more importantly, by the relative values of the parameters within the model. This makes the comparison between the model-order selection algorithms difficult as within the same model with a given order one could find an example for which one of the methods performs favourably well or fails [6, 8]. Our aim is to improve the performance of the model order selection criteria in cases where the SNR is low by considering a model-selection searching procedure that takes into account not only the full-model order search but also a partial model order search within the given model order. Understandably, the improvement in the performance of the model order estimation is at the expense of additional computational complexity.

Analysis of selection criteria for design-builders through the analysis of Request for Proposals (RFPs)

Selecting an appropriate design-builder is critical to the success of DB projects. The objective of this study is to identify selection criteria for design-builders and compare their relative importance by means of a robust content analysis of 94 Request For Proposals (RFPs) for public DB projects. These DB projects had an aggregate contract value of over US$3.5 billion and were advertised between 2000 and 2010. This study summarized twenty-six selection criteria and classified into ten categories, i.e.: price, experience, technical approach, management approach, qualification, schedule, past performance, financial capability, responsiveness to the RFP, and legal status in descending order of their relative importance. The results showed that even though price still remains as the most important selection category, its relative importance declines significantly in the last decade. The categories of qualification, experience, past performance, by contrast, have been becoming more important to DB owners for selecting design-builders. Finally, it is found that the importance weighting of price in large projects is significantly higher than that in small projects. This study provides a useful reference for owners in selecting their preferred design-builders.

Identification of selection criteria for operational variations of the design-build system : a delphi study in China

The design-build (DB) system has been demonstrated as an effective delivery method and has gained popularity worldwide. However it is observed that a number of operational variations of DB system have emerged since the last decade to cater for different client’s requirements. After the client decides to procure his project through the DB system, he still has to choose an appropriate configuration to deliver their projects optimally. However, there is little research on the selection of DB operational variations. One of the main reasons for this is the lack of evaluation criteria for determining the appropriateness of each operational variation. To obtain such criteria, a three-round Delphi survey has been conducted with 20 construction experts in the People’s Republic of China (PRC). Seven top selection criteria were identified. These are: (1) availability of competent design-builders; (2) client’s capabilities; (3) project complexity; (4) client’s control of project; (5) early commencement & short duration; (6) reduced responsibility or involvement; and (7) clearly defined end user’s requirements. These selection criteria were found to have a statistically significant agreement. These findings may furnish various stakeholders, DB clients in particular, with better insight to understand and compare the different operational variations of the DB system.

Selection criteria in the search for a sperm donor: Behavioural traits versus physical appearance

Despite extensive literature on female mate choice, empirical evidence on women’s mating preferences in the search for a sperm donor is scarce, even though this search, by isolating a male’s genetic impact on offspring from other factors like paternal investment, offers a naturally ”controlled” research setting. In this paper, we work to fill this void by examining the rapidly growing online sperm donor market, which is raising new challenges by offering women novel ways to seek out donor sperm. We not only identify individual factors that influence women’s mating preferences but find strong support for the proposition that behavioural traits (inner values) are more important in these choices than physical appearance (exterior values). We also report evidence that physical factors matter more than resources or other external cues of material success, perhaps because the relevance of good character in donor selection is part of a female psychological adaptation throughout evolutionary history. The lack of evidence on a preference for material resources, on the other hand, may indicate the ability of peer socialization and better access to resources to rapidly shape the female decision process. Overall, the paper makes useful contributions to both the literature on human behaviour and that on decision-making in extreme and highly important situations.

Intra-cluster correlation structure in longitudinal data analysis: Selection criteria and misspecification tests

Selection criteria and misspecification tests for the intra-cluster correlation structure (ICS) in longitudinal data analysis are considered. In particular, the asymptotical distribution of the correlation information criterion (CIC) is derived and a new method for selecting a working ICS is proposed by standardizing the selection criterion as the p-value. The CIC test is found to be powerful in detecting misspecification of the working ICS structures, while with respect to the working ICS selection, the standardized CIC test is also shown to have satisfactory performance. Some simulation studies and applications to two real longitudinal datasets are made to illustrate how these criteria and tests might be useful.

Application of the analytic hierarchy process (AHP) in multi-criteria analysis of the selection of intelligent building systems

The availability of innumerable intelligent building (IB) products, and the current dearth of inclusive building component selection methods suggest that decision makers might be confronted with the quandary of forming a particular combination of components to suit the needs of a specific IB project. Despite this problem, few empirical studies have so far been undertaken to analyse the selection of the IB systems, and to identify key selection criteria for major IB systems. This study is designed to fill these research gaps. Two surveys: a general survey and the analytic hierarchy process (AHP) survey are proposed to achieve these objectives. The first general survey aims to collect general views from IB experts and practitioners to identify the perceived critical selection criteria, while the AHP survey was conducted to prioritize and assign the important weightings for the perceived criteria in the general survey. Results generally suggest that each IB system was determined by a disparate set of selection criteria with different weightings. ‘Work efficiency’ is perceived to be most important core selection criterion for various IB systems, while ‘user comfort’, ‘safety’ and ‘cost effectiveness’ are also considered to be significant. Two sub-criteria, ‘reliability’ and ‘operating and maintenance costs’, are regarded as prime factors to be considered in selecting IB systems. The current study contributes to the industry and IB research in at least two aspects. First, it widens the understanding of the selection criteria, as well as their degree of importance, of the IB systems. It also adopts a multi-criteria AHP approach which is a new method to analyse and select the building systems in IB. Further research would investigate the inter-relationship amongst the selection criteria.

The applicability of biased estimation in model and model order selection

Biased estimation has the advantage of reducing the mean squared error (MSE) of an estimator. The question of interest is how biased estimation affects model selection. In this paper, we introduce biased estimation to a range of model selection criteria. Specifically, we analyze the performance of the minimum description length (MDL) criterion based on biased and unbiased estimation and compare it against modern model selection criteria such as Kay's conditional model order estimator (CME), the bootstrap and the more recently proposed hook-and-loop resampling based model selection. The advantages and limitations of the considered techniques are discussed. The results indicate that, in some cases, biased estimators can slightly improve the selection of the correct model. We also give an example for which the CME with an unbiased estimator fails, but could regain its power when a biased estimator is used.

Construction, application and validation of selection evaluation model (SEM) for intelligent HVAC control system

Design teams are confronted with the quandary of choosing apposite building control systems to suit the needs of particular intelligent building projects, due to the availability of innumerable ‘intelligent’ building products and a dearth of inclusive evaluation tools. This paper is organised to develop a model for facilitating the selection evaluation for intelligent HVAC control systems for commercial intelligent buildings. To achieve these objectives, systematic research activities have been conducted to first develop, test and refine the general conceptual model using consecutive surveys; then, to convert the developed conceptual framework into a practical model; and, finally, to evaluate the effectiveness of the model by means of expert validation. The results of the surveys are that ‘total energy use’ is perceived as the top selection criterion, followed by the‘system reliability and stability’, ‘operating and maintenance costs’, and ‘control of indoor humidity and temperature’. This research not only presents a systematic and structured approach to evaluate candidate intelligent HVAC control system against the critical selection criteria (CSC), but it also suggests a benchmark for the selection of one control system candidate against another.

Selection of project procurement methods in the consumer electronics industry : a Hong Kong study

Under pressure from both the ever increasing level of market competition and the global financial crisis, clients in consumer electronics (CE) industry are keen to understand how to choose the most appropriate procurement method and hence to improve their competitiveness. Four rounds of Delphi questionnaire survey were conducted with 12 experts in order to identify the most appropriate procurement method in the Hong Kong CE industry. Five key selection criteria in the CE industry are highlighted, including product quality, capability, price competition, flexibility and speed. This study also revealed that product quality was found to be the most important criteria for the “First type used commercially” and “Major functional improvements” projects. As for “Minor functional improvements” projects, price competition was the most crucial factor to be considered during the PP selection. These research findings provide owners with useful insights to select the procurement strategies.

Developing a fuzzy multi-criteria decision-making model for selecting design-build operational variations

Many academic researchers have conducted studies on the selection of design-build (DB) delivery method; however, there are few studies on the selection of DB operational variations, which poses challenges to many clients. The selection of DB operational variation is a multi-criteria decision making process that requires clients to objectively evaluate the performance of each DB operational variation with reference to the selection criteria. This evaluation process is often characterized by subjectivity and uncertainty. In order to resolve this deficiency, the current investigation aimed to establish a fuzzy multicriteria decision-making (FMCDM) model for selecting the most suitable DB operational variation. A three-round Delphi questionnaire survey was conducted to identify the selection criteria and their relative importance. A fuzzy set theory approach, namely the modified horizontal approach with the bisector error method, was applied to establish the fuzzy membership functions, which enables clients to perform quantitative calculations on the performance of each DB operational variation. The FMCDM was developed using the weighted mean method to aggregate the overall performance of DB operational variations with regard to the selection criteria. The proposed FMCDM model enables clients to perform quantitative calculations in a fuzzy decision-making environment and provides a useful tool to cope with different project attributes.

Defining sustainability requirements for design-build (DB) contractor selection in public sector projects

The design-build (DB) delivery system is an effective means of delivering a green construction project and selecting an appropriate contractor is critical to project success. Moreover, the delivery of green buildings requires specific design, construction and operation and maintenance considerations not generally encountered in the procurement of conventional buildings. Specifying clear sustainability requirements to potential contractors is particularly important in achieving sustainable project goals. However, many client/owners either do not explicitly specify sustainability requirements or do so in a prescriptive manner during the project procurement process. This paper investigates the current state-of-the-art procurement process used in specifying the sustainability requirements of the public sector in the USA construction market by means of a robust content analysis of 40 design-build requests for proposals (RFPs). The results of the content analysis indicate that the sustainability requirement is one of the most important dimensions in the best-value evaluation of DB contractors. Client/owners predominantly specify the LEED certification levels (e.g. LEED Certified, Silver, Gold, and Platinum) for a particular facility, and include the sustainability requirements as selection criteria (with specific importance weightings) for contractor evolution. Additionally, larger size projects tend to allocate higher importance weightings to sustainability requirements.This study provides public DB client/owners with a number of practical implications for selecting appropriate design-builders for sustainable DB projects.

Design-build contractor selection for public sustainable buildings

Design-build (DB) is regarded as an effective means of delivering high performance green buildings, and the selection of DB contractors is of critical importance. The objective of this study is to evaluate the selection of design-builders for public buildings seeking Leadership in Energy and Environmental Design (LEED) certification and compare the selection practices involved with those of non-LEED-seeking DB projects through a robust content analysis of 74 DB request for proposals (RFPs) for public DB projects. The results of the content analysis reveal that the level of LEED certification is the dominant means of conveying the sustainability requirements in RFPs for contractor selection, with the majority of RFPs (60%) including sustainability requirements as part of the contractor evaluation package. With the exception of contractors' past performance, there is no statistically significant difference in the importance weightings of selection criteria between LEED-seeking and non-LEED-seeking buildings, and DB owners tend to place more emphasis on innovative technical solutions rather than the past performance of DB contractors. Additionally, the research findings also indicate that owners of LEED-seeking building projects tend to provide less design decisions in RFPs in order to solicit innovative design alternatives from potential DB contractors. This study provides DB owners with a number of practical implications for selecting appropriate design-builders for green DB projects.

An exploratory study into director selection : who do directors want on their boards and how do they select them?

This exploratory study into director selection involved in-depth interviews with Australian non-executive directors to identify what directors consider as important criteria when selecting new members and the approach taken to identify and select candidates. The findings indicate boards select new members based not only on their ability to contribute complementary skills and experience but also on a perceived compatibility with incumbent board members. While these two selection criteria are considered equal in importance, not all selection approaches are able to adequately assess both criteria. As a result many selections fail to realise their selection criteria.

A hybrid quality function deployment and cybernetic analytic network process model for project manager selection

Identifying appropriate decision criteria and making optimal decisions in a structured way is a complex process. This paper presents an approach for doing this in the form of a hybrid Quality Function Deployment (QFD) and Cybernetic Analytic Network Process (CANP) model for project manager selection. This involves the use of QFD to translate the owner's project management expectations into selection criteria and the CANP to weight the expectations and selection criteria. The supermatrix approach then prioritises the candidates with respect to the overall decision-making goal. A case study is used to demonstrate the use of the model in selecting a renovation project manager. This involves the development of 18 selection criteria in response to the owner's three main expectations of time, cost and quality.

Rethinking board composition: Balancing competence and compatibility in director selection

Board composition is critical to board effectiveness. Shaping an effective board begins with the selection of directors. While much attention has been paid to the skills and qualifications directors require, there has been less focus on the necessity for board members to interact and work well together. This exploratory study offers insights into what qualities directors look for when selecting new members and the approach adopted to identify and select them. The findings of 10 in-depth interviews with Australian directors suggest new members are selected both on competencies and compatibility. Yet not all selection approaches adequately assess candidates for these two criteria. As a result many appointments fail to realise the selection criteria reducing capacity to reach its full potential.