Tests examining skill outcomes in sport: A systematic review of measurement properties and feasibility

Background
A high level of participant skill is influential in determining the outcome of many sports. Thus, tests assessing skill outcomes in sport are commonly used by coaches and researchers to estimate an athlete’s ability level, to evaluate the effectiveness of interventions or for the purpose of talent identification.

Objective
The objective of this systematic review was to examine the methodological quality, measurement properties and feasibility characteristics of sporting skill outcome tests reported in the peer-reviewed literature.

Data Sources
A search of both SPORTDiscus and MEDLINE databases was undertaken.

Study Selection
Studies that examined tests of sporting skill outcomes were reviewed. Only studies that investigated measurement properties of the test (reliability or validity) were included. A total of 22 studies met the inclusion/exclusion criteria.

Study Appraisal and Synthesis Methods
A customised checklist of assessment criteria, based on previous research, was utilised for the purpose of this review.

Results
A range of sports were the subject of the 22 studies included in this review, with considerations relating to methodological quality being generally well addressed by authors. A range of methods and statistical procedures were used by researchers to determine the measurement properties of their skill outcome tests. The majority (95 %) of the reviewed studies investigated test–retest reliability, and where relevant, inter and intra-rater reliability was also determined. Content validity was examined in 68 % of the studies, with most tests investigating multiple skill domains relevant to the sport. Only 18 % of studies assessed all three reviewed forms of validity (content, construct and criterion), with just 14 % investigating the predictive validity of the test. Test responsiveness was reported in only 9 % of studies, whilst feasibility received varying levels of attention.

Limitations
In organised sport, further tests may exist which have not been investigated in this review. This could be due to such tests firstly not being published in the peer-review literature and secondly, not having their measurement properties (i.e., reliability or validity) examined formally.

Conclusions
Of the 22 studies included in this review, items relating to test methodological quality were, on the whole, well addressed. Test–retest reliability was determined in all but one of the reviewed studies, whilst most studies investigated at least two aspects of validity (i.e., content, construct or criterion-related validity). Few studies examined predictive validity or responsiveness. While feasibility was addressed in over half of the studies, practicality and test limitations were rarely addressed. Consideration of study quality, measurement properties and feasibility components assessed in this review can assist future researchers when developing or modifying tests of sporting skill outcomes.

A critical review of psychological instruments used in police officer selection

Purpose
The purpose of this paper is to critically examine the instruments used in the screening process, with particular attention given to supporting research validation. Psychological screening is a well-established process used in the selection of employees across public safety industries, particularly in police settings. Screening in and screening out are both possible, with screening out being the most commonly used method. Little attention, however, has been given to evaluating the comparative validities of the instruments used.

Design/methodology/approach
This review investigates literature supporting the use of the Minnesota Multiphasic Personality Inventory (MMPI), the California Personality Inventory (CPI), the Inwald Personality Inventory (IPI), the Australian Institute of Forensic Psychology's test battery (AIFP), and some other less researched tests. Research supporting the validity of each test is discussed.

Findings
It was found that no test possesses unequivocal research support, although the CPI and AIFP tests show promise. Most formal research into the validity of the instruments lacks appropriate experimental structure and is therefore less powerful as “evidence” of the utility of the instrument(s).

Practical implications
This research raises the notion that many current screening practices are likely to be adding minimal value to the selection process by way of using instruments that are not “cut out” for the job. This has implications for policy and practice at the recruitment stage of police employment.

Originality/value
This research provides a critical overview of the instruments and their validity studies rather than examining the general process of psychological screening. As such, it is useful to those working in selection who are facing the choice of psychological instrument. Possibilities for future research are presented, and development opportunities for a best practice instrument are discussed.

Effectiveness and feasibility of virtual reality and gaming system use at home by older adults for enabling physical activity to improve health-related domains: a systematic review

Background
use of virtual reality and commercial gaming systems (VR/gaming) at home by older adults is receiving attention as a means of enabling physical activity.

Objective
to summarise evidence for the effectiveness and feasibility of VR/gaming system utilisation by older adults at home for enabling physical activity to improve impairments, activity limitations or participation.

Methods
a systematic review searching 12 electronic databases from 1 January 2000–10 July 2012 using key search terms. Two independent reviewers screened yield articles using pre-determined selection criteria, extracted data using customised forms and applied the Cochrane Collaboration Risk of Bias Tool and the Downs and Black Checklist to rate study quality.

Results
fourteen studies investigating the effects of VR/gaming system use by healthy older adults and people with neurological conditions on activity limitations, body functions and physical impairments and cognitive and emotional well-being met the selection criteria. Study quality ratings were low and, therefore, evidence was not strong enough to conclude that interventions were effective. Feasibility was inconsistently reported in studies. Where feasibility was discussed, strong retention (≥70%) and adherence (≥64%) was reported. Initial assistance to use the technologies, and the need for monitoring exertion, aggravation of musculoskeletal symptoms and falls risk were reported.

Conclusions
existing evidence to support the feasibility and effectiveness VR/gaming systems use by older adults at home to enable physical activity to address impairments, activity limitations and participation is weak with a high risk of bias. The findings of this review may inform future, more rigorous research.

Patterns and perceptions of physical activity and sedentary time in male transport drivers working in regional Australia

 Abstract

Objective: To objectively measure physical activity (PA) patterns and sedentary time, and explore perceptions of workplace PA opportunities in regional male transport workers.

Methods: A multi-method study involving 28 drivers (52.4±9.69years) working at a bus company in South-East Queensland, Australia. PA was measured using accelerometers (n=23) to determine the proportion of time spent in sedentary (<150 cpm), light (151–2,689 cpm) and moderate+ (≥2,690 cpm) intensity categories. Paired sample t-tests were used to evaluate differences between categories on a workday/off-workday (n=16), and during work/non-work time (n=15). Interviews were conducted with 28 drivers and six managers to explore perceptions and ideas relating to workplace PA opportunities.

Results: Sedentary time was significantly higher on off-work (64% of wear time) than work (52%) days (p<0.05), while the opposite was the case for light intensity time (off-workday=33%; workday=44%; p<0.05). On workdays, sedentary time was significantly lower when employees were working (44%) than when not working (60%; p<0.05). No significant differences were found for time spent in moderate+ PA. Driver perceptions indicated that PA opportunities (walking club and corporate gym membership) were being adopted by some drivers. However, at this depot, perceived health issues and organisational barriers (shift work and irregular driving routines), tended to preclude some drivers from engaging with these opportunities.

Conclusions: Findings contest the notion that a sedentary occupation such as driving necessitates an inactive work environment.

Implications: This research informs ongoing intervention efforts to target inactive drivers who are struggling to take advantage of existing workplace-related PA opportunities.

Evaluation of Actical equations and thresholds to predict physical activity intensity in young children

This study examined the validity of current Actical activity energy expenditure (AEE) equations and intensity cut-points in preschoolers using AEE and direct observation as criterion measures. Forty 4–6-year-olds (5.3 ± 1.0 years) completed a ~150-min room calorimeter protocol involving age-appropriate sedentary behaviours (SBs), light intensity physical activities (LPAs) and moderate-to-vigorous intensity physical activities (MVPAs). AEE and/or physical activity intensity were calculated using Actical equations and cut-points by Adolph, Evenson, Pfeiffer and Puyau. Predictive validity was examined using paired sample t-tests. Classification accuracy was evaluated using weighted kappas, sensitivity, specificity and area under the receiver operating characteristic curve. The Pfeiffer equation significantly overestimated AEE during SB and underestimated AEE during LPA (P < 0.0125 for both). There was no significant difference between measured and predicted AEEs during MVPA. The Adolph cut-point showed significantly higher accuracy for classifying SB, LPA and MVPA than all others. The available Actical equation does not provide accurate estimates of AEE across all intensities in preschoolers. However, the Pfeiffer equation performed reasonably well for MVPA. Using cut-points of ≤6 counts · 15 s−1, 7–286 counts · 15 s−1 and ≥ 287 counts · 15 s−1 when classifying SB, LPA and MVPA, respectively, is recommended.

Lessons from silkworm cocoons for future protective materials design

 Evolved over millions of years’ natural selection, very thin and lightweight wild silkworm cocoons can protect silkworms from environmental hazards and physical attacks from predators while supporting their metabolic activity. The knowledge of structure-property-function relationship of multi-layered composite silk cocoon shells gives insight into the design of next-generation protection materials. The mechanical and thermal insulation properties of both domestic (Bombyx mori, or B. moriand Samia. cynthia, or S. cynthia) and wild (Antheraea pernyi and Antheraea mylitta, or A. pernyi and A. mylitta) silkworm cocoons were investigated. The research findings are of relevance to the bio-inspired design of new protective materials and structures.
The 180 degree peel tests and needle penetration tests were used for examining the peel resistance and needle penetration resistance of both domestic and wild silkworm cocoon walls. The temperatures inside and outside of the whole silkworm cocoons under warm, cold and windy conditions were monitored for investigating the cocoon’s thermal insulation function. Computational fluid dynamics (CFD) models were created to simulate the heat transfer through the A. pernyi cocoon wall.
The wild cocoons experienced much higher peeling peak loads than the domestic cocoon. This transfers to a maximum work-of-fracture (WOF) of about 1000 J/m2 from the A. pernyi outer layer, which was 10 times of the B. mori cocoon. The A. pernyi wild cocoon exhibited a maximum penetration force (11 N) that is 70 % higher than a woven aramid fabric. Silk sericin is shown to play a critical role in providing needle penetration resistance of the non-woven composite cocoon structure by restricting the relative motion of fibres, which prevents the sharp tip of the needle from pushing aside fibres and penetrating between them. The wild A. pernyi cocoon exhibits superior thermal buffer over the domestic B. mori cocoon. The unique structure of the A. pernyi cocoon wall with mineral crystals deposited on the cocoon outer surface, can prohibit most of the air from flowing inside of the cocoon structure, which shows strong wind resistance under windy conditions.

Small-sample improved seasonal unit root tests for trending and breaking series

In this article three unit root tests that allow for a break in both the seasonal mean and linear trend of the data are proposed. The tests, which can be seen as small-sample corrected versions of already known asymptotic tests, are shown to perform very well in simulations, and much better than their asymptotic counterparts.

How important is young children's actual and perceived movement skill competence to their physical activity?

OBJECTIVES: To determine the associations between young children's actual and perceived object control and locomotor skills and physical activity and whether associations differ by sex. DESIGN: Cross sectional study. METHODS: A total of 136 children consented. Children had actual skill (Test of Gross Motor Development-2), perceived skill (Pictorial Scale of Perceived Movement Skill Competence for Young Children), and moderate- to vigorous-intensity physical activity (MVPA) (accelerometers) assessed. Independent t-tests assessed sex differences. A regression (with MVPA as the outcome) was performed with all predictor variables (i.e. Actual Object Control, Actual Locomotor, Perceived Object Control, and Perceived Locomotor). Model 2 also adjusted for age, sex, accelerometer wear time and whether the child was from an English speaking background. Interaction terms between the respective actual or perceived skill factor and sex were added to assess sex differences. RESULTS: Analyses were conducted on 109 children (59 boys, 50 girls; mean age=6.5 years, SD=1.0). Boys had higher actual and perceived object control skill and were more active by an average of 19min per day. There were no sex differences in locomotor skills. There were no associations between skill factors and MVPA, except for girls, where locomotor skill was a significant predictor of MVPA (B=3.66, p=0.016). CONCLUSIONS: Actual rather than perceived skill competence was more important to MVPA in this sample. Locomotor skill competence may be more important than object control skill competence for girls as they may engage in types of physical activity that do not require object control mastery.

Habitat selection and energetics of the fiddler crab (Uca tangeri)

We tried to unravel the possible links between the skewed predation risk in Uca tangeri (where large individuals are more at risk from avian predators) and size-dependent changes in the physiology and habitat choice of this fiddler crab species. Over a transect running from low to high in the tidal zone of a beach in Mauritania, the temperature profile at various depths in the substrate, the water-table level of seep water, salt concentration of seep water, depth of the aerobic level, operative temperatures on the surface, and size distribution of crabs were assessed. In addition, resting metabolic rates, Q10 and thermal and starvation tolerances were estimated. Going from low to high in the tidal zone, crab size and burrow depth increased. At the preferred burrowing depth, microclimatological conditions appeared to be equally favourable at all sites. At the surface, conditions were more favourable low in the tidal zone, where also food availability is sufficient to enable small crabs to forage in the vicinity of their burrows. Large crabs have higher energy requirements and are thereby forced to forage in flocks low in the tidal zone where food is probably more abundant. Low in the tidal zone, digging deeply is impossible as the aerobic layer is rather thin. Large crabs prefer living high in the tidal zone as (1) deep burrows ensure better protection against predators, (2) more time is available for digging holes and (3) the substrate is better suited for reproduction. Energy reserves in late summer ensured an average of 34 days of survival. It is argued that the allotment of energy to growth must be considerable even in reproducing animals; the rewards of growth being the disproportional increase in reproductive output with size.

Whole-body vibration and occupational physical performance: a review

Introduction: In the occupational environment, there are a considerable number of stressors that can affect physical performance in job tasks. Whole-body vibration (WBV), which arises from vehicle transit, is one such stressor that has been demonstrated to alter human function in several ways. This study identifies the known physical changes to human function which result from WBV, to comment on changes which may translate to performance in physically demanding occupational tasks. Methods: A systematic review is performed on the literature relating to changes in the neuromuscular, physiological and biomechanical properties of the human body, when exposed to WBV. Selection criteria are constructed to synthesise articles which strictly relate to in-vehicle WBV and physical responses. Results: In total, 29 articles were identified which satisfied the criteria for inclusion. A range of physical responses produced from WBV are presented; however, little consistency exists in study design and the responses reported. Discussion: Given the inconsistency in the reported responses, the precise changes to human function remain unknown. However, there is sufficient evidence to warrant the design of studies which investigate occupationally relevant physical performance changes following WBV.

The concept of physical limitations in knee osteoarthritis: as viewed by patients and health professionals

Objective: To comprehensively identify components of the physical limitation concept in knee osteoarthritis (OA) and to rate the clinical importance of these using perspectives of both patients and health professionals.Design: Concept mapping, a structured group process, was used to identify and organize data in focus groups (patients) and via a global web-based survey (professionals). Ideas were elicited through a nominal group technique and then organized using multidimensional scaling, cluster analysis, participant validation, rating of clinical importance, and thematic analyses to generate a conceptual model of physical limitations in knee OA. Results: Fifteen Danish patients and 200 international professionals contributed to generating the conceptual model. Five clusters emerged: ‘Limitations/physical deficits’; ‘Everyday hurdles’; ‘You’re not the person you used to be’; ‘Need to adjust way of living’; and ‘External limitations,’ each with sub-clusters. Patients generally found their limitations more important than the professionals did. Conclusion: Patients and professionals agreed largely on the physical limitation concept in knee OA. Some limitations of high importance to patients were lower rated by the professionals, highlighting the importance of including patients when conceptualizing patient outcomes. These data offer new knowledge to guide selection of clinically relevant outcomes and development of outcome measures in knee OA.

New tools for understanding the local asymptotic power of panel unit root tests

Abstract Motivated by the previously documented discrepancy between actual and predicted power, the present paper provides new tools for analyzing the local asymptotic power of panel unit root tests. These tools are appropriate in general when considering panel data with a dominant autoregressive root of the form ^ρi=1+^ciN-^κT-^τ, where i=1,...,N indexes the cross-sectional units, T is the number of time periods and ^ci is a random local-to-unity parameter. A limit theory for the sample moments of such panel data is developed and is shown to involve infinite-order series expansions in the moments of ^ci, in which existing theories can be seen as mere first-order approximations. The new theory is applied to study the asymptotic local power functions of some known test statistics for a unit root. These functions can be expressed in terms of the expansions in the moments of ^ci, and include existing local power functions as special cases. Monte Carlo evidence is provided to suggest that the new results go a long way toward bridging the gap between actual and predicted power.

Nonparametric rank tests for non-stationary panels

We develop a set of nonparametric rank tests for non-stationary panels based on multivariate variance ratios which use untruncated kernels. As such, the tests do not require the choice of tuning parameters associated with bandwidth or lag length and also do not require choices with respect to numbers of common factors. The tests allow for unrestricted cross-sectional dependence and dynamic heterogeneity among the units of the panel, provided simply that a joint functional central limit theorem holds for the panel of differenced series. We provide a discussion of the relationships between our setting and the settings for which first- and second generation panel unit root tests are designed. In Monte Carlo simulations we illustrate the small-sample performance of our tests when they are used as panel unit root tests under the more restrictive DGPs for which panel unit root tests are typically designed, and for more general DGPs we also compare the small-sample performance of our nonparametric tests to parametric rank tests. Finally, we provide an empirical illustration by testing for income convergence among countries.

The effect of recursive detrending on panel unit root tests

This paper analyzes the properties of panel unit root tests based on recursively detrended data. The analysis is conducted while allowing for a (potentially) non-linear trend function, which represents a more general consideration than the current state of affairs with (at most) a linear trend. A new test statistic is proposed whose asymptotic behavior under the unit root null hypothesis, and the simplifying assumptions of a polynomial trend and iid errors are shown to be surprisingly simple. Indeed, the test statistic is not only asymptotically independent of the true trend polynomial, but also is in fact unique in that it is independent also of the degree of the fitted polynomial. However, this invariance property does not carry over to the local alternative, under which it is shown that local power is a decreasing function of the trend degree. But while power does decrease, the rate of shrinking of the local alternative is generally constant in the trend degree, which goes against the common belief that the rate of shrinking should be decreasing in the trend degree. The above results are based on simplifying assumptions. To compensate for this lack of generality, a second, robust, test statistic is proposed, whose validity does not require that the trend function is a polynomial or that the errors are iid.

New improved tests for cointegration with structural breaks

This article proposes Lagrange multiplier-based tests for the null hypothesis of no cointegration. The tests are general enough to allow for heteroskedastic and serially correlated errors, deterministic trends, and a structural break of unknown timing in both the intercept and slope. The limiting distributions of the test statistics are derived, and are found to be invariant not only with respect to the trend and structural break, but also with respect to the regressors. A small Monte Carlo study is also conducted to investigate the small-sample properties of the tests. The results reveal that the tests have small size distortions and good power relative to other tests. © 2007 The Authors Journal compilation 2007 Blackwell Publishing Ltd.