Validity and reliability of a pictorial instrument for assessing perceived motor competence in Portuguese children

BACKGROUND: It is important to assess young children's perceived Fundamental Movement Skill (FMS) competence in order to examine the role of perceived FMS competence in motivation toward physical activity. Children's perceptions of motor competence may vary according to the culture/country of origin; therefore, it is also important to measure perceptions in different cultural contexts. The purpose was to assess the face validity, internal consistency, test-retest reliability and construct validity of the 12 FMS items in the Pictorial Scale for Perceived Movement Skill Competence for Young Children (PMSC) in a Portuguese sample.

METHODS: Two hundred one Portuguese children (girls, n = 112), 5 to 10 years of age (7.6 ± 1.4), participated. All children completed the PMSC once. Ordinal alpha assessed internal consistency. A random subsamples (n = 47) were reassessed one week later to determine test-retest reliability with Bland-Altman method. Children were asked questions after the second administration to determine face validity. Construct validity was assessed on the whole sample with a Bayesian Structural Equation Modelling (BSEM) approach. The hypothesized theoretical model used the 12 items and two hypothesized factors: object control and locomotor skills.

RESULTS: The majority of children correctly identified the skills and could understand most of the pictures. Test-retest reliability analysis was good, with an agreement ration between 0.99 and 1.02. Ordinal alpha values ranged from acceptable (object control 0.73, locomotor 0.68) to good (all FMS 0.81). The hypothesized BSEM model had an adequate fit.

CONCLUSIONS: The PMSC can be used to investigate perceptions of children's FMS competence. This instrument can also be satisfactorily used among Portuguese children.

Adaptation and validation of a survey instrument measuring perceived preparedness of nursing graduates

BACKGROUND: In Australia, a significant percentage of bachelor of nursing students are employed in the aged care sector, or in aged care settings, as assistants in nursing (AINs) or personal care assistants. However the value of aged care in nursing education is often overlooked. AIM: To outline the adaptation and validation of a survey, originally developed for medical graduates, for use with nursing graduates. DISCUSSION: Adaptation of the instrument was undertaken as part of a doctoral study that aimed to explore whether employment as an undergraduate assistant in nursing (AIN) in aged care prepares new graduates for clinical work. CONCLUSION: Outlining each step of the modification process can help nurse researchers who want to adapt existing instruments to meet their research objectives. IMPLICATIONS FOR PRACTICE: Undergraduate AIN employment has the potential to supplement clinical learning without the restrictions inherent in the student role. Furthermore, it has the potential to enhance recruitment and retention in the aged care sector.

Developing a Process to Create and Validate an Instrument Assessing Student Attainment of Competencies at an Intercultural University in Mexico

This study took place at one of the intercultural universities (IUs) of Mexico that serve primarily indigenous students. The IUs are pioneers in higher education despite their numerous challenges (Bertely, 1998; Dietz, 2008; Pineda & Landorf, 2010; Schmelkes, 2009). To overcome educational inequalities among their students (Ahuja, Berumen, Casillas, Crispín, Delgado et al., 2004; Schmelkes, 2009), the IUs have embraced performance-based assessment (PBA; Casillas & Santini, 2006). PBA allows a shared model of power and control related to learning and evaluation (Anderson, 1998). While conducting a review on PBA strategies of the IUs, the researcher did not find a PBA instrument with valid and reliable estimates. The purpose of this study was to develop a process to create a PBA instrument, an analytic general rubric, with acceptable validity and reliability estimates to assess students’ attainment of competencies in one of the IU’s majors, Intercultural Development Management. The Human Capabilities Approach (HCA) was the theoretical framework and a sequential mixed method (Creswell, 2003; Teddlie & Tashakkori, 2009) was the research design. IU participants created a rubric during two focus groups, and seven Spanish-speaking professors in Mexico and the US piloted using students’ research projects. The evidence that demonstrates the attainment of competencies at the IU is a complex set of actual, potential and/or desired performances or achievements, also conceptualized as “functional capabilities” (FCs; Walker, 2008), that can be used to develop a rubric. Results indicate that the rubric’s validity and reliability estimates reached acceptable estimates of 80% agreement, surpassing minimum requirements (Newman, Newman, & Newman, 2011). Implications for practice involve the use of PBA within a formative assessment framework, and dynamic inclusion of constituencies. Recommendations for further research include introducing this study’s instrument-development process to other IUs, conducting parallel mixed design studies exploring the intersection between HCA and assessment, and conducting a case study exploring assessment in intercultural settings. Education articulated through the HCA empowers students (Unterhalter & Brighouse, 2007; Walker, 2008). This study aimed to contribute to the quality of student learning assessment at the IUs by providing a participatory process to develop a PBA instrument.

The inter-rater reliability of the Risk Instrument for Screening in the Community

Predicting risk of adverse healthcare outcomes is important to enable targeted delivery of interventions. The Risk Instrument for Screening in the Community (RISC), designed for use by public health nurses (PHNs), measures the one-year risk of hospitalisation, institutionalisation and death in community-dwelling older adults according to a five-point global risk score: from low (score 1,2), medium (3) and high (4,5). We examined the inter-rater reliability (IRR) of the RISC between student PHNs (n=32) and expert raters using six cases (two low, medium and high-risk), scored before and after RISC training. Correlations increased for each adverse outcome, statistically significantly for institutionalisation (r=0.72 to 0.80,p=0.04) and hospitalisation, (r=0.51 to 0.71,p<0.01) but not death. Training improved accuracy for low-risk but not all high-risk cases. Overall, the RISC showed good IRR, which increased after RISC training. That reliability reduced for some high-risk cases suggests that the training programme requires adjustment to further improve IRR.

Quality in palliative care from the patient perspective : Instrument development, perceptions of care received and the importance of care

The overall aim was to investigate the quality of palliative care from the patient perspective, to adapt and psychometrically evaluate the Quality from Patients’ Perspective instrument specific to palliative care (QPP-PC) and investigate the relationship between the combination of person- and organization-related conditions and patients’ perceptions of care quality. Methods: In the systematic literature review (I), 23 studies from 6 databases and reference lists in 2014 were synthesized by integrative thematic analysis. The quantitative studies (II–IV) had cross-sectional designs including 191 patients (73% RR) from hospice inpatient care, hospice day care, palliative units in nursing homes and home care in 2013–2014. A modified version of QPP was used. Additionally, person- and organization-related conditions were assessed. Psychometric evaluation, descriptive and inferential statistics were used. Main findings: Patients’ preferences for palliative care included living a meaningful life and responsive healthcare personnel, care environment and organization of care (I). The QPP-PC was developed, comprising 12 factors (49 items), 3 single items and 4 dimensions: medical–technical competence, physical–technical conditions, identity–oriented approach, and socio-cultural atmosphere (II). QPP-PC measured patients’ perceived reality (PR) and subjective importance (SI) of care quality. PR differed across settings, but SI did not (III). All settings exhibited areas of strength and for improvement (II, III). Person-related conditions seemed to be related to SI, and person- and organization-related conditions to PR, explaining 18–30 and 22-29% respectively of the variance (IV). Conclusions: The patient perspective of care quality (SI and PR) should be integrated into daily care and improvement initiatives in palliative care. The QPP-PC can measure patients’ perceptions of care quality. Registered nurses and other healthcare personnel need awareness of person- and organization-related conditions to provide high-quality person-centred care.

Music of Today Series presents Newband featuring the Harry Partch Instrument Collection November 7, 2012

Concert Program

Longitudinal Student Evaluations of a Postgraduate Unit Using Importance-Performance Analysis

The Australian tourism tertiary education sector operates in a competitive and dynamic environment, which necessitates a market orientation to be successful. Academic staff and management in the sector must regularly assess the perceptions of prospective and current students, and monitor the satisfaction levels of current students. This study is concerned with the setting and monitoring of satisfaction levels of current students, reporting the results of three longitudinal investigations of student satisfaction in a postgraduate unit. The study also addresses a limitation of a university’s generic teaching evaluation instrument. Importance-performance analysis (IPA) has been recommended as a simple but effective tool for overcoming the deficiencies of many student evaluation studies, which have generally measured only attribute importance or importance at the end of a semester. IPA was used to compare student expectations of the unit at the beginning of semester with their perceptions of performance ten weeks later. The first stage documented key benchmarks for which amendments to the unit based on student feedback could be evaluated during subsequent teaching periods.

Survey design: Insights from a public sector ERP success story

Research on the impact of Information Systems (IS) reported in both academic literature and popular press has reported confounding results. Some studies have reported encouraging results of IS, while others have reported nil or detrimental results. The contradictory results of these research studies can be partially attributed to the weaknesses in survey instruments. In an attempt to increase the validity of conclusions of IS assessment studies, survey instrument design should follow a rigorous and scientific procedure. This paper illustrates key validity and reliability issues in measuring Information Systems performance, using examples from a study designed to assess Enterprise Resource Planning systems success. The article emphasizes on the importance of the survey method and the theoretical considerations of item derivation, scale development and item evaluation. Examples are provided from the ERP assessment study to supplement the readers understanding of the theoretical concepts of survey design.

A balanced scorecard approach to enterprise systems performance measurement

A range of influences, both technical and organisational, has encouraged the wide spread adoption of Enterprise Systems (ES). Nevertheless, there is a growing consensus that Enterprise Systems have in many cases failed to provide expected benefits. The increasing role of, and dependency on ES (and IT in general), and the ‘uncertainty’ of these large investments, have created a strong need to monitor and measure ES performance. This paper reports on a research project aimed at deriving an ‘Enterprise Systems benefits measurement instrument’. The research seeks to identify how Enterprise Systems benefits can be usefully measured, with a ‘balance’ between qualitative and quantitative factors.

The function of property in creating an effective legal regime for trade in environmental offsets

Market-based environmental regulation is becoming increasingly common within international and national frameworks. Environmental offset and trading regimes are part of the market-based instrument revolution. This paper proposes that environmental market mechanisms could be used to introduce an ethic of land holder responsibility. In order for market based regimes to attract sufficient levels of stakeholder engagement, participants within such scheme require an incentive to participate and furthermore need to feel a sense of security about investing in such processes. A sense of security is often associated with property based interests. This paper explores the property related issues connected with environmental offset and trading scheme initiatives. Relevant property-related considerations include land tenure considerations, public versus private management of land choices, characteristics and powers associated with property interests, theories defining property and the recognition of legal proprietal interests. The Biodiversity Banking Scheme in New South Wales is then examined as a case study followed by a critique on the role of environmental markets.

The STRATIFY tool and clinical judgment were poor predictors of falling in an acute hospital setting

Objective: To compare the effectiveness of the STRATIFY falls tool with nurses’ clinical judgments in predicting patient falls. Study Design and Setting: A prospective cohort study was conducted among the inpatients of an acute tertiary hospital. Participants were patients over 65 years of age admitted to any hospital unit. Sensitivity, specificity, and positive predictive value (PPV) and negative predictive values (NPV) of the instrument and nurses’ clinical judgments in predicting falls were calculated. Results: Seven hundred and eighty-eight patients were screened and followed up during the study period. The fall prevalence was 9.2%. Of the 335 patients classified as being ‘‘at risk’’ for falling using the STRATIFY tool, 59 (17.6%) did sustain a fall (sensitivity50.82, specificity50.61, PPV50.18, NPV50.97). Nurses judged that 501 patients were at risk of falling and, of these, 60 (12.0%) fell (sensitivity50.84, specificity50.38, PPV50.12, NPV50.96). The STRATIFY tool correctly identified significantly more patients as either fallers or nonfallers than the nurses (P50.027). Conclusion: Considering the poor specificity and high rates of false-positive results for both the STRATIFY tool and nurses’ clinical judgments, we conclude that neither of these approaches are useful for screening of falls in acute hospital settings.

Measuring the use of information and communication technologies (ICTs) in the classroom

In 2003, the “ICT Curriculum Integration Performance Measurement Instrument” was developed froman extensive review ofthe contemporary international and Australian research pertaining to the definition and measurement of ICT curriculum integration in classrooms (Proctor, Watson, & Finger, 2003). The 45-item instrument that resulted was based on theories and methodologies identified by the literature review. This paper describes psychometric results from a large-scale evaluation of the instrument subsequently conducted, as recommended by Proctor, Watson, and Finger (2003). The resultant 20-item, two-factor instrument, now called “Learning with ICTs: Measuring ICT Use in the Curriculum,” is both statistically and theoretically robust. This paper should be read in association with the original paper published in Computers in the Schools(Proctor, Watson, & Finger, 2003) that described in detail the theoretical framework underpinning the development of the instrument.

Operationalizing the resource based view for nascent and young firms : developing scales for resource advantages and disadvantages

Principal Topic: It is well known that most new ventures suffer from a significant lack of resources, which increases the risk of failure (Shepherd, Douglas and Shanley, 2000) and makes it difficult to attract stakeholders and financing for the venture (Bhide & Stevenson, 1999). The Resource-Based View (RBV) (Barney, 1991; Wernerfelt, 1984) is a dominant theoretical base increasingly drawn on within Strategic Management. While theoretical contributions applying RBV in the domain of entrepreneurship can arguably be traced back to Penrose (1959), there has been renewed attention recently (e.g. Alvarez & Busenitz, 2001; Alvarez & Barney, 2004). This said, empirical work is in its infancy. In part, this may be due to a lack of well developed measuring instruments for testing ideas derived from RBV. The purpose of this study is to develop a measurement scales that can serve to assist such empirical investigations. In so doing we will try to overcome three deficiencies in current empirical measures used for the application of RBV to the entrepreneurship arena. First, measures for resource characteristics and configurations associated with typical competitive advantages found in entrepreneurial firms need to be developed. These include such things as alertness and industry knowledge (Kirzner, 1973), flexibility (Ebben & Johnson, 2005), strong networks (Lee et al., 2001) and within knowledge intensive contexts, unique technical expertise (Wiklund and Shepard, 2003). Second, the RBV has the important limitations of being relatively static and modelled on large, established firms. In that context, traditional RBV focuses on competitive advantages. However, newly established firms often face disadvantages, especially those associated with the liabilities of newness (Aldrich & Auster, 1986). It is therefore important in entrepreneurial contexts to expand to an investigation of responses to competitive disadvantage through an RBV lens. Conversely, recent research has suggested that resource constraints actually have a positive effect on firm growth and performance under some circumstances (e.g., George, 2005; Katila & Shane, 2005; Mishina et al., 2004; Mosakowski, 2002; cf. also Baker & Nelson, 2005). Third, current empirical applications of RBV measured levels or amounts of particular resources available to a firm. They infer that these resources deliver firms competitive advantage by establishing a relationship between these resource levels and performance (e.g. via regression on profitability). However, there is the opportunity to directly measure the characteristics of resource configurations that deliver competitive advantage, such as Barney´s well known VRIO (Valuable, Rare, Inimitable and Organized) framework (Barney, 1997). Key Propositions and Methods: The aim of our study is to develop and test scales for measuring resource advantages (and disadvantages) and inimitability for entrepreneurial firms. The study proceeds in three stages. The first stage developed our initial scales based on earlier literature. Where possible, we adapt scales based on previous work. The first block of the scales related to the level of resource advantages and disadvantages. Respondents were asked the degree to which each resource category represented an advantage or disadvantage relative to other businesses in their industry on a 5 point response scale: Major Disadvantage, Slight Disadvantage, No Advantage or Disadvantage, Slight Advantage and Major Advantage. Items were developed as follows. Network capabilities (3 items) were adapted from (Madsen, Alsos, Borch, Ljunggren & Brastad, 2006). Knowledge resources marketing expertise / customer service (3 items) and technical expertise (3 items) were adapted from Wiklund and Shepard (2003). flexibility (2 items), costs (4 items) were adapted from JIBS B97. New scales were developed for industry knowledge / alertness (3 items) and product / service advantages. The second block asked the respondent to nominate the most important resource advantage (and disadvantage) of the firm. For the advantage, they were then asked four questions to determine how easy it would be for other firms to imitate and/or substitute this resource on a 5 point likert scale. For the disadvantage, they were asked corresponding questions related to overcoming this disadvantage. The second stage involved two pre-tests of the instrument to refine the scales. The first was an on-line convenience sample of 38 respondents. The second pre-test was a telephone interview with a random sample of 31 Nascent firms and 47 Young firms (< 3 years in operation) generated using a PSED method of randomly calling households (Gartner et al. 2004). Several items were dropped or reworded based on the pre-tests. The third stage (currently in progress) is part of Wave 1 of CAUSEE (Nascent Firms) and FEDP (Young Firms), a PSED type study being conducted in Australia. The scales will be tested and analysed with a random sample of approximately 700 Nascent and Young firms respectively. In addition, a judgement sample of approximately 100 high potential businesses in each category will be included. Findings and Implications: The paper will report the results of the main study (stage 3 – currently data collection is in progress) will allow comparison of the level of resource advantage / disadvantage across various sub-groups of the population. Of particular interest will be a comparison of the high potential firms with the random sample. Based on the smaller pre-tests (N=38 and N=78) the factor structure of the items confirmed the distinctiveness of the constructs. The reliabilities are within an acceptable range: Cronbach alpha ranged from 0.701 to 0.927. The study will provide an opportunity for researchers to better operationalize RBV theory in studies within the domain of entrepreneurship. This is a fundamental requirement for the ability to test hypotheses derived from RBV in systematic, large scale research studies.

Measuring the quality of teaching in elementary school classrooms

This paper reviews the means by which teacher quality has been measured. It considers data sources such as students, peers, experts, and examines the psychometrics and scaleproperties of teacher quality assessment instruments with respect to reliability and validity. A list of items for possible inclusion in an elementary student focussed instrument is considered, together with the potential use of such an instrument in measuring teacher quality.

Elementary students, creativity, and technology

This article reports on the impact on student personal creativity of a longitudinal study that had as its major goal the creation of a unique intervention program for elementary students. The intervention was based on the National Profile and Statement (Curriculum Corporation, 1994a, 1994b) for the curriculum area of technology. The intervention program comprised thematically based units of work that integrated all eight Australian Key Learning Areas (KLA). A pretest/posttest control group design investigation (Campbell & Stanley, 1963) was undertaken with 580 students from 7 schools and 24 class groups that were randomly divided into 3 treatment groups. One group (10 classes) formed the control group. Another 7 classes received the year-long intervention program, while the remaining 7 classes received the intervention, but with the added seamless integration of their available classroom computer technologies. The effect of the intervention on the personal creativity characteristics of the students involved in the study was assessed using the Creativity Checklist, an instrument that was developed during the study. The results suggest that the purposeful integration of computer technology with the intervention program positively affects the personal creativity characteristics of students.