An investigation of the impacts of Excellence in Research for Australia : a case study on accounting for research

This is the first study to explore the way Excellence in Research for Australia (ERA), a research assessment exercise introduced in the Australian higher education sector in 2010, fostered the development of strategically oriented Management Accounting technologies in the form of Performance Management Systems (PMS) to achieve research excellence within an Australian university. It identifies ERA's intended and unintended consequences. While ERA enabled the creation of tighter controls in the PMS of faculties, departments and individual academics within the university, enhancing its reported research performance, the impact on academics was low job satisfaction, increased workload and a higher focus on research than teaching.

Play the game or get played? Researchers’ strategies around R&D policies

This chapter focuses on possible effects of current R&D policies in the scientific work, exploring some of the dilemmas they cause to researchers. In a transnational scale, R&D policies embrace performance-based research funding systems, calling for a growing accountability and a more useful and published research. Often justified by the importance of knowledge in public policies or as part of the new managerialism regime, these trends emphasize performativity on research. In this scenario, how researchers receive and interpret R&D policies is influenced according to their values and interests? Do they play the game or do they get played by it? These questions rely on a conceptual framework that conceives the research as a political scene, where researchers and R&D policies meet. Moreover, researchers’ strategies are perceived as political, considering that it is in the context of the practices that policy is interpreted and reinvented. The chapter presents an empirical study conducted in Portugal, which will be taken as an example of what Waitere et al (2011) already named as “choosing whether to resist or reinforce” R&D policies. In fact, the study revealed a strategic calculation made by researchers and the coexistence of convergent and divergent strategies concerning R&D policies. I will argue that the tensions in this strategic game are both a reflex and generator of the dilemmas of scientific work today and a sign of the complexity of public policies.

Third-mission policy goals and incentives from performance-based funding: Are they aligned?

In competitive knowledge-based economies, policymakers recognize the importance of universities’ engagement in third mission activities. This article investigates how a specific policy approach to encourage third mission engagement—the use of performance-based funding to reward universities’ success in this domain—aligns with the broader goals of third mission policy. Considering the case of the UK, the first country to have implemented a system of this kind, we analyse how the system has come into being and how it has evolved, and we discuss whether its implementation is likely to encourage universities to behave in ways that are aligned with the goals of third mission policy, as outlined in government documents. We argue that the system encourages universities to focus on a narrow range of income-producing third mission activities, and this is not well aligned with the policy goal to support a complex innovation ecosystem comprising universities with different third mission objectives and strategies. The article concludes by proposing possible avenues for achieving greater alignment between incentives and policy goals.

Community college faculty commitment to Florida performance -based funding indicators for academic programs

In 1996, the State of Florida implemented a performance-based funding program for the Associate in Arts degree offered by community colleges. Additional funds are allocated for distribution among public community colleges based on performance indicators. The indicators are comprised of 10 performance goals that refer to productivity indexed by overall degree completions as well as subgroups: special disadvantaged populations, transfers, job placements, and education acceleration. ^ This study examined the level of self-reported commitment of community college faculty to the 10 Florida performance-based funding indicators for academic programs. Also examined were the relationships between commitment and (a) self-efficacy in contributing to the achievement of the indicators and (b) personal financial reward expectation for contributing to the achievement of the indicators. The relationships between commitment and (a) gender, (b) academic rank, and (c) types of courses taught were analyzed based on secondary analyses. ^ The participants were 303 full-time faculty members of Miami-Dade Community College who taught courses taken by students pursuing the Associate in Arts degree. A questionnaire was developed to measure commitment, self-efficacy, and expectation of financial reward for each of the 10 indicators. ^ The mean composite commitment score for faculty members who responded to the survey was 4.07 in a scale of 1 to 5. Greater commitment was reported for indicators closely related to the traditional mission of community colleges (i.e., facilitating progress of special groups in earning the AA degree in preparation for transferring to a four-year university). Lower commitment was reported for indicators oriented to State priorities such as education acceleration mechanisms and job placements. Commitment was correlated with three variables: self-efficacy, expectation of financial reward, and types of courses taught. However, commitment was not related to gender and academic rank. Although a cause-effect relationship cannot be inferred from this study, the findings depict a positive relationship between faculty commitment to performance-based funding indicators and faculty self-efficacy to contribute to the achievement of the indicators. ^

Predicting Academic Performance Based on Students' Blog and Microblog Posts

This study investigates the degree to which textual complexity indices applied on students’ online contributions, corroborated with a longitudinal analysis performed on their weekly posts, predict academic performance. The source of student writing consists of blog and microblog posts, created in the context of a project-based learning scenario run on our eMUSE platform. Data is collected from six student cohorts, from six consecutive installments of the Web Applications Design course, comprising of 343 students. A significant model was obtained by relying on the textual complexity and longitudinal analysis indices, applied on the English contributions of 148 students that were actively involved in the undertaken projects.

Performance Research in Rangeland Communities

In 2003, Bill Dunstone, John McCallum and Paul Makeham began a collaboration with researchers at the Centre for the Management of Arid Environments (CMAE) in Kalgoorlie, Western Australia. CMAE researchers are keen to develop 'people-oriented' strategies for implementing agricultural extension initiatives in their region. Traditional hierarchies of knowledge-transfer have impeded the 'connectedness' between community and researchers that gives meaning and relevance to useful practice (Ison and Russell, 2000). Our aim is to establish a partnership between the Live Events Research Network (LERN) and CMAE, investigating ways to link creative, performance-based research and practice with the scientific methodologies associated with natural resources management. This accords with recent work undertaken by Deborah Mills and Paul Brown, showing how community cultural development strategies enhance the implementation of policy concerned with community wellbeing. Mills and Brown 'adopted a concept of wellbeing which builds on a social and environmental view of health', and considered such themes as ecological sustainability, rural economic revitalisation, community strengthening, health and wellbeing (Mills, 2003). We propose that rangeland communities can creatively manage some of the challenges confronting them through performance-based projects which: - activate the stories through which a community enacts its sense of place; - facilitate live events in which the community enacts ownership of its culture and identity; - directly involve the community in the formulation of research issues

The implementation of performance based planning in Queensland under the integrated planning act 1997 : an evaluation of perceptions and planning schemes

Performance based planning is a form of planning regulation that is not well understood and the theoretical advantages of this type of planning are rarely achieved in practice. Normatively, this type of regulation relies on performance standards that are quantifiable and technically based which are designed to manage the effects of development, where performance standards provide certainty in respect of the level of performance and the means of achievement is flexible. Few empirical studies have attempted to examine how performance based planning has been conceptualised and implemented in practice. Existing literature is predominately anecdotal and consultant based (Baker et al. 2006) and has not sought to quantitatively examine how land use has been managed or determine how context influences implementation. The Integrated Planning Act 1997 (IPA) operated as Queensland’s principal planning legislation between March 1998 and December 2009. The IPA prevented Local Governments from prohibiting development or use and the term zone was absent from the legislation. While the IPA did not use the term performance based planning, the system is widely considered to be performance based in practice (e.g. Baker et al. 2006; Steele 2009a, 2009b). However, the degree to which the IPA and the planning system in Queensland is performance based is debated (e.g. Yearbury 1998; England 2004). Four research questions guided the research framework using Queensland as the case study. The questions sought to: determine if there is a common understanding of performance based planning; identify how performance based planning was expressed under the IPA; understand how performance based planning was implemented in plans; and explore the experiences of participants in the planning system. The research developed a performance adoption spectrum. The spectrum describes how performance based planning is implemented, ranging between pure and hybrid interpretations. An ex-post evaluation of seventeen IPA plans sought to determine plan performativity within the conceptual spectrum. Land use was examined from the procedural dimension of performance (Assessment Tables) and the substantive dimension of performance (Codes). A documentary analysis and forty one interviews supplemented the research. The analytical framework considered how context influenced performance based planning, including whether: the location of the local government affected land use management techniques; temporal variation in implementation exists; plan-making guidelines affected implementation; different perceptions of the concept exist; this type of planning applies to a range of spatial scales. Outcomes were viewed as the medium for determining the acceptability of development in Queensland, a significant departure from pure approaches found in the United States. Interviews highlighted the absence of plan-making direction in the IPA, which contributed to the confusion about the intended direction of the planning system and the myth that the IPA would guarantee a performance based system. A hybridised form of performance based planning evolved in Queensland which was dependent on prescriptive land use zones and specification of land use type, with some local governments going to extreme lengths to discourage certain activities in a predetermined manner. Context had varying degrees of influence on plan-making methods. Decision-making was found to be inconsistent and the system created a range of unforeseen consequences including difficulties associated with land valuation, increased development speculation, and the role of planners in court was found to be less critical than in the previous planning system.

The life course of Australian researchers : experiences of applying for research funding

The life course of Australian researchers includes regular funding applications, which incur large personal and time costs. We previously estimated that Australian researchers spent 550 years preparing 3,727 proposals for the 2012 NHMRC Project Grant funding round, at an estimated annual salary cost of AU$66 million. Despite the worldwide importance of funding rounds, there is little evidence on what researchers think of the application process. We conducted a web-based survey of Australian researchers (May–July 2013) asking about their experience with NHMRC Project Grants. Almost all researchers (n=224 at 31 May) supported changes to the application (96%) and peer-review (88%) processes; 73% supported the introduction of shorter initial Expressions of Interest; and half (50%) provided extensive comments on the NHMRC processes. Researchers agreed preparing their proposals always took top priority over other work (97%) and personal (87%) commitments. More than half (57%) provided extensive comments on the ongoing personal impact of concurrent grant-writing and holiday seasons on family, children and other relationships. Researchers with experience on Grant Review Panels (34%) or as External Reviewers (78%) reported many sections of the proposals were rarely or never read, which suggests these sections could be cut with no impact on the quality of peer review. Our findings provide evidence on the experience of Australian researchers as applicants. The process of preparing, submitting and reviewing proposals could be streamlined to minimise the burden on applicants and peer reviewers, giving Australian researchers more time to work on actual research and be with their families.

Using performance-based evaluation to close the loop between biological and robotic navigation

In this paper we describe the benefits of a performance-based approach to modeling biological systems for use in robotics. Specifically, we describe the RatSLAM system, a computational model of the navigation processes thought to drive navigation in a part of the rodent brain called the hippocampus. Unlike typical computational modeling approaches, which focus on biological fidelity, RatSLAM’s development cycle has been driven primarily by performance evaluation on robots navigating in a wide variety of challenging, real world environments. We briefly describe three seminal results, two in robotics and one in biology. In addition, we present current research on brain-inspired learning algorithms with the aim of enabling a robot to autonomously learn how best to use its sensor suite to navigate, without requiring any specific knowledge of the robot, sensor types or environment characteristics. Our aim is to drive discussion on the merits of practical, performance-focused implementations of biological models in robotics.

Performance-based structural fire design and testing of structures

Fire incident in buildings is common in Hong Kong and this could lead to heavy casualties due to its high population density, so the fire safety design of the framed structure is an important research topic. This paper describes a computer tool for determination of capacity of structural safety against various fire scenarios and the well-accepted second-order direct plastic analysis is adopted for simulation of material yielding and buckling. A computer method is developed to predict structural behaviour of bare steel framed structures at elevated temperatures but the work can be applied to structures made of other materials. These effects of thermal expansion and material degradation due to heating are required to be considered in order to capture the actual behavior of the structure under fire. Degradation of material strength with increasing temperature is included by a set of temperature-stress-strain curves according to BS5950 Part 8 mainly, which implicitly allows for creep deformation. Several numerical and experimental verifications of framed structures are presented and compared against solutions by other researchers. The proposed method allows us to adopt the truly performance-based structural fire analysis and design with significant saving in cost and time.

Streamlined research funding using short proposals and accelerated peer review : an observational study

Background Despite the widely recognised importance of sustainable health care systems, health services research remains generally underfunded in Australia. The Australian Centre for Health Services Innovation (AusHSI) is funding health services research in the state of Queensland. AusHSI has developed a streamlined protocol for applying and awarding funding using a short proposal and accelerated peer review. Method An observational study of proposals for four health services research funding rounds from May 2012 to November 2013. A short proposal of less than 1,200 words was submitted using a secure web-based portal. The primary outcome measures are: time spent preparing proposals; a simplified scoring of grant proposals (reject, revise or accept for interview) by a scientific review committee; and progressing from submission to funding outcomes within eight weeks. Proposals outside of health services research were deemed ineligible. Results There were 228 eligible proposals across 4 funding rounds: from 29% to 79% were shortlisted and 9% to 32% were accepted for interview. Success rates increased from 6% (in 2012) to 16% (in 2013) of eligible proposals. Applicants were notified of the outcomes within two weeks from the interview; which was a maximum of eight weeks after the submission deadline. Applicants spent 7 days on average preparing their proposal. Applicants with a ranking of reject or revise received written feedback and suggested improvements for their proposals, and resubmissions composed one third of the 2013 rounds. Conclusions The AusHSI funding scheme is a streamlined application process that has simplified the process of allocating health services research funding for both applicants and peer reviewers. The AusHSI process has minimised the time from submission to notification of funding outcomes.

What is the impact of research funding on research productivity?

A research protocol for our prospective study of research funding. How much research funding improves research productivity is a question that has relevance for all funding agencies and governments around the world. Previous studies have used observational data that compares productivity between winners of different amounts of funding, but researchers who win lots of funding are usually very different from those who win little or no funding. This difference creates potentially serious confounding which biases any estimate of the effect of funding based on observational data that simply compares research output for those who did and did not win funding. This means we do not currently know the return on investment for our research dollars, of which billions are invested around the world every year. By using a study design that incorporates randomisation this will be the world’s first unbiased study of the impact of researcher funding.

Performance-Based Quality Assurance/Quality Control (QA/QC) Acceptance Procedures for In-Place Soil Testing Phase 3

One of the objectives of this study was to evaluate soil testing equipment based on its capability of measuring in-place stiffness or modulus values. As design criteria transition from empirical to mechanistic-empirical, soil test methods and equipment that measure properties such as stiffness and modulus and how they relate to Florida materials are needed. Requirements for the selected equipment are that they be portable, cost effective, reliable, a ccurate, and repeatable. A second objective is that the selected equipment measures soil properties without the use of nuclear materials.The current device used to measure soil compaction is the nuclear density gauge (NDG). Equipment evaluated in this research included lightweight deflectometers (LWD) from different manufacturers, a dynamic cone penetrometer (DCP), a GeoGauge, a Clegg impact soil tester (CIST), a Briaud compaction device (BCD), and a seismic pavement analyzer (SPA). Evaluations were conducted over ranges of measured densities and moistures.Testing (Phases I and II) was conducted in a test box and test pits. Phase III testing was conducted on materials found on five construction projects located in the Jacksonville, Florida, area. Phase I analyses determined that the GeoGauge had the lowest overall coefficient of variance (COV). In ascending order of COV were the accelerometer-type LWD, the geophone-type LWD, the DCP, the BCD, and the SPA which had the highest overall COV. As a result, the BCD and the SPA were excluded from Phase II testing.In Phase II, measurements obtained from the selected equipment were compared to the modulus values obtained by the static plate load test (PLT), the resilient modulus (MR) from laboratory testing, and the NDG measurements. To minimize soil and moisture content variability, the single spot testing sequence was developed. At each location, test results obtained from the portable equipment under evaluation were compared to the values from adjacent NDG, PLT, and laboratory MR measurements. Correlations were developed through statistical analysis. Target values were developed for various soils for verification on similar soils that were field tested in Phase III. The single spot testing sequence also was employed in Phase III, field testing performed on A-3 and A-2-4 embankments, limerock-stabilized subgrade, limerock base, and graded aggregate base found on Florida Department of Transportation construction projects. The Phase II and Phase III results provided potential trend information for future research—specifically, data collection for in-depth statistical analysis for correlations with the laboratory MR for specific soil types under specific moisture conditions. With the collection of enough data, stronger relationships could be expected between measurements from the portable equipment and the MR values. Based on the statistical analyses and the experience gained from extensive use of the equipment, the combination of the DCP and the LWD was selected for in-place soil testing for compaction control acceptance. Test methods and developmental specifications were written for the DCP and the LWD. The developmental specifications include target values for the compaction control of embankment, subgrade, and base materials.

Performance-based contracting for roads – Experiences of Australia and Indonesia

Many countries over the last decade, have used performance-based contracting (PBC) to manage and maintain roads. The implementation of PBC provides additional benefits for the government/public such as cost savings and improved conditions of contracted road assets. In Australia, PBC is already being implemented on all categories of roads: national, state, urban and rural. Australian PBC arrangement is designed to turn over control and responsibility for roadway system maintenance, rehabilitation, and capital improvement projects to private contractors. Contractors’ responsibilities include determination of treatment types, the design, programming and the undertaking of works needed to maintain road networks at predetermined performance levels. Indonesia initiated two PBC pilot projects in 2011, the Pantura Section Demak-Trengguli (7.68 kilometers) in Central Java Province and Section Ciasem-Pamanukan (18.5 kilometers) in West Java Province. Both sections are categorized as national roads. The contract duration for both of these projects is four years. To facilitate a possible way forward, it is proposed to conduct a study to understand Australia's experiences of advancing from pilot projects to nation-wide programs using PBC. The study focuses on the scope of contracts, bidding processes, risk allocation, and key drivers, using relevant PBC case studies from Australia. Recommendations for future PBC deployment nation-wide should be based on more research associated with risk allocation. This will include investigation of standard conditions of contract. Implications of the contract clauses for the risk management strategy to be adopted by contractors. Based on the nature of risks, some are best managed by the project owner. It is very important that all parties involved to be open to the new rules of contract and to convince themselves about the potential increased benefits of the use of PBC. The most recent states of challenging issues were explored and described.