Discourse and assessment

Many contemporary currents in applied linguistics have favored discourse studies within assessment; there have been calls for cross-fertilization with other areas within applied linguistics, critiques of the positivist tradition within language testing research, and the growing impact of Conversation Analysis (CA) and sociocultural theory. This chapter focuses on the resulting increase in discourse-based studies of oral proficiency assessment techniques. These studies initially focused on the traditional oral proficiency interview but have since been extended to new test formats, including paired and group interaction. We discuss the research carried out on a number of factors in the assessment setting, including the role of the interlocutor, candidate, and rater, and the impact of tasks, task performance conditions, and rating criteria. Recent research has also concentrated more specifically on the assessment of pragmatic competence and on the applications of technology within the assessment of spoken language, including the comparability of semidirect and direct methods for such assessment and the use of computer corpora.

Evaluation of functional performance in commercial buildings

This report fully summarises a project designed to enhance commercial real estate performance within both operational and investment contexts through the development of a model aimed at supporting improved decision-making. The model is based on a risk adjusted discounted cash flow, providing a valuable toolkit for building managers, owners, and potential investors for evaluating individual building performance in terms of financial, social and environmental criteria over the complete life-cycle of the asset. The ‘triple bottom line’ approach to the evaluation of commercial property has much significance for the administrators of public property portfolios in particular. It also has applications more generally for the wider real estate industry given that the advent of ‘green’ construction requires new methods for evaluating both new and existing building stocks. The research is unique in that it focuses on the accuracy of the input variables required for the model. These key variables were largely determined by market-based research and an extensive literature review, and have been fine-tuned with extensive testing. In essence, the project has considered probability-based risk analysis techniques that required market-based assessment. The projections listed in the partner engineers’ building audit reports of the four case study buildings were fed into the property evaluation model developed by the research team. The results are strongly consistent with previously existing, less robust evaluation techniques. And importantly, this model pioneers an approach for taking full account of the triple bottom line, establishing a benchmark for related research to follow. The project’s industry partners expressed a high degree of satisfaction with the project outcomes at a recent demonstration seminar. The project in its existing form has not been geared towards commercial applications but it is anticipated that QDPW and other industry partners will benefit greatly by using this tool for the performance evaluation of property assets. The project met the objectives of the original proposal as well as all the specified milestones. The project has been completed within budget and on time. This research project has achieved the objective by establishing research foci on the model structure, the key input variable identification, the drivers of the relevant property markets, the determinants of the key variables (Research Engine no.1), the examination of risk measurement, the incorporation of risk simulation exercises (Research Engine no.2), the importance of both environmental and social factors and, finally the impact of the triple bottom line measures on the asset (Research Engine no. 3).

Venture idea newness, relatedness and performance in nascent ventures

Principal Topic The study of the origin and characteristics of venture ideas - or ''opportunities'' as they are often called - and their contextual fit are key research goals in entrepreneurship (Davidsson, 2004). We define venture idea as ''the core ideas of an entrepreneur about what to sell, how to sell, whom to sell and how an entrepreneur acquire or produce the product or service which he/she sells'' for the purpose of this study. When realized the venture idea becomes a ''business model''. Even though venture ideas are central to entrepreneurship yet its characteristics and their effect to the entrepreneurial process is mysterious. According to Schumpeter (1934) entrepreneurs could creatively destruct the existing market condition by introducing new product/service, new production methods, new markets, and new sources of supply and reorganization of industries. The introduction, development and use of new ideas are generally called as ''innovation'' (Damanpour & Wischnevsky, 2006) and ''newness'' is a property of innovation and is a relative term which means that the degree of unfamiliarity of venture idea either to a firm or to a market. However Schumpeter's (1934) discusses five different types of newness, indicating that type of newness is an important issue. More recently, Shane and Venkataraman (2000) called for research taking into consideration not only the variation of characteristics of individuals but also heterogeneity of venture ideas, Empirically, Samuelson (2001, 2004) investigated process differences between innovative venture ideas and imitative venture ideas. However, he used only a crude dichotomy regarding the venture idea newness. According to Davidsson, (2004) as entrepreneurs could introduce new economic activities ranging from pure imitation to being new to the entire world market, highlighting that newness is a matter of degree. Dahlqvist (2007) examined the venture idea newness and made and attempt at more refined assessment of the degree and type of newness of venture idea. Building on these predecessors our study refines the assessment of venture idea newness by measuring the degree of venture idea newness (new to the world, new to the market, substantially improved while not entirely new, and imitation) for four different types of newness (product/service, method of production, method of promotion, and customer/target market). We then related type and degree of newness to the pace of progress in nascent venturing process. We hypothesize that newness will slow down the business creation process. Shane & Venkataraman (2000) introduced entrepreneurship as the nexus of opportunities and individuals. In line with this some scholars has investigated the relationship between individuals and opportunities. For example Shane (2000) investigates the relatedness between individuals' prior knowledge and identification of opportunities. Shepherd & DeTinne (2005) identified that there is a positive relationship between potential financial reward and the identification of innovative venture ideas. Sarasvathy's 'Effectuation Theory'' assumes high degree of relatedness with founders' skills, knowledge and resources in the selection of venture ideas. However entrepreneurship literature is scant with analyses of how this relatedness affects to the progress of venturing process. Therefore, we assess the venture ideas' degree of relatedness to prior knowledge and resources, and relate these, too, to the pace of progress in nascent venturing process. We hypothesize that relatedness will increase the speed of business creation. Methodology For this study we will compare early findings from data collected through the Comprehensive Australian Study of Entrepreneurial Emergence (CAUSEE). CAUSEE is a longitudinal study whose primary objective is to uncover the factors that initiate, hinder and facilitate the process of emergence and development of new firms. Data were collected from a representative sample of some 30,000 households in Australia using random digit dialing (RDD) telephone survey interviews. Through the first round of data collection identified 600 entrepreneurs who are currently involved in the business start-up process. The unit of the analysis is the emerging venture, with the respondent acting as its spokesperson. The study methodology allows researchers to identify ventures in early stages of creation and to longitudinally follow their progression through data collection periods over time. Our measures of newness build on previous work by Dahlqvist (2007). Our adapted version was developed over two pre-tests with about 80 participants in each. The measures of relatedness were developed through the two rounds of pre-testing. The pace of progress in the venture creation process is assessed with the help of time-stamped gestation activities; a technique developed in the Panel Study of Entrepreneurial Dynamics (PSED). Results and Implications We hypothesized that venture idea newness slows down the venturing process whereas relatedness facilitates the venturing process. Results of 600 nascent entrepreneurs in Australia indicated that there is marginal support for the hypothesis that relatedness assists the gestation progress. Newness is significant but is the opposite sign to the hypothesized. The results give number of implications for researchers, business founders, consultants and policy makers in terms of better knowledge of the venture creation process.

A collaborative model for learning and assessment of legal placements

This paper will report on the evaluation of a new undergraduate legal workplace unit, LWB421 Learning in Professional Practice. LWB421 was developed in response to the QUT’s strategic planning and a growing view that work experience is essential to developing the skills that law graduates need in order to be effective legal practitioners (Stuckey, 2007). Work integrated learning provides a context for students to develop their skills, to see the link between theory and practice and support students in making the transition from university to practice (Shirley, 2006). The literature in Australian legal education has given little consideration to the design of legal internship subjects (as distinct from legal clinic programs). Accordingly the design of placement subjects needs to be carefully considered to ensure alignment of learning objectives, learning tasks and assessment. Legal placements offer students the opportunity to develop their professional skills in practice, reflect on their own learning and job performance and take responsibility for their career development and planning. This paper will examine the literature relating to the design of placement subjects, particularly in a legal context. It will propose a collaborative model to facilitate learning and assessment of legal work placement subjects. The basis of the model is a negotiated learning contract between the student, workplace supervisor and academic supervisor. Finally the paper will evaluate the model in the context of LWB421. The evaluation will be based on data from surveys of students and supervisors and focus group sessions.

Integrating eco-efficiency assessment of commercial buildings into the design process : LCADesign

The ability to assess a commercial building for its impact on the environment at the earliest stage of design is a goal which is achievable by integrating several approaches into a single procedure directly from the 3D CAD representation. Such an approach enables building design professionals to make informed decisions on the environmental impact of building and its alternatives during the design development stage instead of at the post-design stage where options become limited. The indicators of interest are those which relate to consumption of resources and energy, contributions to pollution of air, water and soil, and impacts on the health and wellbeing of people in the built environment as a result of constructing and operating buildings. 3D object-oriented CAD files contain a wealth of building information which can be interrogated for details required for analysis of the performance of a design. The quantities of all components in the building can be automatically obtained from the 3D CAD objects and their constituent materials identified to calculate a complete list of the amounts of all building products such as concrete, steel, timber, plastic etc. When this information is combined with a life cycle inventory database, key internationally recognised environmental indicators can be estimated. Such a fully integrated tool known as LCADesign has been created for automated ecoefficiency assessment of commercial buildings direct from 3D CAD. This paper outlines the key features of LCADesign and its application to environmental assessment of commercial buildings.

LCADesign : an integrated approach to automatic eco-efficiency assessment of commercial buildings

Buildings consume resources and energy, contribute to pollution of our air, water and soil, impact the health and well-being of populations and constitute an important part of the built environment in which we live. The ability to assess their design with a view to reducing that impact automatically from their 3D CAD representations enables building design professionals to make informed decisions on the environmental impact of building structures. Contemporary 3D object-oriented CAD files contain a wealth of building information. LCADesign has been designed as a fully integrated approach for automated eco-efficiency assessment of commercial buildings direct from 3D CAD. LCADesign accesses the 3D CAD detail through Industry Foundation Classes (IFCs) - the international standard file format for defining architectural and constructional CAD graphic data as 3D real-world objects - to permit construction professionals to interrogate these intelligent drawing objects for analysis of the performance of a design. The automated take-off provides quantities of all building components whose specific production processes, logistics and raw material inputs, where necessary, are identified to calculate a complete list of quantities for all products such as concrete, steel, timber, plastic etc and combines this information with the life cycle inventory database, to estimate key internationally recognised environmental indicators such as CML, EPS and Eco-indicator 99. This paper outlines the key modules of LCADesign and their role in delivering an automated eco-efficiency assessment for commercial buildings.

Authentic and conventional assessment in Singapore schools : an empirical study of teacher assignments and student work

This is an empirical examination of the quality of teacher assignments and student work in Singapore schools. Using a theoretical framework based on principles of authentic assessment and intellectual quality, two sets of criteria and scoring rubrics were developed for the training of expert teachers to judge the quality of assignments and student work. Following rigorous training, the inter-rater reliability of expert teacher scoring was high. Samples of teacher assignments and student work were collected in English, social studies, mathematics, and science subject areas from a random stratified sample of 30 elementary schools and 29 high schools. For both grade levels, there were significant differences for the authentic intellectual quality of teachers’ assignments by subject area. Likewise, the differences of authentic intellectual quality for student work were significant and varied by subject area. Subject area effect was large. The correlations between the quality of teachers’ assignment tasks and student work were strong and significant at both grade levels. Where teachers set more intellectually demanding tasks, students were more likely to generate work or artefacts judged to be of higher quality. The findings suggest that teacher professional development in authentic intellectual assessment task design can contribute to the improvement of student learning and performance. It is argued that this will be a key requisite of educational systems like Singapore that are seeking to expand pedagogy and student outcomes beyond a focus on factual and rote knowledge.

Youth Festival Music Performance : Kingston

Research Background - Young people with negative experiences of mainstream education often display low levels of traditional academic achievement. These young people tend to display considerable cultural and social resources developed through their repeated experiences of adversity. Education research has a duty to provide these young people with opportunities to showcase, assess and translate their social and cultural resources into symbolic forms of capital. This creative work addresses the research question, how can educators maximise the social and cultural capital they help young people acquire through live music performances and studio recordings? Research Contribution - This live music performance, built on existing artistic reputations of the artists, saw the lads support one of their local heroes from Brisbane Hip Hop music scene. In doing so they showcased what their three years of concerted musical engagement can achieve within supportive flexible learning environments. The new knowledge derived from this research focuses on the academic and self confidence benefits for disengaged young people using festival performances as authentic learning activities. Research Significance - This research is significant because it aims to maximise the number of tangible outcomes related to a school-based arts project. The young participants gained technical, artistic, social and commercial status during this project. Individual performances were distributed and downloaded via creative commons licences at the Australian Creative Resource Archive. This performance also contributed to their certified qualifications and acted as pilot research data for two competitively funded ARC grants (DP0209421 & LP0883643)

Dancing between diversity and consistency : refining assessment in postgraduate degrees in dance

This publication is the culmination of a 2 year Australian Learning and Teaching Council's Project Priority Programs Research Grant which investigates key issues and challenges in developing flexible guidelines lines for best practice in Australian Doctoral and Masters by Research Examination, encompassing the two modes of investigation, written and multi-modal (practice-led/based) theses, their distinctiveness and their potential interplay. The aims of the project were to address issues of assessment legitimacy raised by the entry of practice-orientated dance studies into Australian higher degrees; examine literal embodiment and presence, as opposed to cultural studies about states of embodiment; foreground the validity of questions around subjectivity and corporeal intelligence/s and the reliability of artistic/aesthetic communications, and finally to celebrate ‘performance mastery’(Melrose 2003) as a rigorous and legitimate mode of higher research. The project began with questions which centred around: the functions of higher degree dance research; concepts of 'master-ness’ and ‘doctorateness’; the kinds of languages, structures and processes which may guide candidates, supervisors, examiners and research personnel; the purpose of evaluation/examination; addressing positive and negative attributes of examination. Finally the study examined ways in which academic/professional, writing/dancing, tradition/creation and diversity/consistency relationships might be fostered to embrace change. Over two years, the authors undertook a qualitative national study encompassing a triangulation of semi-structured face to face interviews and industry forums to gather views from the profession, together with an analysis of existing guidelines, and recent literature in the field. The most significant primary data emerged from 74 qualitative interviews with supervisors, examiners, research deans and administrators, and candidates in dance and more broadly across the creative arts. Qualitative data gathered from the two primary sources, was coded and analysed using the NVivo software program. Further perspectives were drawn from international consultant and dance researcher Susan Melrose, as well as publications in the field, and initial feedback from a draft document circulated at the World Dance Alliance Global Summit in July 2008 in Brisbane. Refinement of data occurred in a continual sifting process until the final publication was produced. This process resulted in a set of guidelines in the form of a complex dynamic system for both product and process oriented outcomes of multi-modal theses, along with short position papers on issues which arose from the research such as contested definitions, embodiment and ephemerality, ‘liveness’ in performance research higher degrees, dissolving theory/practice binaries, the relationship between academe and industry, documenting practices and a re-consideration of the viva voce.

Comparative assessment of characteristics influencing the industrial property markets in Sydney and Hong Kong

Industrial property is commonly located in a designated ‘industrial’ precinct. An industrial property has a specific design and a number of services to support industrial activities including manufacture, distribution and transportation. Although it has a unique characteristic, certain industrial factor might operate differently in different countries. The aim of this paper is to provide a comparison between the Sydney and Hong Kong industrial property characteristics and to highlight their similarities and differences. This exploratory research used secondary data to provide background information of government policy and market conditions. Two case studies were use to illustrate similarities, trends, differences and to explore town planning, specific property characteristics including location, design and layout. Then, analyse whether these factors influence the performance and value of an industrial asset. The location of industrial properties varies between each country and depends heavily on infrastructure. It was noted that the town planning restrictions not only vary between markets and cities but also between property lots. The market conditions of both industrial markets were investigated and the supply and demand and rental levels in both cities were distinctly opposite.

Harnessing assessment and feedback in the first year to support learning success, engagement and retention

It has been argued that intentional first year curriculum design has a critical role to play in enhancing first year student engagement, success and retention (Kift, 2008). A fundamental first year curriculum objective should be to assist students to make the successful transition to assessment in higher education. Scott (2006) has identified that ‘relevant, consistent and integrated assessment … [with] prompt and constructive feedback’ are particularly relevant to student retention generally; while Nicol (2007) suggests that ‘lack of clarity regarding expectations in the first year, low levels of teacher feedback and poor motivation’ are key issues in the first year. At the very minimum, if we expect first year students to become independent and self-managing learners, they need to be supported in their early development and acquisition of tertiary assessment literacies (Orrell, 2005). Critical to this attainment is the necessity to alleviate early anxieties around assessment information, instructions, guidance, and performance. This includes, for example:  inducting students thoroughly into the academic languages and assessment genres they will encounter as the vehicles for evidencing learning success; and  making expectations about the quality of this evidence clear. Most importantly, students should receive regular formative feedback of their work early in their program of study to aid their learning and to provide information to both students and teachers on progress and achievement. Leveraging research conducted under an ALTC Senior Fellowship that has sought to articulate a research-based 'transition pedagogy' (Kift & Nelson, 2005) – a guiding philosophy for intentional first year curriculum design and support that carefully scaffolds and mediates the first year learning experience for contemporary heterogeneous cohorts – this paper will discuss theoretical and practical strategies and examples that should be of assistance in implementing good assessment and feedback practices across a range of disciplines in the first year.

The refractive folio : exegesis and assessment in the open studio

Folio submission is universally regarded as the most appropriate means for measuring a student’s performance in the studio. However, developing meaningful and defensible assessment criteria is persistent challenge for all tertiary art educators. In discipline-based studios, the parameters provided by medium and technique provide useful points of reference for assessing creative performance. But how can student performance be evaluated when there is no discipline-based framework to act as a point of reference? The ‘open’ studio approach to undergraduate teaching presents these and other pedagogical challenges. This paper discusses the innovative approaches to studio-based teaching and assessment at QUT. Vital to the QUT open studio model is the studio rationale – an exegetical document that establishes an individualised theoretical framework through which a student’s understandings can be, in part, evaluated. This paper argues that the exegetical folio effectively reconciles the frequently divergent imperatives of creative, professional and academic skills, while retaining the centrality of the studio as a site for the production of new material, processual and conceptual understandings.

Performance of public private partnerships : an evolutionary perspective

PPPs are held to be a powerful way of mobilising private finance and resources to deliver public infrastructure. Theoretically, research into procurement has begun to acknowledge difficulties with the classification and assessment of different types of procurement, particularly those which do not sufficiently acknowledge variety within specific types of procurement methods. This paper advances a theoretical framework based on an evolutionary economic conceptualisation of a routine, which can accommodate the variety evident in procurement projects, in particular PPPs. The paper tests how the various elements of a PPP, as advanced in the theoretical framework, affect performance across 10 case studies. It concludes, that a limited number of elements of a PPP affect their performance, and provides strong evidence for the theoretical model advanced in this paper.

Monitoring the Performance of Complex Projects from Multiple Perspectives over Multiple Time Frames

When complex projects go wrong they can go horribly wrong with severe financial consequences. We are undertaking research to develop leading performance indicators for complex projects, metrics to provide early warning of potential difficulties. The assessment of success of complex projects can be made by a range of stakeholders over different time scales, against different levels of project results: the project’s outputs at the end of the project; the project’s outcomes in the months following project completion; and the project’s impact in the years following completion. We aim to identify leading performance indicators, which may include both success criteria and success factors, and which can be measured by the project team during project delivery to forecast success as assessed by key stakeholders in the days, months and years following the project. The hope is the leading performance indicators will act as alarm bells to show if a project is diverting from plan so early corrective action can be taken. It may be that different combinations of the leading performance indicators will be appropriate depending on the nature of project complexity. In this paper we develop a new model of project success, whereby success is assessed by different stakeholders over different time frames against different levels of project results. We then relate this to measurements that can be taken during project delivery. A methodology is described to evaluate the early parts of this model. Its implications and limitations are described. This paper describes work in progress.