Assessment in tertiary mathematics

There is a growing consensus among many educators that the goals of teaching and learning mathematics are to help students solve real-life problems, participate intelligently in daily affairs, and prepare them for jobs (Gardiner, 1994; Roeber, 1995). These goals suggest that the role of routine procedural skills should be diminished while more emphasis ought to be placed on learners gaining conceptual insights and analytical skills that appear essential in real-life mathematical problem solving (Schoenfeld, 1993; Stenmark, 1989).

Implementation of an integrated, technology-based, discovery mode assessment item involving an incubation period to enhance learning outcomes for engineering maths students

In this paper we discuss our current efforts to develop and implement an exploratory, discovery mode assessment item into the total learning and assessment profile for a target group of about 100 second level engineering mathematics students. The assessment item under development is composed of 2 parts, namely, a set of "pre-lab" homework problems (which focus on relevant prior mathematical knowledge, concepts and skills), and complementary computing laboratory exercises which are undertaken within a fixed (1 hour) time frame. In particular, the computing exercises exploit the algebraic manipulation and visualisation capabilities of the symbolic algebra package MAPLE, with the aim of promoting understanding of certain mathematical concepts and skills via visual and intuitive reasoning, rather than a formal or rigorous approach. The assessment task we are developing is aimed at providing students with a significant learning experience, in addition to providing feedback on their individual knowledge and skills. To this end, a noteworthy feature of the scheme is that marks awarded for the laboratory work are primarily based on the extent to which reflective, critical thinking is demonstrated, rather than the amount of CBE-style tasks completed by the student within the allowed time. With regard to student learning outcomes, a novel and potentially critical feature of our scheme is that the assessment task is designed to be intimately linked to the overall course content, in that it aims to introduce important concepts and skills (via individual student exploration) which will be revisited somewhat later in the pedagogically more restrictive formal lecture component of the course (typically a large group plenary format). Furthermore, the time delay involved, or "incubation period", is also a deliberate design feature: it is intended to allow students the opportunity to undergo potentially important internal re-adjustments in their understanding, before being exposed to lectures on related course content which are invariably delivered in a more condensed, formal and mathematically rigorous manner. In our presentation, we will discuss in more detail our motivation and rationale for trailing such a scheme for the targeted student group. Some of the advantages and disadvantages of our approach (as we perceived them at the initial stages) will also be enumerated. In a companion paper, the theoretical framework for our approach will be more fully elaborated, and measures of student learning outcomes (as obtained from eg. student provided feedback) will be discussed.

Changing assessment practices : the case for online moderation

Online moderation meetings have the potential to support the collaborative professional development of teachers, and the formation of a common understanding of what denotes quality in student work in a standards based assessment system. In doing so systemic calls for consistency across education systems are also being met. In this paper a case for employing online moderation meetings is developed through recourse to the demands of learning in the twenty-first century and the place of assessment within those discourses. It is argued that empirical data is needed on the efficacy of online moderation meetings to guide future practice as the use of information and communication technologies increases in education systems. Online moderation is one way of gathering teachers across vast distances to share their understandings and develop common meanings of assessment. While it is suggested that online moderation is one possible procedure to meet systemic requirements and support teachers’ professional collaboration, the implementation of such a system also introduces new challenges for schools and teachers. Meeting online to discuss professional understandings is a new way of operating for teachers and involves technology that has not yet been fully utilised within education departments. Issues such as the types of interactions that are afforded within such an environment, as well as technical operating problems that occur when using technology impact on the employment of online meetings. Online moderation meetings while potentially solving the issue of developing common understandings across an entire department also pose new issues to be resolved. There is a need for research into the efficacy of online moderation meetings so that future policy decisions may be based on sound empirical data. It is imperative that as new ways of knowing and acting are incorporated into school curriculum and pedagogy, assessment practices are also aligned. Online moderation meetings can support such practices by enabling teachers to communicate with a wider and more diverse group of teachers to establish common understandings.

Aligning ICT in assessment with teaching and learning : enhancing student achievement in the middle years

Student underachievement in the middle years (typically Years 4 to 9) is a concern in education. Incorporating Information and Communication Technologies (ICT) in assessment that is aligned to teaching and learning has the potential to engage students in higher cognitive processes that lead to increased student achievement. To examine this proposition an investigation was undertaken into teachers’ perceptions of alignment and the implications of those for student achievement in ICT enhanced middle years assessment tasks. This investigation used a collective case study design underpinned by socio-cultural theory. Two methods were used for data collection, namely, semi-structured interviews with individual teachers and a focus group discussion with teachers and another with students. Findings revealed teachers’ perceptions that alignment: assists in mediating achievement of learning outcomes in quality middle years assessment tasks, assists in creating a challenging but supportive environment in which positive learning dispositions and success is encouraged for all students, and contributes to more rigorous use of ICT in assessment. The process of implementing alignment was found to be complex but assisted through prioritising particular practices. These findings enabled the development of eight steps which serve as a guide to the effective implementation of alignment in middle years assessment tasks.

Formative and Summative Assessment and the Notion of Constructive Allignment

This paper discusses perceptions of first year accounting students about their tutorial activities and their engagements in assessment. As the literature suggests, unless participation in learning activities forms part of graded assessment it is often difficult to engage students in these activities. Using an action research model, this paper reports the study of first year accounting students' responses to action-orientated learning tasks in tutorials. The paper focuses on the importance of aligning curriculum objectives, learning and teaching activities and assessment,i.e. the notion of constructive alignment. However, as the research findings indicate, without support at institutional level, applying constructive alignment to facilitate quality student learning outcomes is a difficult task. Thus, the impacts of policy constraints on curriculum issues are also discussed, focusing on the limitations faced by tutors and their lack of involvement in curriculum development.

Assessment and curriculum reform in China : the college English test and tertiary English as a foreign language education.

Educational assessment was a worldwide commonplace practice in the last century. With the theoretical underpinnings of education shifting from behaviourism and social efficiency to constructivism and cognitive theories in the past two decades, the assessment theories and practices show a widespread changing movement. The emergent assessment paradigm, with a futurist perspective, indicates a deviation away from the prevailing large scale high-stakes standardised testing and an inclination towards classroom-based formative assessment. Innovations and reforms initiated in attempts to achieve better education outcomes for a sustainable future via more developed learning and assessment theories have included the 2007 College English Reform Program (CERP) in Chinese higher education context. This paper focuses on the College English Test (CET) - the national English as a Foreign Language (EFL) testing system for non-English majors at tertiary level in China. It seeks to explore the roles that the CET played in the past two College English curriculum reforms, and the new role that testing and assessment assumed in the newly launched reform. The paper holds that the CET was operationalised to uplift the standards. However, the extended use of this standardised testing system brings constraints as well as negative washback effects on the tertiary EFL education. Therefore in the newly launched reform -CERP, a new assessment model which combines summative and formative assessment approaches is proposed. The testing and assessment, assumed a new role - to engender desirable education outcomes. The question asked is: will the mixed approach to formative and summative assessment provide the intended cure to the agony that tertiary EFL education in China has long been suffering - spending much time, yet achieving little effects? The paper reports the progresses and challenges as informed by the available research literature, yet asserts a lot needs to be explored on the potential of the assessment mix in this examination tradition deep-rooted and examination-obsessed society.

Vision, functional and cognitive determinants of motor vehicle incidents in older drivers

Background: The proportion of older individuals in the driving population is predicted to increase in the next 50 years. This has important implications for driving safety as abilities which are important for safe driving, such as vision (which accounts for the majority of the sensory input required for driving), processing ability and cognition have been shown to decline with age. The current methods employed for screening older drivers upon re-licensure are also vision based. This study, which investigated social, behavioural and professional aspects involved with older drivers, aimed to determine: (i) if the current visual standards in place for testing upon re-licensure are effective in reducing the older driver fatality rate in Australia; (ii) if the recommended visual standards are actually implemented as part of the testing procedures by Australian optometrists; and (iii) if there are other non-standardised tests which may be better at predicting the on-road incident-risk (including near misses and minor incidents) in older drivers than those tests recommended in the standards. Methods: For the first phase of the study, state-based age- and gender-stratified numbers of older driver fatalities for 2000-2003 were obtained from the Australian Transportation Safety Bureau database. Poisson regression analyses of fatality rates were considered by renewal frequency and jurisdiction (as separate models), adjusting for possible confounding variables of age, gender and year. For the second phase, all practising optometrists in Australia were surveyed on the vision tests they conduct in consultations relating to driving and their knowledge of vision requirements for older drivers. Finally, for the third phase of the study to investigate determinants of on-road incident risk, a stratified random sample of 600 Brisbane residents aged 60 years and were selected and invited to participate using an introductory letter explaining the project requirements. In order to capture the number and type of road incidents which occurred for each participant over 12 months (including near misses and minor incidents), an important component of the prospective research study was the development and validation of a driving diary. The diary was a tool in which incidents that occurred could be logged at that time (or very close in time to which they occurred) and thus, in comparison with relying on participant memory over time, recall bias of incident occurrence was minimised. Association between all visual tests, cognition and scores obtained for non-standard functional tests with retrospective and prospective incident occurrence was investigated. Results: In the first phase,rivers aged 60-69 years had a 33% lower fatality risk (Rate Ratio [RR] = 0.75, 95% CI 0.32-1.77) in states with vision testing upon re-licensure compared with states with no vision testing upon re-licensure, however, because the CIs are wide, crossing 1.00, this result should be regarded with caution. However, overall fatality rates and fatality rates for those aged 70 years and older (RR=1.17, CI 0.64-2.13) did not differ between states with and without license renewal procedures, indicating no apparent benefit in vision testing legislation. For the second phase of the study, nearly all optometrists measured visual acuity (VA) as part of a vision assessment for re-licensing, however, 20% of optometrists did not perform any visual field (VF) testing and only 20% routinely performed automated VF on older drivers, despite the standards for licensing advocating automated VF as part of the vision standard. This demonstrates the need for more effective communication between the policy makers and those responsible for carrying out the standards. It may also indicate that the overall higher driver fatality rate in jurisdictions with vision testing requirements is resultant as the tests recommended by the standards are only partially being conducted by optometrists. Hence a standardised protocol for the screening of older drivers for re-licensure across the nation must be established. The opinions of Australian optometrists with regard to the responsibility of reporting older drivers who fail to meet the licensing standards highlighted the conflict between maintaining patient confidentiality or upholding public safety. Mandatory reporting requirements of those drivers who fail to reach the standards necessary for driving would minimise potential conflict between the patient and their practitioner, and help maintain patient trust and goodwill. The final phase of the PhD program investigated the efficacy of vision, functional and cognitive tests to discriminate between at-risk and safe older drivers. Nearly 80% of the participants experienced an incident of some form over the prospective 12 months, with the total incident rate being 4.65/10 000 km. Sixty-three percent reported having a near miss and 28% had a minor incident. The results from the prospective diary study indicate that the current vision screening tests (VA and VF) used for re-licensure do not accurately predict older drivers who are at increased odds of having an on-road incident. However, the variation in visual measurements of the cohort was narrow, also affecting the results seen with the visual functon questionnaires. Hence a larger cohort with greater variability should be considered for a future study. A slightly lower cognitive level (as measured with the Mini-Mental State Examination [MMSE]) did show an association with incident involvement as did slower reaction time (RT), however the Useful-Field-of-View (UFOV) provided the most compelling results of the study. Cut-off values of UFOV processing (>23.3ms), divided attention (>113ms), selective attention (>258ms) and overall score (moderate/ high/ very high risk) were effective in determining older drivers at increased odds of having any on-road incident and the occurrence of minor incidents. Discussion: The results have shown that for the 60-69 year age-group, there is a potential benefit in testing vision upon licence renewal. However, overall fatality rates and fatality rates for those aged 70 years and older indicated no benefit in vision testing legislation and suggests a need for inclusion of screening tests which better predict on-road incidents. Although VA is routinely performed by Australian optometrists on older drivers renewing their licence, VF is not. Therefore there is a need for a protocol to be developed and administered which would result in standardised methods conducted throughout the nation for the screening of older drivers upon re-licensure. Communication between the community, policy makers and those conducting the protocol should be maximised. By implementing a standardised screening protocol which incorporates a level of mandatory reporting by the practitioner, the ethical dilemma of breaching patient confidentiality would also be resolved. The tests which should be included in this screening protocol, however, cannot solely be ones which have been implemented in the past. In this investigation, RT, MMSE and UFOV were shown to be better determinants of on-road incidents in older drivers than VA and VF, however, as previously mentioned, there was a lack of variability in visual status within the cohort. Nevertheless, it is the recommendation from this investigation, that subject to appropriate sensitivity and specificity being demonstrated in the future using a cohort with wider variation in vision, functional performance and cognition, these tests of cognition and information processing should be added to the current protocol for the screening of older drivers which may be conducted at licensing centres across the nation.

Assessment of Speech Dialog Systems using Multi-Modal Cognitive Load Analysis and Driving Performance Metrics

In this paper, cognitive load analysis via acoustic- and CAN-Bus-based driver performance metrics is employed to assess two different commercial speech dialog systems (SDS) during in-vehicle use. Several metrics are proposed to measure increases in stress, distraction and cognitive load and we compare these measures with statistical analysis of the speech recognition component of each SDS. It is found that care must be taken when designing an SDS as it may increase cognitive load which can be observed through increased speech response delay (SRD), changes in speech production due to negative emotion towards the SDS, and decreased driving performance on lateral control tasks. From this study, guidelines are presented for designing systems which are to be used in vehicular environments.

Assessment in education : principles, policy and practice

The English examination system provides a market in which a limited number of providers are accredited to offer curriculum-based examinations in many subject areas and at several levels. The most significant are the General Certificate of Secondary Education (GCSE) and the General Certificate of Education, Advanced (A level). Because these examinations are used for high-stakes purposes, including higher education and employment selection for individuals and programme evaluation for institutions, it is desired that scores from various exams be ‘comparable’ in several respects: across syllabuses and examination boards within a subject area, across years, and even across subject areas. Just how to accomplish this goal has been a topic of continual research and debate for over 50 years, through many changes of examination and institutional structures. But ever year, tens of thousands of scores must be reported, and every year, users expect them to ‘be comparable’ and use them as if they are.

Assessment for learning revisited : an Asia-Pacific perspective

In March of this year, 2009, the Third International Conference on Assessment for Learning was held in Dunedin, New Zealand. Colleagues from Australia (4), Canada (6), Europe (5), New Zealand (7), United Kingdom (5) and the United States of America (4) met to advance the understanding and practices of Assessment for Learning at all levels of education. An important outcome of this meeting was a position paper on Assessment for Learning (AfL) that has been reproduced with permission in this editorial because of its significance to a recurrent theme of the majority of the articles published in this special Asia-Pacific issue of the journal. The text of this statement is set out in italics below.

Comparative assessment of characteristics influencing the industrial property markets in Sydney and Hong Kong

Industrial property is commonly located in a designated ‘industrial’ precinct. An industrial property has a specific design and a number of services to support industrial activities including manufacture, distribution and transportation. Although it has a unique characteristic, certain industrial factor might operate differently in different countries. The aim of this paper is to provide a comparison between the Sydney and Hong Kong industrial property characteristics and to highlight their similarities and differences. This exploratory research used secondary data to provide background information of government policy and market conditions. Two case studies were use to illustrate similarities, trends, differences and to explore town planning, specific property characteristics including location, design and layout. Then, analyse whether these factors influence the performance and value of an industrial asset. The location of industrial properties varies between each country and depends heavily on infrastructure. It was noted that the town planning restrictions not only vary between markets and cities but also between property lots. The market conditions of both industrial markets were investigated and the supply and demand and rental levels in both cities were distinctly opposite.

Is implementing criterion-referenced assessment worth the effort with gen Y?

Well implemented criterion-referenced assessment (CRA) requires dedicated time and effort, especially in describing realistic expectations of evidence of achievement to students in the form of criteria sheets (or grading rubrics). It is also takes time out of delivering content to teach students how to judge their own work using criteria sheets. In 2007, to engage third year Microbiology students in using criteria sheets for the first time in their degree, we devised an innovative assessment tutorial supported by online resources. We were sceptical of much of the literature that reported ‘agreed’ characteristics of our predominantly gen Y cohort, because of the older ages of the majority of authors. These authors claim gen Y has a propensity for digital media, overconfidence in their own abilities and a collaborative orientation. We rejected this stereotype when developing the tutorial. Evaluations by students were positive and there was no dramatic change to grades for the unit. These results are similar to those in the literature for non gen Y cohorts. This lends support to our claim that giving students control over their own learning, irrespective of their generational label, is worth the time and effort.

Risk-assessment practices in the private rental sector : implications for low-income renters (Final Report)

This report focuses on risk-assessment practices in the private rental market, with particular consideration of their impact on low-income renters. It is based on the fieldwork undertaken in the second stage of the research process that followed completion of the Positioning Paper. The key research question this study addressed was: What are the various factors included in ‘risk-assessments’ by real estate agents in allocating ‘affordable’ tenancies? How are these risks quantified and managed? What are the key outcomes of their decision-making? The study builds on previous research demonstrating that a relatively large proportion of low-cost private rental accommodation is occupied by moderate- to high-income households (Wulff and Yates 2001; Seelig 2001; Yates et al. 2004). This is occurring in an environment where the private rental sector is now the de facto main provider of rental housing for lower-income households across Australia (Seelig et al. 2005) and where a number of factors are implicated in patterns of ‘income–rent mismatching’. These include ongoing shifts in public housing assistance; issues concerning eligibility for rent assistance; ‘supply’ factors, such as loss of low-cost rental stock through upgrading and/or transfer to owner-occupied housing; patterns of supply and demand driven largely by middle- to high-income owner-investors and renters; and patterns of housing need among low-income households for whom affordable housing is not appropriate. In formulating a way of approaching the analysis of ‘risk-assessment’ in rental housing management, this study has applied three sociological perspectives on risk: Beck’s (1992) formulation of risk society as entailing processes of ‘individualisation’; a socio-cultural perspective which emphasises the situated nature of perceptions of risk; and a perspective which has drawn attention to different modes of institutional governance of subjects, as ‘carriers of specific indicators of risk’. The private rental market was viewed as a social institution, and the research strategy was informed by ‘institutional ethnography’ as a method of enquiry. The study was based on interviews with property managers, real estate industry representatives, tenant advocates and community housing providers. The primary focus of inquiry was on ‘the moment of allocation’. Six local areas across metropolitan and regional Queensland, New South Wales, and South Australia were selected as case study localities. In terms of the main findings, it is evident that access to private rental housing is not just a matter of ‘supply and demand’. It is also about assessment of risk among applicants. Risk – perceived or actual – is thus a critical factor in deciding who gets housed, and how. Risk and its assessment matter in the context of housing provision and in the development of policy responses. The outcomes from this study also highlight a number of salient points: 1.There are two principal forms of risk associated with property management: financial risk and risk of litigation. 2. Certain tenant characteristics and/or circumstances – ability to pay and ability to care for the rented property – are the main factors focused on in assessing risk among applicants for rental housing. Signals of either ‘(in)ability to pay’ and/or ‘(in)ability to care for the property’ are almost always interpreted as markers of high levels of risk. 3. The processing of tenancy applications entails a complex and variable mix of formal and informal strategies of risk-assessment and allocation where sorting (out), ranking, discriminating and handing over characterise the process. 4. In the eyes of property managers, ‘suitable’ tenants can be conceptualised as those who are resourceful, reputable, competent, strategic and presentable. 5. Property managers clearly articulated concern about risks entailed in a number of characteristics or situations. Being on a low income was the principal and overarching factor which agents considered. Others included: - unemployment - ‘big’ families; sole parent families - domestic violence - marital breakdown - shift from home ownership to private rental - Aboriginality and specific ethnicities - physical incapacity - aspects of ‘presentation’. The financial vulnerability of applicants in these groups can be invoked, alongside expressed concerns about compromised capacities to manage income and/or ‘care for’ the property, as legitimate grounds for rejection or a lower ranking. 6. At the level of face-to-face interaction between the property manager and applicants, more intuitive assessments of risk based upon past experience or ‘gut feelings’ come into play. These judgements are interwoven with more systematic procedures of tenant selection. The findings suggest that considerable ‘risk’ is associated with low-income status, either directly or insofar as it is associated with other forms of perceived risk, and that such risks are likely to impede access to the professionally managed private rental market. Detailed analysis suggests that opportunities for access to housing by low-income householders also arise where, for example: - the ‘local experience’ of an agency and/or property manager works in favour of particular applicants - applicants can demonstrate available social support and financial guarantors - an applicant’s preference or need for longer-term rental is seen to provide a level of financial security for the landlord - applicants are prepared to agree to specific, more stringent conditions for inspection of properties and review of contracts - the particular circumstances and motivations of landlords lead them to consider a wider range of applicants - In particular circumstances, property managers are prepared to give special consideration to applicants who appear worthy, albeit ‘risky’. The strategic actions of demonstrating and documenting on the part of vulnerable (low-income) tenant applicants can improve their chances of being perceived as resourceful, capable and ‘savvy’. Such actions are significant because they help to persuade property managers not only that the applicant may have sufficient resources (personal and material) but that they accept that the onus is on themselves to show they are reputable, and that they have valued ‘competencies’ and understand ‘how the system works’. The parameters of the market do shape the processes of risk-assessment and, ultimately, the strategic relation of power between property manager and the tenant applicant. Low vacancy rates and limited supply of lower-cost rental stock, in all areas, mean that there are many more tenant applicants than available properties, creating a highly competitive environment for applicants. The fundamental problem of supply is an aspect of the market that severely limits the chances of access to appropriate and affordable housing for low-income rental housing applicants. There is recognition of the impact of this problem of supply. The study indicates three main directions for future focus in policy and program development: providing appropriate supports to tenants to access and sustain private rental housing, addressing issues of discrimination and privacy arising in the processes of selecting suitable tenants, and addressing problems of supply.

Harnessing assessment and feedback in the first year to support learning success, engagement and retention

It has been argued that intentional first year curriculum design has a critical role to play in enhancing first year student engagement, success and retention (Kift, 2008). A fundamental first year curriculum objective should be to assist students to make the successful transition to assessment in higher education. Scott (2006) has identified that ‘relevant, consistent and integrated assessment … [with] prompt and constructive feedback’ are particularly relevant to student retention generally; while Nicol (2007) suggests that ‘lack of clarity regarding expectations in the first year, low levels of teacher feedback and poor motivation’ are key issues in the first year. At the very minimum, if we expect first year students to become independent and self-managing learners, they need to be supported in their early development and acquisition of tertiary assessment literacies (Orrell, 2005). Critical to this attainment is the necessity to alleviate early anxieties around assessment information, instructions, guidance, and performance. This includes, for example:  inducting students thoroughly into the academic languages and assessment genres they will encounter as the vehicles for evidencing learning success; and  making expectations about the quality of this evidence clear. Most importantly, students should receive regular formative feedback of their work early in their program of study to aid their learning and to provide information to both students and teachers on progress and achievement. Leveraging research conducted under an ALTC Senior Fellowship that has sought to articulate a research-based 'transition pedagogy' (Kift & Nelson, 2005) – a guiding philosophy for intentional first year curriculum design and support that carefully scaffolds and mediates the first year learning experience for contemporary heterogeneous cohorts – this paper will discuss theoretical and practical strategies and examples that should be of assistance in implementing good assessment and feedback practices across a range of disciplines in the first year.

A re-assessment of the debate over sex education

The intention of this paper is to call into question the apparent dichotomy between left and right wing positions concerning the implementation of sex education in Queensland Schools. By adopting a more historicised approach, it is suggested that these positions are part of dual and supplementary strategies which work through specific problematisations associated with youth. Accordingly, the emphasis should be shifted from generalised depictions of the adolescent/youth to more focused accounts, questioning how teaching children to manage their sex became a governmental imperative.