Breaching the standardised assessment boundary : assessment for learning as a field of exchange

The introduction of the Australian curriculum, the use of standardised testing (e.g. NAPLAN) and the My School website are couched in a context of accountability. This circumstance has stimulated and in some cases renewed a range of boundaries in Australian Education. The consequences that arise from standardised testing have accentuated the boundaries produced by social reproduction in education which has led to an increase in the numbers of students disengaging from mainstream education and applying for enrolment at the Edmund Rice Education Australia Flexible Learning Centre Network (EREAFLCN). Boundaries are created for many young people who are denied access to credentials and certification as a result of being excluded from or in some way disengaging from standardised education and testing. Young people who participate at the EREAFLCN arrive with a variety of forms of cultural capital that are not valued in current education and employment fields. This is not to say that these young people’s different forms of cultural capital have no value, but rather that such funds of knowledge, repertoires and cultural capital are not valued by the majority of powerful agents in educational and employment fields. How then can the qualitative value of traditionally unorthodox - yet often intricate, ingenious, and astute - versions of cultural capital evident in the habitus of many young people be made to count, be recognised, be valuated? Can a process of educational assessment be a field of capital exchange and a space which breaches boundaries through a valuating process? This paper reports on the development of an innovative approach to assessment in an alternative education institution designed for the re-engagement of ‘at risk’ youth who have left formal schooling. A case study approach has been used to document the engagement of six young people, with an educational approach described as assessment for learning as a field of exchange across two sites in the EREAFLCN. In order to capture the broad range of students’ cultural and social capital, an electronic portfolio system (EPS) is under trial. The model draws on categories from sociological models of capital and reconceptualises the eportfolio as a sociocultural zone of learning and development. Results from the trial show a general tendency towards engagement with the EPS and potential for the attainment of socially valued cultural capital in the form of school credentials. In this way restrictive boundaries can be breached and a more equitable outcome achieved for many young Australians.

Towards general capabilities through technology : assessment for learning as a field of exchange

The introduction of the Australian curriculum, the use of standardised testing (e.g. NAPLAN) and the My School website have stimulated and in some cases renewed a range of boundaries for young people in Australian Education. Standardised testing has accentuated social reproduction in education with an increase in the numbers of students disengaging from mainstream education and applying for enrolment at the Edmund Rice Education Australia Flexible Learning Centre Network (EREAFLCN). Many young people are denied access to credentials and certification as they become excluded from standardised education and testing. The creativity and skills of marginalised youth are often evidence of general capabilities and yet do not appear to be recognised in mainstream educational institutions when standardised approaches are adopted. Young people who participate at the EREAFLCN arrive with a variety of forms of cultural capital, frequently utilising general capabilities, which are not able to be valued in current education and employment fields. This is not to say that these young people‟s different forms of cultural capital have no value, but rather that such funds of knowledge, repertoires and cultural capital are not valued by the majority of powerful agents in educational and employment fields. How then can the inherent value of traditionally unorthodox - yet often intricate, ingenious, and astute-versions of cultural capital evident in the habitus of many young people be made to count, be recognised, be valuated?Can a process of educational assessment be a field of capital exchange and a space which crosses boundaries through a valuating process? This paper reports on the development of an innovative approach to assessment in an alternative education institution designed for the re engagement of „at risk‟ youth who have left formal schooling. A case study approach has been used to document the engagement of six young people, with an educational approach described as assessment for learning as a field of exchange across two sites in the EREAFLCN. In order to capture the broad range of students‟ cultural and social capital, an electronic portfolio system (EPS) is under trial. The model draws on categories from sociological models of capital and reconceptualises the eportfolio as a sociocultural zone of learning and development. Results from the trial show a general tendency towards engagement with the EPS and potential for the attainment of socially valued cultural capital in the form of school credentials. In this way restrictive boundaries can be breached and a more equitable outcome achieved for many young Australians.

Turning histories into futures : assessment for learning as a field of exchange

Standardised testing does not recognise the creativity and skills of marginalised youth. Young people who come to the Edmund Rice Education Australia Flexible Learning Centre Network (EREAFLCN) in Australia arrive with forms of cultural capital that are not valued in the field of education and employment. This is not to say that young people‟s different modes of cultural capital have no value, but rather that such funds of knowledge, repertoires and cultural capital are not valued by the powerful agents in educational and employment fields. The forms of cultural capital which are valued by these institutions are measurable in certain structured formats which are largely inaccessible for what is seen in Australia to be a growing segment of the community. How then can the inherent value of traditionally unorthodox - yet often intricate, adroit, ingenious, and astute - versions of cultural capital evident in the habitus of many young people be made to count, be recognised, be valuated? Can a process of educational assessment be used as a marketplace, a field of capital exchange? This paper reports on the development of an innovative approach to assessment in an alternative education institution designed for the re-engagement of „at risk‟ youth who have left formal schooling. In order to capture the broad range of students‟ cultural and social capital, an electronic portfolio system (EPS) is under trial. The model draws on categories from sociological models of capital and reconceptualises the eportfolio as a sociocultural zone of learning and development. Initial results from the trial show a general tendency towards engagement with the EPS and potential for the attainment of socially valued cultural capital in the form of school credentials.

Mining a data reasoning model for personalized text classification

It is a big challenge to acquire correct user profiles for personalized text classification since users may be unsure in providing their interests. Traditional approaches to user profiling adopt machine learning (ML) to automatically discover classification knowledge from explicit user feedback in describing personal interests. However, the accuracy of ML-based methods cannot be significantly improved in many cases due to the term independence assumption and uncertainties associated with them. This paper presents a novel relevance feedback approach for personalized text classification. It basically applies data mining to discover knowledge from relevant and non-relevant text and constraints specific knowledge by reasoning rules to eliminate some conflicting information. We also developed a Dempster-Shafer (DS) approach as the means to utilise the specific knowledge to build high-quality data models for classification. The experimental results conducted on Reuters Corpus Volume 1 and TREC topics support that the proposed technique achieves encouraging performance in comparing with the state-of-the-art relevance feedback models.

ACE research briefing paper 002: learning and adaptation in the business creation process

This year marks the completion of data collection for year three (Wave 3) of the CAUSEE study. This report uses data from the first three years and focuses on the process of learning and adaptation in the business creation process. Most start-ups need to change their business model, their product, their marketing plan, their market or something else about the business to be successful. PayPal changed their product at least five times, moving from handheld security, to enterprise apps, to consumer apps, to a digital wallet, to payments between handhelds before finally stumbling on the model that made the a multi-billion dollar company revolving around email-based payments. PayPal is not alone and anecdotes abounds of start-ups changing direction: Sysmantec started as an artificial intelligence company, Apple started selling plans to build computers and Microsoft tried to peddle compilers before licensing an operating system out of New Mexico. To what extent do Australian new ventures change and adapt as their ideas and business develop? As a longitudinal study, CAUSEE was designed specifically to observe development in the venture creation process. In this research briefing paper, we compare development over time of randomly sampled Nascent Firms (NF) and Young Firms(YF), concentrating on the surviving cases. We also compare NFs with YFs at each yearly interval. The 'high potential' over sample is not used in this report.

New maps of learning for quality art education : what pre-service teachers should learn and be able to do

It would be a rare thing to visit an early years setting or classroom in Australia that does not display examples of young children’s artworks. This practice serves to give schools a particular ‘look’, but is no guarantee of quality art education. The Australian National Review of Visual Arts Education (NRVE) (2009) has called for changes to visual art education in schools. The planned new National Curriculum includes the arts (music, dance, drama, media and visual arts) as one of the five learning areas. Research shows that it is the classroom teacher that makes the difference, and teacher education has a large part to play in reforms to art education. This paper provides an account of one foundation unit of study (Unit 1) for first year university students enrolled in a 4-year Bachelor degree program who are preparing to teach in the early years (0–8 years). To prepare pre-service teachers to meet the needs of children in the 21st century, Unit 1 blends old and new ways of seeing art, child and pedagogy. Claims for the effectiveness of this model are supported with evidence-based research, conducted over the six years of iterations and ongoing development of Unit 1.

One more tier, no more tears: Students’ perceptions of QUT Library’s 2nd tier learning and study support

This nuts and bolts session discusses QUT Library’s Study Solutions service which is staffed by academic skills advisors and librarians as the 2nd tier of its learning and study support model. Firstly, it will discuss the rationale behind the Study Solutions model and provide a brief profile of the service. Secondly, it will outline what distinguishes it from other modes of one-to-one learning support. Thirdly, it will report findings from a student perception study conducted to determine what difference this model of individual study assistance made to academic confidence, ability to transfer academic skills and capacity to assist peers. Finally, this session will include small group discussions to consider the feasibility of this model as best practice for other tertiary institutions and student perception as a valuable measure of the impact of learning support services.

Challenges of Measuring Learning Outcomes for Property Students Engaged in Work Integrated Learning

Work integrated learning (WIL) or professional practice units are recognised as providing learning experiences that help students make successful transitions to professional practice. These units require students to engage in learning in the workplace; to reflect on this learning; and to integrate it with learning at university. However, an analysis of a recent cohort of property economics students at a large urban university provides evidence that there is great variation in work based learning experiences undertaken and that this impacts on students’capacity to respond to assessment tasks which involve critiquing these experiences in the form of reflective reports. This paper highlights the need to recognise the diversity of work based experiences; the impact this has on learning outcomes; and to find more effective and equitable ways of measuring these outcomes. The paper briefly discusses assessing learning outcomes in WIL and then describes the model of WIL in the Faculty of Built Environment and Engineering at the Queensland University of Technology (QUT). The paper elaborates on the diversity of students’ experiences and backgrounds including variations in the length of work experience, placement opportunities and conditions of employment.For example, the analysis shows that students with limited work experience often have difficulty critiquing this work experience and producing high level reflective reports. On the other hand students with extensive, discipline relevant work experience can be frustrated by assessment requirements that do not take their experience into account. Added to this the Global Financial Crisis (GFC) has restricted both part time and full time placement opportunities for some students. These factors affect students’ capacity to a) secure a relevant work experience, b) reflect critically on the work experiences and c) appreciate the impact the overall experience can have on their learning outcomes and future professional opportunities. Our investigation highlights some of the challenges faced in implementing effective and equitable approaches across diverse student cohorts. We suggest that increased flexibility in assessment requirements and increased feedback from industry may help address these challenges.

An exploration of the maturity model concept as a vehicle for higher education institutions to assess their capability to address student engagement : a work in progress

Gaining a competitive edge in the area of the engagement, success and retention of commencing students is a significant issue in higher education, made more so currently because of the considerable and increasing pressure on teaching and learning from the new standards framework and performance funding. This paper introduces the concept of maturity models (MMs) and their application to assessing the capability of higher education institutions (HEIs) to address student engagement, success and retention (SESR). A concise description of the features of maturity models is presented with reference to an SESR-MM currently being developed. The SESR-MM is proposed as a viable instrument for assisting HEIs in the management and improvement of their SESR activities.

The maturity model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention : new horizon or false dawn?

While the engagement, success and retention of first year students are ongoing issues in higher education, they are currently of considerable and increasing importance as the pressures on teaching and learning from the new standards framework and performance funding intensifies. This Nuts & Bolts presentation introduces the concept of a maturity model and its application to the assessment of the capability of higher education institutions to address student engagement, success and retention. Participants will be provided with (a) a concise description of the concept and features of a maturity model; and (b) the opportunity to explore the potential application of maturity models (i) to the management of student engagement and retention programs and strategies within an institution and (ii) to the improvement of these features by benchmarking across the sector.

A comparison of methods for classifying clinical samples based on proteomics data : a case study for statistical and machine learning approaches

The discovery of protein variation is an important strategy in disease diagnosis within the biological sciences. The current benchmark for elucidating information from multiple biological variables is the so called “omics” disciplines of the biological sciences. Such variability is uncovered by implementation of multivariable data mining techniques which come under two primary categories, machine learning strategies and statistical based approaches. Typically proteomic studies can produce hundreds or thousands of variables, p, per observation, n, depending on the analytical platform or method employed to generate the data. Many classification methods are limited by an n≪p constraint, and as such, require pre-treatment to reduce the dimensionality prior to classification. Recently machine learning techniques have gained popularity in the field for their ability to successfully classify unknown samples. One limitation of such methods is the lack of a functional model allowing meaningful interpretation of results in terms of the features used for classification. This is a problem that might be solved using a statistical model-based approach where not only is the importance of the individual protein explicit, they are combined into a readily interpretable classification rule without relying on a black box approach. Here we incorporate statistical dimension reduction techniques Partial Least Squares (PLS) and Principal Components Analysis (PCA) followed by both statistical and machine learning classification methods, and compared them to a popular machine learning technique, Support Vector Machines (SVM). Both PLS and SVM demonstrate strong utility for proteomic classification problems.

Pragmatics, perceptions and strategies in Chinese college English learning

This study investigated Chinese College English students. perceptions of pragmatics, their pragmatic competence in selected speech acts, strategies they employed in acquiring pragmatic knowledge, as well as their general approach to learning English as a foreign language. The research was triggered by a national curriculum initiative that prioritizes the need for College English students to enhance their ability to use English effectively in different social interactions (Chinese College English Education and Supervisory Committee, 2007). The traditional "grammar-translation" and "examination-oriented" method is believed to have reduced Chinese College English students to what is dubbed "mute" and "deaf" language learners (Zhang, 2008; Zhao, 2009). Many students lack pragmatic knowledge on how to interpret discourse by relating utterances to their meanings, understanding the intention of language users, and how language is used in specific settings (Bachman & Palmer, 1996, 2010). There is an increasing body of literature on awareness-raising of the importance of pragmatic knowledge and strategies for classroom instruction. However, to date, researchers have tended to focus largely on the teaching of pragmatics, rather than on how students acquire pragmatic competence (Bardovi-Harlig & Dornyei, 1998; Du, 2004; Hou, 2007; Ruan, 2007; Schauer, 2009). It is this gap in the research that this study fills, with a focus on different types of pragmatic knowledge, learner perceptions of such knowledge, and learning strategies that College English students employ in the process of learning English in general, and pragmatics in particular. Three strands of theories of second language acquisition (Ellis, 1985, 1994): pragmatics (Levinson, 1983; Mey, 2001; Yule, 1996), intercultural communications (Kramsch, 1998; Samovar & Porter, 1997; Samovar, Porter & McDaniel, 2009) and English as a lingua franca (ELF) (Canagarajah, 2006; Firth, 1996; Pennycook, 2010) were employed to establish a conceptual framework for data collection and analyses. Key constructs derived from the three related theories helped to form a typology for a detailed examination and theorization of the empirical evidence gathered from different sources. Four research instruments: a questionnaire (N=237), Discourse Completion Tasks (DCTs) (N=55), focus group interviews (N=18), and a textbook tasks analysis were employed to collect data for this systematic inquiry. Data collected by different instruments were analyzed and compared by way of a triangulation to enhance its validity and reliability. Major findings derived from different sources highlighted that, although College English students were grammatically advanced language learners, they displayed limited pragmatic knowledge and a highly restricted repertoire of language learning strategies. The majority of the respondents, however, believed that pragmatic knowledge was as important as linguistic knowledge in the process of developing communicative competence for interaction in different contexts. It was argued that a combination of a less than sufficient English proficiency, limited knowledge of pragmatics, inadequate language materials and tasks, and a small stock of language learning strategies, were a major hindrance to effective learning and communication, resulting in pragmatic failures in many intercultural communication situations. As the first systematic study of how Chinese College English students learned pragmatics, the research provided a solid empirical base for developing a tentative model for the learning of pragmatics in a College English classroom in China and similar educational contexts. The model was strengthened by a unique combination of theories of pragmatics, intercultural communication and ELF. Findings from this research provided insights into how Chinese College English students perceived pragmatics in the English as foreign language (EFL) curriculum, the processes of learning, as well as strategies they utilized in developing linguistic and pragmatic knowledge and competence.

Verifying a Student Engagement, Success and Retention Maturity Model (SESR-MM): A reality check of the model using data derived from three universities

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). We have proposed that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities and we are currently articulating a Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012). Our research aims to address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. Our research extends the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). Our research also moves beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. The issues explored in this workshop are (i) whether the maturity model concept can be usefully applied to provide a measure of institutional capability for SESR; (ii) whether the SESR-MM can be used to assess the maturity of a particular set of institutional practices; and (iii) whether a collective assessment of an institution’s SESR capabilities can provide an indication of the maturity of the institution’s SESR activities. The workshop will be approached in three stages. Firstly, participants will be introduced to the key characteristics of maturity models, followed by a discussion of the SESR-MM and the processes involved in its development. Secondly, participants will be provided with resources to facilitate the development of a maturity model and an assessment instrument for a range of institutional processes and related practices. In the final stage of the workshop, participants will “assess” the capability of these practices to provide a collective assessment of the maturity of these processes. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf

A model for curricula integration using the Australian curriculum

Targeting students’ learning is at the centre of education. In addition, education is promoted as a solution on various issues; consequently educators seek ways for teachers to address societal needs, students’ learning needs, and the overcrowded curriculum. There are definition debates and issues around integrating curricula. However, the rationale for primary students undertaking curricula integrated learning can provide motivation for primary teachers to devise and implement curricula integrated lessons in the classroom. More exploration is required to present models for the practical implementation of curricula integration. This paper provides practical ideas for curricula integration that focus on combining achievement standards from the Australian Curriculum: Science and other key learning areas.