Australian higher education evaluation through assurance of learning

A collaborative research project conducted by five Australian universities inquired into the philosophy and motivation for Assurance of Learning (AoL) as a process of education evaluation. Associate Deans Teaching and Learning representing Business schools from twenty-five universities across Australia participated in telephone interviews. Data was analysed using NVIVO9. Results indicated that articulated rationale for AoL was both ensuring that students had acquired the attributes and skills the universities claimed they had, and the philosophy of continuous improvement. AoL was motivated both by ritualistic objectives to satisfy accreditation requirements and virtuous agendas for quality improvement. Closing-the-loop was emphasised, but was mostly wishful thinking for next steps beyond data collection and reporting. AoL was conceptualised as one element within the larger context of quality review, but there was no evidence of comprehensive frameworks or strategic plans.

Assurance of learning – are academics and senior management singing from the same hymn sheet?

Assurance of learning (AoL) is an important process in quality education, designed to measure the accomplishment of educational aims at the core of an institution’s programs, whilst encouraging faculty to continuously develop and improve the programs and courses. This paper reports on a study of Australian business schools to investigate current AoL practices through semi structured interviews with senior faculty leaders followed by focus group interviews with groups of senior program leaders and groups of academic teaching staff. Initial findings indicate there are significant challenges in encouraging academic staff to commit to the process and recognise the benefits of assuring learning. The differences in understanding between the various leaders and the academics were highlighted through the different focus groups. Leaders’ stressed strategic issues such as staff engagement and change, while academics focussed on process issues such as teaching graduate attributes and external accreditation. Understanding the differences in the perspectives of leaders and faculty is important, as without a shared understanding between the two groups, there is likely to be limited engagement, which creates difficulties in developing effective assurance of learning processes. Findings indicate that successful strategies developed to foster shared values on assurance of learning include: strong senior leaders’ commitment; developing champions among program and unit level staff; providing professional development opportunities; promoting and celebrating success and effectiveness; and ensuring an inclusive process with academics of all levels collaborating in the development and implementation of the process.

Enabling innovation and facilitating learning in kifs: the role of HRM practices

The aim of this paper is to contribute to our understanding of the link between HR practices, learning orientation and types of innovation in knowledge-intensive firms (KIFs). To this end, we first compiled a comprehensive overview of the existing literature on HR practices aimed at supporting innovation. On the basis of this literature, we then collected and analyzed data from a qualitative study of 19 Danish KIFs recognized for their innovation performance, focusing on links between the HR practices they use to support exploratory and exploitive learning behaviors to enhance incremental and/or radical innovation. The findings from this study demonstrate that KIFs utilize a range of HR practices that enable different learning orientations, based on the type of innovation compatible with their organizational goals.

Learning communities learning series

In 2008 the introduction of the National Assessment Program – Literacy and Numeracy (NAPLAN), combined with the publication of the international comparative analyses of student achievement data (such as the Programme of International Student Assessment (PISA) developed by the Organisation for Economic Co-operation and Development (OECD) and the Trends in International Mathematics and Science Study (TIMSS) of the International Association for the Evaluation of Educational Achievement (IEA)) highlighted a significant priority for Australian education by identifying low levels of equity.

Multimodal and monomodal discourses of marketization in higher education : power, ideology, and the absence of the image

The ubiquity of multimodality in hypermedia environments is undeniable. Bezemer and Kress (2008) have argued that writing has been displaced by image as the central mode for representation. Given the current technical affordances of digital technology and user-friendly interfaces that enable the ease of multimodal design, the conspicuous absence of images in certain domains of cyberspace is deserving of critical analysis. In this presentation, I examine the politics of discourses implicit within hypertextual spaces, drawing textual examples from a higher education website. I critically examine the role of writing and other modes of production used in what Fairclough (1993) refers to as discourses of marketisation in higher education, tracing four pervasive discourses of teaching and learning in the current economy: i) materialization, ii) personalization, iii) technologisation, and iv) commodification (Fairclough, 1999). Each of these arguments is supported by the critical analysis of multimodal texts. The first is a podcast highlighting the new architectonic features of a university learning space. The second is a podcast and transcript of a university Open Day interview with prospective students. The third is a time-lapse video showing the construction of a new science and engineering precinct. These three multimodal texts contrast a final web-based text that exhibits a predominance of writing and the powerful absence or silencing of the image. I connect the weightiness of words and the function of monomodality in the commodification of discourses, and its resistance to the multimodal affordances of web-based technologies, and how this is used to establish particular sets of subject positions and ideologies through which readers are constrained to occupy. Applying principles of critical language study by theorists that include Fairclough, Kress, Lemke, and others whose semiotic analysis of texts focuses on the connections between language, power, and ideology, I demonstrate how the denial of image and the privileging of written words in the multimodality of cyberspace is an ideological effect to accentuate the dominance of the institution.

Meaningful musical engagement : the potential impact of generative music systems on learning experiences

In order to create music, the student must establish a relationship with the musical materials. In this thesis, I examine the capacity of a generative music system called jam2jam to offer individuals a virtual musical play-space to explore. I outline the development of an iteration of software development named jam2jam blue and the evolution of a games-like user interface in the research design that jointly revealed the nature of this musical exploration. The findings suggest that the jam2jam blue interface provided an expressive gestural instrument to jam and experience musicmaking. By using the computer as an instrument, participants in this study were given access to meaningful musical experiences in both solo and ensemble situations and the researcher is allowed a view of their development of a relationship with the musical materials from the perspective of the individual participants. Through an iterative software development methodology, pedagogy and experience design were created simultaneously. The research reveals the potential for the jam2jam software to be used as a reflective tool for feedback and assessment purposes. The power of access to ensemble music making is realised though the participants’ virtual experiences which are brought into their physical space by sharing their experience with others. It is suggested that this interaction creates an environment conducive to self-initiated learning in which music is the language of interaction. The research concludes that the development of a relationship between the explorer and the musical materials is subject to the collaborative nature of the interaction through which the music is experienced.

Pragmatics, perceptions and strategies in Chinese college English learning

This study investigated Chinese College English students. perceptions of pragmatics, their pragmatic competence in selected speech acts, strategies they employed in acquiring pragmatic knowledge, as well as their general approach to learning English as a foreign language. The research was triggered by a national curriculum initiative that prioritizes the need for College English students to enhance their ability to use English effectively in different social interactions (Chinese College English Education and Supervisory Committee, 2007). The traditional "grammar-translation" and "examination-oriented" method is believed to have reduced Chinese College English students to what is dubbed "mute" and "deaf" language learners (Zhang, 2008; Zhao, 2009). Many students lack pragmatic knowledge on how to interpret discourse by relating utterances to their meanings, understanding the intention of language users, and how language is used in specific settings (Bachman & Palmer, 1996, 2010). There is an increasing body of literature on awareness-raising of the importance of pragmatic knowledge and strategies for classroom instruction. However, to date, researchers have tended to focus largely on the teaching of pragmatics, rather than on how students acquire pragmatic competence (Bardovi-Harlig & Dornyei, 1998; Du, 2004; Hou, 2007; Ruan, 2007; Schauer, 2009). It is this gap in the research that this study fills, with a focus on different types of pragmatic knowledge, learner perceptions of such knowledge, and learning strategies that College English students employ in the process of learning English in general, and pragmatics in particular. Three strands of theories of second language acquisition (Ellis, 1985, 1994): pragmatics (Levinson, 1983; Mey, 2001; Yule, 1996), intercultural communications (Kramsch, 1998; Samovar & Porter, 1997; Samovar, Porter & McDaniel, 2009) and English as a lingua franca (ELF) (Canagarajah, 2006; Firth, 1996; Pennycook, 2010) were employed to establish a conceptual framework for data collection and analyses. Key constructs derived from the three related theories helped to form a typology for a detailed examination and theorization of the empirical evidence gathered from different sources. Four research instruments: a questionnaire (N=237), Discourse Completion Tasks (DCTs) (N=55), focus group interviews (N=18), and a textbook tasks analysis were employed to collect data for this systematic inquiry. Data collected by different instruments were analyzed and compared by way of a triangulation to enhance its validity and reliability. Major findings derived from different sources highlighted that, although College English students were grammatically advanced language learners, they displayed limited pragmatic knowledge and a highly restricted repertoire of language learning strategies. The majority of the respondents, however, believed that pragmatic knowledge was as important as linguistic knowledge in the process of developing communicative competence for interaction in different contexts. It was argued that a combination of a less than sufficient English proficiency, limited knowledge of pragmatics, inadequate language materials and tasks, and a small stock of language learning strategies, were a major hindrance to effective learning and communication, resulting in pragmatic failures in many intercultural communication situations. As the first systematic study of how Chinese College English students learned pragmatics, the research provided a solid empirical base for developing a tentative model for the learning of pragmatics in a College English classroom in China and similar educational contexts. The model was strengthened by a unique combination of theories of pragmatics, intercultural communication and ELF. Findings from this research provided insights into how Chinese College English students perceived pragmatics in the English as foreign language (EFL) curriculum, the processes of learning, as well as strategies they utilized in developing linguistic and pragmatic knowledge and competence.

Contextualising learning for a real-world university: how an inverted curriculum in the first year can help better student retention

An identified issue within higher education is the high rates of student attrition after the first year, especially in the STEM disciplines. To address this issue, it is essential to reexamine and redesign the first year curriculum to engage and retain the students' interests while also scaffolding their learning experience. This session reports on an initiative based on the principles of the “inverted curriculum” within the Bachelor of Technology (BIT) course at the Queensland University of Technology (QUT) that began in 2009 and has resulted in a reduction in first-year attrition rates from 18% in 2008 to 10% in 2009 and 2010 despite a growth in student intake of 15% to 40% in the past two years. We present the process and methods that helped achieve this and initiate a discussion on the innovations that are possible within this concept of inverted curriculum and how it can be implemented.

Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Assessment of flood-related recharge to alluvial aquifers of an irrigated catchment following extended drought conditions using 3D visualisation and environmental tracers, Lockyer Valley, southeast Queensland, Australia.

The Lockyer Valley in southeast Queensland, Australia, hosts an economically significant alluvial aquifer system which has been impacted by prolonged drought conditions (~1997 to ~ 2009). Throughout this time, the system was under continued groundwater extraction, resulting in severe aquifer depletion. By 2008, much of the aquifer was at <30% of storage but some relief occurred with rains in early 2009. However, between December 2010 and January 2011, most of southeast Queensland experienced unprecedented flooding, which generated significant aquifer recharge. In order to understand the spatial and temporal controls of groundwater recharge in the alluvium, a detailed 3D lithological property model of gravels, sands and clays was developed using GOCAD software. The spatial distribution of recharge throughout the catchment was assessed using hydrograph data from about 400 groundwater observation wells screened at the base of the alluvium. Water levels from these bores were integrated into a catchment-wide 3D geological model using the 3D geological modelling software GOCAD; the model highlights the complexity of recharge mechanisms. To support this analysis, groundwater tracers (e.g. major and minor ions, stable isotopes, 3H and 14C) were used as independent verification. The use of these complementary methods has allowed the identification of zones where alluvial recharge primarily occurs from stream water during episodic flood events. However, the study also demonstrates that in some sections of the alluvium, rainfall recharge and discharge from the underlying basement into the alluvium are the primary recharge mechanisms of the alluvium. This is indicated by the absence of any response to the flood, as well as the observed old radiocarbon ages and distinct basement water chemistry signatures at these locations. Within the 3D geological model, integration of water chemistry and time-series displays of water level surfaces before and after the flood suggests that the spatial variations of the flood response in the alluvium are primarily controlled by the valley morphology and lithological variations within the alluvium. The integration of time-series of groundwater level surfaces in the 3D geological model also enables the quantification of the volumetric change of groundwater stored in the unconfined sections of this alluvial aquifer during drought and following flood events. The 3D representation and analysis of hydraulic and recharge information has considerable advantages over the traditional 2D approach. For example, while many studies focus on singular aspects of catchment dynamics and groundwater-surface water interactions, the 3D approach is capable of integrating multiple types of information (topography, geological, hydraulic, water chemistry and spatial) into a single representation which provides valuable insights into the major factors controlling aquifer processes.

Team teaching : working together to promote student learning

This book chapter examines the concept of team teaching from the perspective of the various stakeholders, in order to discuss the advantages and disadvantages of team teaching for students, to consider the positive and negative dimensions of collaborative teaching for teachers, and to review the implications for educational administration. In addition, attention will be paid to the issues associated with team teaching in the context of e-learning. The chapter concludes with a case study which discusses how the implementation of collaborative teaching within the library and information science discipline at an Australian university helped develop the authors’ understanding of socially constructed knowledge.

Human–computer interaction : the impact of users’ cognitive styles on query reformulation behaviour during web searching

This paper discusses users’ query reformulation behaviour while searching information on the Web. Query reformulations have emerged as an important component of Web search behaviour and human-computer interaction (HCI) because a user’s success of information retrieval (IR) depends on how he or she formulates queries. There are various factors, such as cognitive styles, that influence users’ query reformulation behaviour. Understanding how users with different cognitive styles formulate their queries while performing Web searches can help HCI researchers and information systems (IS) developers to provide assistance to the users. This paper aims to examine the effects of users’ cognitive styles on their query reformation behaviour. To achieve the goal of the study, a user study was conducted in which a total of 3613 search terms and 872 search queries were submitted by 50 users who engaged in 150 scenario-based search tasks. Riding’s (1991) Cognitive Style Analysis (CSA) test was used to assess users’ cognitive style as wholist or analytic, and verbaliser or imager. The study findings show that users’ query reformulation behaviour is affected by their cognitive styles. The results reveal that analytic users tended to prefer Add queries while all other users preferred New queries. A significant difference was found among wholists and analytics in the manner they performed Remove query reformulations. Future HCI researchers and IS developers can utilize the study results to develop interactive and user-cantered search model, and to provide context-based query suggestions for users.