Sampling perceptions; testing reality : an evidence-based approach to measurably improve information literacy and student research skills

Do commencing students possess the level of information literacy (IL) knowledge and skills they need to succeed at university? What impact does embedding IL within the engineering and design curriculum have? This paper reports on the self-perception versus the reality of IL knowledge and skills, across a large cohort of first year built environment and engineering students. Acting on the findings of this evaluation, the authors (a team of academic librarians) developed an intensive IL skills program which was integrated into a faculty wide unit. Perceptions, knowledge and skills were re-evaluated at the end of the semester to determine if embedded IL education made a difference. Findings reveal that both the perception and reality of IL skills were significantly and measurably improved.

Rethinking public service media and citizenship: digital strategies for news and current affairs at Australia’s Special Broadcasting Service

This article considers the concept of media citizenship in relation to the digital strategies of the Special Broadcasting Service (SBS). At SBS, Australia’s multicultural public broadcaster, there is a critical appraisal of its strategies to harness user-created content (UCC) and social media to promote greater audience participation through its news and current affairs Web sites. The article looks at the opportunities and challenges that user-created content presents for public service media organizations as they consolidate multiplatform service delivery. Also analyzed are the implications of radio and television broadcasters’ moves to develop online services. It is proposed that case study methodologies enable an understanding of media citizenship to be developed that maintains a focus on the interaction between delivery technologies, organizational structures and cultures, and program content that is essential for understanding the changing focus of 21st-century public service media.

Faculty mentoring, evidence-based assessment, and student learning : an Australian exploration of American initiatives

There have been many improvements in Australian engineering education since the 1990s. However, given the recent drive for assuring the achievement of identified academic standards, more progress needs to be made, particularly in the area of evidence-based assessment. This paper reports on initiatives gathered from the literature and engineering academics in the USA, through an Australian National Teaching Fellowship program. The program aims to establish a process to help academics in designing and implementing evidence-based assessments that meet the needs of not only students and the staff that teach them, but also industry as well as accreditation bodies. The paper also examines the kinds and levels of support necessary for engineering academics, especially early career ones, to help meet the expectations of the current drive for assured quality and standards of both research and teaching. Academics are experiencing competing demands on their time and energy with very high expectations in research performance and increased teaching responsibilities, although many are researchers who have not had much pedagogic training. Based on the literature and investigation of relevant initiatives in the USA, we conducted interviews with several identified experts and change agents who have wrought effective academic cultural change within their institutions and beyond. These reveal that assuring the standards and quality of student learning outcomes through evidence-based assessments cannot be appropriately addressed without also addressing the issue of pedagogic training for academic staff. To be sustainable, such training needs to be complemented by a culture of on-going mentoring support from senior academics, formalised through the university administration, so that mentors are afforded resources, time, and appropriate recognition.

The introduction of an Inter-professional learning unit: staff and student perspectives

Background The School of Clinical Sciences comprises a number of health disciplines including podiatry, paramedic science, pharmacy, medical imaging and radiation therapy. A new inter-professional unit was introduced in 2014, which covered key introductory learnings applicable for future health practitioners. This study examined teaching staff and student perspectives about their experience with the new unit for first year students. Methods Qualitative interviews with teaching staff (n=9) and focus group interviews with students (5 groups which ranged in size from 4-30) were conducted. Extensive notes were taken during the interviews Issues emerging from the interviews were identified and organised according to themes and subthemes. Results Four major themes were identified namely: Something new; To be or not to be that is the question; Advantages of the new unit; and Areas for improvement. Previous staff experience with inter-professional learning (IPL) had been ad-hoc, whereas the new unit brought together several disciplines in a planned and deliberate way. There was strong philosophical agreement about the value of IPL but some debate about the extent to which the unit provided IPL experience. The unit was seen as assisting students’ social and academic adjustment to university and provided opportunity for professional socialisation, exposure to macro and micro aspects of the Australian health care system and various types of communication. For podiatry students it was their first opportunity to formally meet and work with other podiatry students and moved their identity from ‘university student’ to ‘podiatry student’. Other positives included providing the opportunity for staff and students to interact at an early stage with the perceived benefit of reducing attrition. Areas for unit improvement included institutional arrangements, unit administration aspects and assessment. Conclusion The unit was seen as beneficial by staff and students however, students were more polarised in their views than staff. There was a tension between feeling apart of and learning about one's own profession and feeling apart of and learning about the roles of other health professionals in relation to patient care and the health care system.

'Take away from the dry sixties style marking': Lecturer and student perceptions and experiences of audio feedback

Providing audio feedback to assessment is relatively uncommon in higher education. However, published research suggests that it is preferred over written feedback by students but lecturers were less convinced. The aim of this paper is to examine further these findings in the context of a third year business ethics unit. Data was collected from two sources. The first is a series of in-depth, semi-structured interviews conducted with three lecturers providing audio feeback for the first time in Semester One 2011. The second source of data was drawn from the university student evaluation system. A total of 363 responses were used providing 'before' and 'after' perspectives about the effectiveness of audio feedback versus written feedback. Between 2005 and 2009 the survey data provided information about student attitudes to written assessment feedback (n=261). From 2010 onwards the data relates to audio (mp3) feedback (n=102). The analysis of he interview data indicated that introducing audio feedback should be done with care. The perception of the participating lecturers was mixed, ranging from sceptism to outright enthusiasm, but over time the overall approach became positive. It was found that particular attention needs to be paid to small (but important) technical details, and lecturers need to be convinced of its effectieness, especially that it is not necessarily more time consuming than providing written feedback. For students, the analysis revealed a clear preference for audio feedback. It is concluded that there is cause for concern and reason for optimism. It is a cause for concern because there is a possibility that scepticism on the part of academic staff seems to be based on assumptions about what students prefer and a concern about using the technology. There is reason for optimism because the evidence points towards students preferring audio feedback and as academic staff become more familiar with the technology the scepticism tends to evaporate. While this study is limited in scope, questions are raised about tackling negative staff perceptions of audio feedback that are worthy of further research.

Transforming higher education and student engagement through collaborative review to inform educational design

This paper reports on staff perceptions arising from a review process designed to assist staff in making informed decisions regarding educational design, approaches to engage students in learning, and the technology to support engagement in the classroom and across multiple locations and delivery modes. The aim of the review process was to transform the level of student engagement in the business faculty of an Australian university. The process took a collaborative approach through consultation with academic staff involved in the design and delivery of the units under review, and included targeted professional development as necessary. An institutional framework that characterises engagement indicator contexts and their attributes facilitated dialog during the review process. This paper reports on a mixed method study that included a survey of participants, and purposeful interviews to evaluate the effectiveness of the process. Although the study identified factors that hindered implementation and operationalization of review recommendations in some instances, study participants were generally of the view that recommendations would enhance student engagement. It is demonstrated that the bottom-up approach described in this paper is consistent with theoretical frameworks for transformational change in teaching and learning and the adoption of innovations.

Authentic and conventional assessment in Singapore schools : an empirical study of teacher assignments and student work

This is an empirical examination of the quality of teacher assignments and student work in Singapore schools. Using a theoretical framework based on principles of authentic assessment and intellectual quality, two sets of criteria and scoring rubrics were developed for the training of expert teachers to judge the quality of assignments and student work. Following rigorous training, the inter-rater reliability of expert teacher scoring was high. Samples of teacher assignments and student work were collected in English, social studies, mathematics, and science subject areas from a random stratified sample of 30 elementary schools and 29 high schools. For both grade levels, there were significant differences for the authentic intellectual quality of teachers’ assignments by subject area. Likewise, the differences of authentic intellectual quality for student work were significant and varied by subject area. Subject area effect was large. The correlations between the quality of teachers’ assignment tasks and student work were strong and significant at both grade levels. Where teachers set more intellectually demanding tasks, students were more likely to generate work or artefacts judged to be of higher quality. The findings suggest that teacher professional development in authentic intellectual assessment task design can contribute to the improvement of student learning and performance. It is argued that this will be a key requisite of educational systems like Singapore that are seeking to expand pedagogy and student outcomes beyond a focus on factual and rote knowledge.

Rethinking public service media and citizenship : digital strategies for news and current affairs at Australia’s Special Broadcasting Service (SBS)

This article considers the concept of media citizenship in relation to the digital strategies of the Special Broadcasting Service (SBS). At SBS, Australia’s multicultural public broadcaster, there is a critical appraisal of its strategies to harness user-created content (UCC) and social media to promote greater audience participation through its news and current affairs Web sites. The article looks at the opportunities and challenges that user-related content presents for public service media organizations as they consolidate multiplatform service delivery. Also analyzed are the implications of radio and television broadcasters’ moves to develop online services. It is proposed that case study methodologies enable an understanding of media citizenship to be developed that maintains a focus on the interaction between delivery technologies, organizational structures and cultures, and program content that is essential for understanding the changing focus of 21st-century public service media.

Student engagement with an ePortfolio : a case study of pre-service education students

The emergence of ePortfolios is relatively recent in the university sector as a way to engage students in their learning and assessment, and to produce records of their accomplishments. An ePortfolio is an online tool that students can utilise to record, catalogue, retrieve and present reflections and artefacts that support and demonstrate the development of graduate students’ capabilities and professional standards across university courses. The ePortfolio is therefore considered as both process and product. Although ePortfolios show promise as a useful tool and their uptake has grown, they are not yet a mainstream higher education technology. To date, the emphasis has been on investigating their potential to support the multiple purposes of learning, assessment and employability, but less is known about whether and how students engage with ePortfolios in the university setting. This thesis investigates student engagement with an ePortfolio in one university. As the educational designer for the ePortfolio project at the University, I was uniquely positioned as a researching professional to undertake an inquiry into whether students were engaging with the ePortfolio. The participants in this study were a cohort (defined by enrolment in a unit of study) of second and third year education students (n=105) enrolled in a four year Bachelor of Education degree. The students were introduced to the ePortfolio in an introductory lecture and a hands-on workshop in a computer laboratory. They were subsequently required to complete a compulsory assessment task – a critical reflection - using the ePortfolio. Following that, engagement with the ePortfolio was voluntary. A single case study approach arising from an interpretivist paradigm directed the methodological approach and research design for this study. The study investigated the participants’ own accounts of their experiences with the ePortfolio, including how and when they engaged with the ePortfolio and the factors that impacted on their engagement. Data collection methods consisted of an attitude survey, student interviews, document collection, a researcher reflective journal and researcher observations. The findings of the study show that, while the students were encouraged to use the ePortfolio as a learning and employability tool, most students ultimately chose to disengage after completing the assessment task. Only six of the forty-five students (13%) who completed the research survey had used the ePortfolio in a sustained manner. The data obtained from the students during this research has provided insight into reasons why they disengaged from the ePortfolio. The findings add to the understandings and descriptions of student engagement with technology, and more broadly, advance the understanding of ePortfolios. These findings also contribute to the interdisciplinary field of technology implementation. There are three key outcomes from this study, a model of student engagement with technology, a set of criteria for the design of an ePortfolio, and a set of recommendations for effective practice for those implementing ePortfolios. The first, the Model of Student Engagement with Technology (MSET) (Version 2) explored student engagement with technology by highlighting key engagement decision points for students The model was initially conceptualised by building on work of previous research (Version 1), however, following data analysis a new model emerged, MSET (Version 2). The engagement decision points were identified as: • Prior Knowledge and Experience, leading to imagined usefulness and imagined ease of use; • Initial Supported Engagement, leading to supported experience of usefulness and supported ease of use; • Initial Independent Engagement, leading to actual experience of independent usefulness and actual ease of use; and • Ongoing Independent Engagement, leading to ongoing experience of usefulness and ongoing ease of use. The Model of Student Engagement with Technology (MSET) goes beyond numerical figures of usage to demonstrate student engagement with an ePortfolio. The explanatory power of the model is based on the identification of the types of decisions that students make and when they make them during the engagement process. This model presents a greater depth of understanding student engagement than was previously available and has implications for the direction and timing of future implementation, and academic and student development activities. The second key outcome from this study is a set of criteria for the re-conceptualisation of the University ePortfolio. The knowledge gained from this research has resulted in a new set of design criteria that focus on the student actions of writing reflections and adding artefacts. The process of using the ePortfolio is reconceptualised in terms of privileging student learning over administrative compliance. The focus of the ePortfolio is that the writing of critical reflections is the key function, not the selection of capabilities. The third key outcome from this research consists of five recommendations for university practice that have arisen from this study. They are that, sustainable implementation is more often achieved through small steps building on one another; that a clear definition of the purpose of an ePortfolio is crucial for students and staff; that ePortfolio pedagogy should be the driving force not the technology; that the merit of the ePortfolio is fostered in students and staff; and finally, that supporting delayed task performance is crucial. Students do not adopt an ePortfolio just because it is provided. While students must accept responsibility for their own engagement with the ePortfolio, the institution has to accept responsibility for providing the environment, and technical and pedagogical support to foster engagement. Ultimately, an ePortfolio should be considered as a joint venture between student and institution where strong returns on investment can be realised by both. It is acknowledged that the current implementation strategies for the ePortfolio are just the beginning of a much longer process. The real rewards for students, academics and the university lie in the future.

Moving beyond transition pedagogy: Maturity models and student engagement

Australian higher education institutions (HEIs) have entered a new phase of regulation and accreditation which includes performance-based funding relating to the participation and retention of students from social and cultural groups previously underrepresented in higher education. However, in addressing these priorities, it is critical that HEIs do not further disadvantage students from certain groups by identifying them for attention because of their social or cultural backgrounds, circumstances which are largely beyond the control of students. In response, many HEIs are focusing effort on university-wide approaches to enhancing the student experience because such approaches will enhance the engagement, success and retention of all students, and in doing so, particularly benefit those students who come from underrepresented groups. Measuring and benchmarking student experiences and engagement that arise from these efforts is well supported by extensive collections of student experience survey data. However no comparable instrument exists that measures the capability of institutions to influence and/or enhance student experiences where capability is an indication of how well an organisational process does what it is designed to do (Rosemann & de Bruin, 2005). This paper proposes that the concept of a maturity model (Marshall, 2010; Paulk, 1999) may be useful as a way of assessing the capability of HEIs to provide and implement student engagement, success and retention activities. We will describe the Student Engagement, Success and Retention Maturity Model (SESR-MM), (Clarke, Nelson & Stoodley, 2012; Nelson, Clarke & Stoodley, 2012) we are currently investigating. We will discuss if our research may address the current gap by facilitating the development of an SESR-MM instrument that aims (i) to enable institutions to assess the capability of their current student engagement and retention programs and strategies to influence and respond to student experiences within the institution; and (ii) to provide institutions with the opportunity to understand various practices across the sector with a view to further improving programs and practices relevant to their context. The first aim of our research is to extend the generational approach which has been useful in considering the evolutionary nature of the first year experience (FYE) (Wilson, 2009). Three generations have been identified and explored: First generation approaches that focus on co-curricular strategies (e.g. orientation and peer programs); Second generation approaches that focus on curriculum (e.g. pedagogy, curriculum design, and learning and teaching practice); and third generation approaches—also referred to as transition pedagogy—that focus on the production of an institution-wide integrated holistic intentional blend of curricular and co-curricular activities (Kift, Nelson & Clarke, 2010). The second aim of this research is to move beyond assessments of students’ experiences to focus on assessing institutional processes and their capability to influence student engagement. In essence, we propose to develop and use the maturity model concept to produce an instrument that will indicate the capability of HEIs to manage and improve student engagement, success and retention programs and strategies. References Australian Council for Educational Research. (n.d.). Australasian Survey of Student Engagement. Retrieved from http://www.acer.edu.au/research/ausse/background Clarke, J., Nelson, K., & Stoodley, I. (2012, July). The Maturity Model concept as framework for assessing the capability of higher education institutions to address student engagement, success and retention: New horizon or false dawn? A Nuts & Bolts presentation at the 15th International Conference on the First Year in Higher Education, “New Horizons,” Brisbane, Australia. Kift, S., Nelson, K., & Clarke, J. (2010) Transition pedagogy - a third generation approach to FYE: A case study of policy and practice for the higher education sector. The International Journal of the First Year in Higher Education, 1(1), pp. 1-20. Department of Education, Employment and Workplace Relations. (n.d.). The University Experience Survey. Advancing quality in higher education information sheet. Retrieved from http://www.deewr.gov.au/HigherEducation/Policy/Documents/University_Experience_Survey.pdf Marshall, S. (2010). A quality framework for continuous improvement of e-Learning: The e-Learning Maturity Model. Journal of Distance Education, 24(1), 143-166. Nelson, K., Clarke, J., & Stoodley, I. (2012). An exploration of the Maturity Model concept as a vehicle for higher education institutions to assess their capability to address student engagement. A work in progress. Submitted for publication. Paulk, M. (1999). Using the Software CMM with good judgment, ASQ Software Quality Professional, 1(3), 19-29. Wilson, K. (2009, June–July). The impact of institutional, programmatic and personal interventions on an effective and sustainable first-year student experience. Keynote address presented at the 12th Pacific Rim First Year in Higher Education Conference, “Preparing for Tomorrow Today: The First Year as Foundation,” Townsville, Australia. Retrieved from http://www.fyhe.com.au/past_papers/papers09/ppts/Keithia_Wilson_paper.pdf

Principals, Teachers, and Student's Perception of the Information and Communication Technology in Kuwait Secondary Schools (Rhetoric and reality)

ICT integration has been advocated to provide opportunities to improve students’ achievement and engagement through transforming the educational setting. A valuable tool that contributes in enhancing and developing students’ cognitive skills for lifelong learning, ICT integration has introduced a new educational philosophy, shifting the role of students into a more central position in the pedagogical processes. Kuwait, as with many other countries, has recently planned ICT integration to develop its citizen’s capacities. This study sought to capture the principals’, teachers’, and students’ perceptions of ICT integration in pedagogical activities, as well as how ICT is being used for learning and teaching activities in three ICT leading Kuwaiti secondary schools. Interviews with principals, teachers, and students were conducted, along with an open-ended questionnaire for the teachers, researcher observations, and document analysis. The findings revealed that ICT integration in Kuwait needed to be reinforced to accomplish the ICT integration objectives. A call for further support for teachers, and a reconsideration of the ICT integration strategies were also recommended.