Tracking new directions in the evaluation of postgraduate supervision

This article focuses on the evaluation of thesis supervision and highlights the vast range of problem areas presently documented as significant areas of concern for both graduate students and their supervisors. Additionally, the authors report on a study completed in 1995, which surveyed all Australian universities about current evaluative practices in postgraduate supervision. The conclusion of this study was that the conduct of such evaluations appears to be minimal and is primarily designed to obtain an indicator of the general “health” of a university's postgraduate supervision rather than to foster improved supervisory practices. As part of the same study, the authors conducted University faculty-based student and supervisor focus groups. Key issues emerging from these focus groups were: (1) the importance of relational aspects of supervision as the student communicates over the long term with one or more supervisors; (2) the importance of systematic feedback, monitoring, and evaluation to the supervisory process; and (3) the lack of strategies to facilitate this evaluative feedback process. On the basis of these findings, the authors designed evaluative strategies to facilitate regular ongoing feedback between students and supervisors.

Bouncing back : homegrown. Curated exhibition of student work and design workshop

Bouncing Back Architecture Exhibition: This exhibition showcases interpretations of urban resiliency by 2nd and 4th Year undergraduate architecture students who explore the notion of Bouncing Back from the 2011 Queensland floods, in the context of contemporary Brisbane built environment. Design solutions have been expressed in a variety of forms including emergency shelters, flood-proof housing and a range of urban designs, some of which address extreme environmental conditions. Design Process Workshop | Architecture Workshop with Queensland Academy of Creative Industries Students: In collaboration with Homegrown Facilitator Natalie Wright, Lindy Osborne and Glenda Caldwell and some of their architecture students from the QUT School of Design, extended the university design studio experience to 18 Secondary School students, who brainstormed and designed emergency food distribution shelters for those affected by floods. Designs and models created in the workshop were subsequently included in the Bouncing Back Architecture Exhibition.

Graph-based concept weighting for medical information retrieval

This paper presents a graph-based method to weight medical concepts in documents for the purposes of information retrieval. Medical concepts are extracted from free-text documents using a state-of-the-art technique that maps n-grams to concepts from the SNOMED CT medical ontology. In our graph-based concept representation, concepts are vertices in a graph built from a document, edges represent associations between concepts. This representation naturally captures dependencies between concepts, an important requirement for interpreting medical text, and a feature lacking in bag-of-words representations. We apply existing graph-based term weighting methods to weight medical concepts. Using concepts rather than terms addresses vocabulary mismatch as well as encapsulates terms belonging to a single medical entity into a single concept. In addition, we further extend previous graph-based approaches by injecting domain knowledge that estimates the importance of a concept within the global medical domain. Retrieval experiments on the TREC Medical Records collection show our method outperforms both term and concept baselines. More generally, this work provides a means of integrating background knowledge contained in medical ontologies into data-driven information retrieval approaches.

Biomechanical performance of polycaprolactone (PCL)-based scaffold with rhBMP-2 in a sheep thoracic spine fusion model

Adolescent idiopathic scoliosis is a complex three dimensional deformity affecting 2-3% of the general population. Resulting spine deformities include progressive coronal curvature, hypokyphosis, or frank lordosis in the thoracic spine and vertebral rotation in the axial plane with posterior elements turned into the curve concavity. The potential for curve progression is heightened during the adolescent growth spurt. Success of scoliosis deformity correction depends on solid bony fusion between adjacent vertebrae after the intervertebral discs have been surgically cleared and the disc spaces filled with graft material. Problems with bone graft harvest site morbidity as well as limited bone availability have led to the search for bone graft substitutes. Recently, a bioactive and resorbable scaffold fabricated from medical grade polycaprolactone (PCL) has been developed for bone regeneration at load bearing sites. Combined with recombinant human bone morphogenic protein–2 (rhBMP-2), this has been shown to be successful in acting as a bone graft substitute in acting as a bone graft substitute in a porcine lumbar interbody fusion model when compared to autologous bone graft. This in vivo sheep study intends to evaluate the suitability of a custom designed medical grade PCL scaffold in combination with rhBMP-2 as a bone graft substitute in the setting of mini–thoracotomy surgery as a platform for ongoing research to benefit patients with adolescent idiopathic scoliosis.

Assessing performance and capability in the work place : focusing on essential skills and knowledge in student work placements

The determination of performance standards and assessment practices in regard to student work placements is an essential and important task. Inappropriate, inadequate, or excessively complex assessment tasks can influence levels of student engagement and the quality of learning outcomes. Critical to determining appropriate standards and assessment tasks is an understanding and knowledge of key elements of the learning environment and the extent to which opportunities are provided for students to engage in critical reflection and judgement of their own performance in the contexts of the work environment. This paper focuses on the development of essential skills and knowledge (capabilities) that provide evidence of learning in work placements by describing an approach taken in the science and technology disciplines. Assessment matrices are presented to illustrate a method of assessment for use within the context of the learning environment centred on work placements in science and technology. This study contributes to the debate on the meaning of professional capability, performance standards and assessment practices in work placement programs by providing evidence of an approach that can be adapted by other programs to achieve similar benefits. The approach may also be valuable to other learning contexts where capability and performance are being judged in situations that are outside a controlled teaching and learning environment i.e. in other life-wide learning contexts.

Studios for the masses : can student colaboration replace the master-apprentice relationship in design instruction?

This action research project investigated the use of a collaborative learning approach for addressing issues associated with teaching urban design to large, diverse cohorts. As a case study, I observed two semesters of an urban design unit that I revised between 2011 and 2012 to incorporate collaborative learning activities. Data include instructional materials, participant observations, peer-reviews of collaborative learning activities, feedback from students and instructors and student projects. Themes that emerged through qualitative analysis include the challenge of removing inequalities inherent in the diverse cohort, the challenge of unifying project guidance and marking criteria, and the challenge of providing project guidance for a very large cohort. Most notably, the study revealed a need to clarify learning objectives relating to design principles in order to fully transition to and benefit from a collaborative learning model.

A conceptual model for reflecting on expected learning vs. demonstrated student performance

Educators are faced with many challenging questions in designing an effective curriculum. What prerequisite knowledge do students have before commencing a new subject? At what level of mastery? What is the spread of capabilities between bare-passing students vs. the top performing group? How does the intended learning specification compare to student performance at the end of a subject? In this paper we present a conceptual model that helps in answering some of these questions. It has the following main capabilities: capturing the learning specification in terms of syllabus topics and outcomes; capturing mastery levels to model progression; capturing the minimal vs. aspirational learning design; capturing confidence and reliability metrics for each of these mappings; and finally, comparing and reflecting on the learning specification against actual student performance. We present a web-based implementation of the model, and validate it by mapping the final exams from four programming subjects against the ACM/IEEE CS2013 topics and outcomes, using Bloom's Taxonomy as the mastery scale. We then import the itemised exam grades from 632 students across the four subjects and compare the demonstrated student performance against the expected learning for each of these. Key contributions of this work are the validated conceptual model for capturing and comparing expected learning vs. demonstrated performance, and a web-based implementation of this model, which is made freely available online as a community resource.

Emerging from student to practitioner : a cumulative development tool for students

PURPOSE: This pilot project’s aim was to trial a tool and process for developing students’ ability to engage in self-assessment using reflection on their clinical experiences, including feedback from workplace learning, in order to aid them in linking theory to practice and develop strategies to improve performance. BACKGROUND: In nursing education, students can experience a mismatch in performance compared to theoretical learning, this is referred to as the ‘theory practice gap’ (Scully 2011, Chan Chan & Liu 2011). One specific contributing factor seems to be students’ inability to engage in meaningful reflection and self-correcting behaviours. A self-assessment strategy was implemented within a third year clinical unit to ameliorate this mismatch with encouraging results, as students developed self-direction in addressing learning needs. In this pilot project the above strategy was adapted for implementation between different clinical units, to create a whole of course approach to integrating workplace learning. METHOD: The methodology underpinning this project is a scaffolded, supported reflective practice process. Improved self-assessment skills is achieved by students reflecting on and engaging with feedback, then mapping this to learning outcomes to identify where performance can be improved. Evaluation of this project includes: collation of student feedback identifying successful strategies along with barriers encountered in implementation; feedback from students and teachers via above processes and tools; and comparison of the number of learning contracts issued in clinical nursing units with similar cohorts. RESULTS: Results will be complete by May 2012 and include analysis of the data collected via the above evaluation methods. Other outcomes will include the refined process and tool, plus resources that should improve cost effectiveness without reducing student support. CONCLUSION: Implementing these tools and processes over the entire student’s learning package, will assist them to demonstrate progressive development through the course. Students will have learnt to understand feedback and integrate these skills for life-long learning.

Understanding QC and EQA : an authentic learning model for undergraduate students (Conference Abstract)

Introduction QC and EQA are integral to good pathology laboratory practice. Medical Laboratory Science students undertake a project exploring internal QC and EQA procedures used in chemical pathology laboratories. Each student represents an individual lab and the class group represents the peer group of labs performing the same assay using the same method. Methods Using a manual BCG assay for serum albumin, normal and abnormal controls are run with a patient sample over 7 weeks. The QC results are assessed each week using calculated z-scores and both 2S & 3S control rules to determine whether a run is ‘in control’. At the end of the 7 weeks a completed LJ chart is assessed using the Westgard Multirules. Students investigate causes of error and the implications for both lab practice and patient care if runs are not ‘in control’. Twice in the 7 weeks two EQA samples (with target values unknown) are assayed alongside the weekly QC and patient samples. Results from each student are collated and form the basis of an EQA program. ALP are provided and students complete a Youden Plot, which is used to analyse the performance of each ‘lab’ and the method to identify bias. Students explore the concept of possible clinical implications of a biased method and address the actions that should be taken if a lab is not in consensus with the peer group. Conclusion This project is a model of ‘real world’ practice in which student demonstrate an understanding of the importance of QC procedures in a pathology laboratory, apply and interpret statistics and QC rules and charts, apply critical thinking and analytical skills to quality performance data to make recommendations for further practice and improve their technical competence and confidence.

University Student Depression Inventory (USDI) : confirmatory factor analysis and review of psychometric properties

Background: The 30-item USDI is a self-report measure that assesses depressive symptoms among university students. It consists of three correlated three factors: Lethargy, Cognitive-Emotional and Academic motivation. The current research used confirmatory factor analysis to asses construct validity and determine whether the original factor structure would be replicated in a different sample. Psychometric properties were also examined. Method: Participants were 1148 students (mean age 22.84 years, SD = 6.85) across all faculties from a large Australian metropolitan university. Students completed a questionnaire comprising of the USDI, the Depression Anxiety Stress Scale (DASS) and Life Satisfaction Scale (LSS). Results: The three correlated factor model was shown to be an acceptable fit to the data, indicating sound construct validity. Internal consistency of the scale was also demonstrated to be sound, with high Cronbach Alpha values. Temporal stability of the scale was also shown to be strong through test-retest analysis. Finally, concurrent and discriminant validity was examined with correlations between the USDI and DASS subscales as well as the LSS, with sound results contributing to further support the construct validity of the scale. Cut-off points were also developed to aid total score interpretation. Limitations: Response rates are unclear. In addition, the representativeness of the sample could be improved potentially through targeted recruitment (i.e. reviewing the online sample statistics during data collection, examining the representativeness trends and addressing particular faculties within the university that were underrepresented). Conclusions: The USDI provides a valid and reliable method of assessing depressive symptoms found among university students.

The effects of linguistic factors on student performance on economics multiple choice questions

This paper proposes a framework to analyse performance on multiple choice questions with the focus on linguistic factors. Item Response Theory (IRT) is deployed to estimate ability and question difficulty levels. A logistic regression model is used to detect Differential Item Functioning questions. Probit models testify relationships between performance and linguistic factors controlling the effects of question construction and students’ background. Empirical results have important implications. The lexical density of stems affects performance. The use of non-Economics specialised vocabulary has differing impacts on the performance of students with different language backgrounds. The IRT-based ability and difficulty help explain performance variations.