ClusterEval 1.0 : Cluster quality Evaluation software

This report describes the available functionality and use of the ClusterEval evaluation software. It implements novel and standard measures for the evaluation of cluster quality. This software has been used at the INEX XML Mining track and in the MediaEval Social Event Detection task.

Quality evaluation of Yin Chen Hao Tang extract based on fingerprint chromatogram and simultaneous determination of five bioactive constituents

A completely validated method based on HPLC coupled with photodiode array detector (HPLC-UV) was described for evaluating and controlling quality of Yin Chen Hao Tang extract (YCHTE). First, HPLC-UV fingerprint chromatogram of YCHTE was established for preliminarily elucidating amount and chromatographic trajectory of chemical constituents in YCHTE. Second, for the first time, five mainly bioactive constituents in YCHTE were simultaneously determined based on fingerprint chromatogram for furthermore controlling the quality of YCHTE quantitatively. The developed method was applied to analyze 12 batches of YCHTE samples which consisted of herbal drugs from different places of production, showed acceptable linearity, intraday (RSD <5%), interday precision (RSD <4.80%), and accuracy (RSD <2.80%). As a result, fingerprint chromatogram determined 15 representative general fingerprint peaks, and the fingerprint chromatogram resemblances are all better than 0.9996. The contents of five analytes in different batches of YCHTE samples do not indicate significant difference. So, it is concluded that the developed HPLC-UV method is a more fully validated and complete method for evaluating and controlling the quality of YCHTE.

Comparison of genomic DNA extraction techniques from whole blood samples : a time, cost and quality evaluation study

Genomic DNA obtained from patient whole blood samples is a key element for genomic research. Advantages and disadvantages, in terms of time-efficiency, cost-effectiveness and laboratory requirements, of procedures available to isolate nucleic acids need to be considered before choosing any particular method. These characteristics have not been fully evaluated for some laboratory techniques, such as the salting out method for DNA extraction, which has been excluded from comparison in different studies published to date. We compared three different protocols (a traditional salting out method, a modified salting out method and a commercially available kit method) to determine the most cost-effective and time-efficient method to extract DNA. We extracted genomic DNA from whole blood samples obtained from breast cancer patient volunteers and compared the results of the product obtained in terms of quantity (concentration of DNA extracted and DNA obtained per ml of blood used) and quality (260/280 ratio and polymerase chain reaction product amplification) of the obtained yield. On average, all three methods showed no statistically significant differences between the final result, but when we accounted for time and cost derived for each method, they showed very significant differences. The modified salting out method resulted in a seven- and twofold reduction in cost compared to the commercial kit and traditional salting out method, respectively and reduced time from 3 days to 1 hour compared to the traditional salting out method. This highlights a modified salting out method as a suitable choice to be used in laboratories and research centres, particularly when dealing with a large number of samples.

Perception Quality Evaluation with Visual and Infrared Cameras in Challenging Environmental Conditions

This work aims to contribute to the reliability and integrity of perceptual systems of unmanned ground vehicles (UGV). A method is proposed to evaluate the quality of sensor data prior to its use in a perception system by utilising a quality metric applied to heterogeneous sensor data such as visual and infrared camera images. The concept is illustrated specifically with sensor data that is evaluated prior to the use of the data in a standard SIFT feature extraction and matching technique. The method is then evaluated using various experimental data sets that were collected from a UGV in challenging environmental conditions, represented by the presence of airborne dust and smoke. In the first series of experiments, a motionless vehicle is observing a ’reference’ scene, then the method is extended to the case of a moving vehicle by compensating for its motion. This paper shows that it is possible to anticipate degradation of a perception algorithm by evaluating the input data prior to any actual execution of the algorithm.

User-centered video quality assessment for scalable video coding of H.264/AVC standard

Scalable video coding of H.264/AVC standard enables adaptive and flexible delivery for multiple devices and various network conditions. Only a few works have addressed the influence of different scalability parameters (frame rate, spatial resolution, and SNR) on the user perceived quality within a limited scope. In this paper, we have conducted an experiment of subjective quality assessment for video sequences encoded with H.264/SVC to gain a better understanding of the correlation between video content and UPQ at all scalable layers and the impact of rate-distortion method and different scalabilities on bitrate and UPQ. Findings from this experiment will contribute to a user-centered design of adaptive delivery of scalable video stream.

Kaleidoscope : Reframing learning and teaching evaluation through a shared stakeholder lens

Sector wide interest in Reframe: QUT’s Evaluation Framework continues with a number of institutions requesting finer details as QUT embeds the new approach to evaluation across the university in 2013. This interest, both nationally and internationally has warranted QUT’s collegial response to draw upon its experiences from developing Reframe into distilling and offering Kaleidoscope back to the sector. The word Reframe is a relevant reference for QUT’s specific re-evaluation, reframing and adoption of a new approach to evaluation; whereas Kaleidoscope reflects the unique lens through which any other institution will need to view their own cultural specificity and local context through an extensive user-led stakeholder engagement approach when introducing new approaches to learning and teaching evaluation. Kaleidoscope’s objectives are for QUT to develop its research-based stakeholder approach to distil the successful experience exhibited in the Reframe Project into a transferable set of guidelines for use by other tertiary institutions across the sector. These guidelines will assist others to design, develop, and deploy, their own culturally specific widespread organisational change informed by stakeholder engagement and organisational buy-in. It is intended that these guidelines will promote, support and enable other tertiary institutions to embark on their own evaluation projects and maximise impact. Kaleidoscope offers an institutional case study of widespread organisational change underpinned by Reframe’s (i) evidence-based methodology; (ii) research including published environmental scan, literature review (Alderman, et al., 2012), development of a conceptual model (Alderman, et al., in press 2013), project management principles (Alderman & Melanie, 2012) and national conference peer reviews; and (iii) year-long strategic project with national outreach to collaboratively engage the development of a draft set of National Guidelines. Kaleidoscope’s aims are to inform Higher Education evaluation policy development through national stakeholder engagement, the finalisation of proposed National Guidelines. In correlation with the conference paper, the authors will present a Draft Guidelines and Framework ready for external peer review by evaluation practitioners from the Higher Education sector, as part of Kaleidoscope’s dissemination strategy (Hinton & Gannaway, 2011) applying illuminative evaluation theory (Parlett & Hamilton, 1976), through conference workshops and ongoing discussions (Shapiro, et al., 1983; Jacobs, 2000). The initial National Guidelines will be distilled from the Reframe: QUT’s Evaluation Framework’s Policy, Protocols, and incorporated Business Rules. It is intended that the outcomes of Kaleidoscope are owned by and reflect sectoral engagement, including iterative evaluation through multiple avenues of dissemination and collaboration including the Higher Education sector. The dissemination strategy with the inclusion of Illuminative Evaluation methodology provides an inclusive opportunity for other institutions and stakeholders across the Higher Education sector to give voice through the information-gathering component of evaluating the draft Guidelines, providing a comprehensive understanding of the complex realities experienced across the Higher Education sector, and thereby ‘illuminating’ both the shared and unique lenses and contexts. This process will enable any final guidelines developed to have broader applicability, greater acceptance, enhanced sustainability and additional relevance benefiting the Higher Education sector, and the adoption and adaption by any single institution for their local contexts.

Kaleidoscope : illuminating learning and teaching evaluation for a shared stakeholder lens

Within the Australian higher education sector, institutions are required to evaluate teaching, units and courses to assure the quality of the student learning experience, however with hardly any regulatory parameters guiding institutions, and with disparate practices, there are few opportunities to benchmakr across institutions or the sector. QUT has received interest and requests from national and international universities on accessing Reframe: QUT's Evaluation Framework.

Student feedback systems in higher education: a focused literature review and environmental scan

In recent times, higher education institutions have paid increasing attention to the views of students to obtain feedback on their experience of learning and teaching through internal surveys. This article reviews research in the field and reports on practices in other Australian universities. Findings demonstrate that while student feedback is valued and used by all Australian universities, survey practices are idiosyncratic and in the majority of cases, questionnaires lack validity and reliability; data are used inadequately or inappropriately; and they offer limited potential for cross-sector benchmarking. The study confirms the need for institutions to develop an overarching framework for evaluation in which a valid, reliable, multidimensional and useful student feedback survey constitutes just one part. Given external expectations and internal requirements to collect feedback from students on their experience of learning and teaching, the pursuit of sound evaluation practices will continue to be of interest at local, national and international levels.

Adaptive teachers embracing new ways of learning with robotics in Chinese schools

In July 2010, China announced the “National Plan for Medium and Long-term Education Reform and Development(2010-2020)” (PRC 2010). The Plan calls for an education system that: • promotes an integrated development which harnesses everyone’s talent; • combines learning and thinking; unifies knowledge and practice; • allows teachers to teach according to individuals’ needs; and • reforms education quality evaluation and personnel evaluation systems focusing on performance including character, knowledge, ability and other factors. This paper discusses the design and implementation of a Professional Learning Program (PLP) undertaken by 432 primary, middle and high school teachers in China. The aim of this initiative was to develop adaptive expertise in using technology that facilitated innovative science and technology teaching and learning as envisaged by the Chinese Ministry of Education’s (2010-2020) education reforms. Key principles derived from literature about professional learning and scaffolding of learning informed the design of the PLP. The analysis of data revealed that the participants had made substantial progress towards the development of adaptive expertise. This was manifested not only by advances in the participants’ repertoires of Subject Matter Knowledge and Pedagogical Content Knowledge but also in changes to their levels of confidence and identities as teachers. It was found that through time the participants had coalesced into a professional learning community that readily engaged in the sharing, peer review, reuse and adaption, and collaborative design of innovative science and technology learning and assessment activities.

Disengaged and disruptive : behaviour intervention for boys from year four to year seven

Disengaged and disruptive students have been an ongoing concern for teachers for many years. Teaching is complex—complex students with complex lives and complex behaviours. How best to help these students is an ever-present question without a simple answer. Solutions need to be found. Under a positive behaviour support framework when serious, disruptive behaviour requires intervention, an individualised positive behaviour support plan (PBS plan) is developed and implemented. This multicase study (Stake, 2006) investigated how task engagement was changed for boys from year four to year seven who demonstrated serious, disruptive behaviour. The individualised PBS plan was the primary tool of behaviour intervention in each of the five cases. Using the Behaviour Support Plan Quality Evaluation Scoring Guide II (BSP-QE) (Browning- Wright, Saren & Mayer, 2003) the five PBS plans were evaluated prior to implementation and rated highly in terms of technical quality. Positive changes in student task engagement were forthcoming in all five cases. Eleven advisory visiting teachers in behaviour and eleven classroom teachers, five of whom were case-study participants, took part in this study. The classroom teachers were employed in south-east Queensland primary schools located in suburbs of economic disadvantage. All 22 participants expressed very similar perceptions of serious, disruptive behaviour emphasising the collateral impact upon the teaching and learning. Data obtained through direct observations, surveys and semi-structured interviews confirmed previous research to reveal a strong link between integrity of PBS plan implementation and student behaviour change. While classroom teachers, in the main, effectively managed the implementation of the PBS plan, social validity of goals, procedures and effects; in-class technical assistance and performance feedback were identified as three enablers to effective teacher implementation of the PBS plan. While the purpose of each PBS plan was to influence change in student behaviour, this study found that changing teacher behaviour was also instrumental in achieving positive student outcomes. Changing teacher behaviour and building capacity was facilitated by trusting, collaborative partnerships established between the Advisory Visiting Teacher-Behaviour and the classroom teacher responsible for the plan implementation. The Advisory Visiting Teacher-Behaviour provides assistance to teachers dealing with students who demonstrate ongoing, problematic behaviour. The inclusion of a teaching component as part of the implementation stage of the consultation process appeared to have considerable influence upon successful intervention. Results substantiated earlier understandings of the importance of teacher instruction highlighting the value of explicit teaching and performance feedback to the delivery of effective behaviour intervention. Conclusions drawn from this study have had a major impact upon the work of a regional team of Advisory Visiting Teachers-Behaviour. The focus of behaviour intervention has moved from being primarily upon the individual student to include a greater emphasis upon the critical role of the teacher. Procedures and processes are being re-evaluated to align with evidence-based practice and to include a collaborative consultation approach to improve teacher assistance. The framework and content of staff development and training is being created directly from the findings of this study. This practical application of the results has informed better ways of providing behaviour intervention for students demonstrating serious, disruptive behaviour. What this study has clearly shown is that when it comes to behaviour intervention, the important role of the teacher cannot be underestimated.

An evaluation framework for educational reform projects for teacher quality improvements in developing countries : a case study of Egyptian education reform

The role of the evaluation for Official Development Assistance (ODA) enterprises including educational development has become critical after increasing “aid fatigue” experienced by the international community in the 1990s. To date, however, monitoring and evaluating outcomes of the projects has been limited to the project life. Consequently these have been mainly through the international aid agencies. Furthermore, the monitoring and evaluation led by international aid agencies have paid little attention to aspects of the sustainability of technical cooperation in educational development. To sustain the impact of technical cooperation, the reinforcement of evaluation has drawn increasing attention in light of the emerging modalities in international development. Therefore this research was inspired to investigate alternative evaluation frameworks for an educational reform project for teacher quality improvement that may increase possibilities for long term sustainability. Importantly, the new modalities in international development and educational issues provide new options. In addition, the research reviewed theoretical and practical issues surrounding evaluation in general, and highlighted the evaluation of education reform projects. The research reported explored via case studies, the evaluation processes employed by the Egyptian education reform projects implemented by the Japan International Cooperation Agency (JICA) and the United Nations Children’s Fund (UNICEF). The case studies used three data sources (archival and relevant documents, a survey questionnaire and interviews) to illuminate the contextually-embedded evaluation processes. The research found that process evaluation is a potential alternative method since it is likely to be locally institutionalised, which may yield long-term sustainability of the projects.

Evaluation on service quality of railway timetable

From the business viewpoint, the railway timetable is a list of the products presented by the railway transportation operators to the customers, specifying the schedules of all the train services on a railway line or network. In order to evaluate the quality of the train service schedules, a number of indices are proposed in this paper. These indices primarily take the passengers’ needs, such as waiting time, transfer time and transport capacity into consideration. Delay rate is usually used in post-evaluation. In this study, we propose to give an evaluation on the probability that the scheduled train services are likely to be delayed and the recovery ability of the timetable after delay has occurred. The evaluation identifies the possible problems in the services, such as excessive waiting time, non-seamless transfer, and high possibility of delay. This paper also discusses the improvement of these problems through certain adjustments on the timetable. The indices for evaluation and the adjustment method on timetable are then applied to a case study on the Hu-Ning-Hang railway in China, followed by the discussions of the merits of the proposed indices for timetable evaluation and possible improvement.