Enhancing quality in online learning : scaffolding planning and design through proactive evaluation

A major issue confronting educators is the extent to which they wish to conform to so-called paradigm shifts in teaching and learning. In the contemporary world of tertiary education these shifts embrace both pedagogy (from instructivist to constructivist) and technology (from classroom to online). As teachers and learners are faced with the potential of these new learning environments, the extent to which learning outcomes are achieved remains a high priority and subject to a wide range of evaluation strategies. Conventionally, evaluation has been positioned at the end of the instructional development cycle, to assess first whether or not the creative effort achieved the original product goals and second whether or not the desired learning outcomes were realized. In the context of online teaching and learning environments, however, the level of understanding teachers, learners and developers have of the medium can impact the ultimate effectiveness of the product. This paper articulates an additional dimension to post-development evaluation processes in proposing proactive evaluation, a framework that identifies critical online learning factors and influences that will better inform the planning, design and development of learning resources. This notion of proactive evaluation advocates resource development being undertaken where all planning activities are assessed against the evaluation criteria that would normally be applied during formative assessment. By performing these evaluation checks proactively, online learning resources will, in principle, work first time as all relevant factors and issues will have been considered and resolved. More importantly, for those participants who are new to online environments, proactive evaluation will perform a scaffolding and professional development role by enhancing online teaching or learning competencies.

Mobile Architecture and Built Environment Laboratory (MABEL) - a building performance evaluation tool

This paper describes the Mobile Architecture and Built Environment Laboratory (MABEL) and its application for systematic building performance evaluation for compliance testing, commissioning, strategic and operational facility management and continuous improvement in the built environment.

The first part of the paper introduces the application areas of on-site building performance evaluation and discusses the shortcomings in this regard in current practice. It emphasises the need for on-site investigations to generate information on 'as built performance' for the 'feedback' loop between design, operation and occupancy of new buildings, retrofit or adjustment.

The second part introduces the Energy-Comfort-Behaviour Framework for 'across-the-board' building evaluation and discusses MABEL's role in this scheme. MABEL's objectives, procedures and the performance measurement matrix are explained and discussed.

Proactive evaluation : new perspectives for ensuring quality in online learning applications

A major issue confronting educators is the extent to which they wish to conform to so-called paradigm shifts in teaching and learning. In the contemporary world of tertiary education these shifts embrace both pedagogy (from instructivist to constructivist) and technology (classroom to online). As teachers and learners are faced with the potential of these new learning environments, the extent to which the learning outcomes are achieved remains a high priority and subject to a wide range of evaluation strategies. Conventionally, evaluation is often conceptualised as occurring at the end of the development process, to assess first (formatively) whether or not the creative effort has achieved the original product goals and second (summatively) whether or not the desired learning outcomes were achieved. However, in the context of imperatives to implement online learning paradigms, the level of understanding teachers and developers have of the medium can impact the effectiveness of the product. This paper presents an additional perspective to the post-development, reactive evaluation processes in proposing the concept of proactive evaluation, a framework that identifies critical online learning factors and influences to better inform the development of learning resources. In essence, the proposal advocates an approach where development is undertaken within an environment where all activities are assessed using the evaluation criteria that would be applied when the product is assessed reactively. By performing these checks proactively, online learning resources will, in principle, work first time as all relevant factors and issues will have been considered and resolved.

Potential of telemedicine : a preliminary evaluation through the innovation diffusion lens

With the rapid diffusion of innovative technologies in the healthcare industry, the potential of telemedicine has increasingly come into focus. Touted as a catalyst for an equitable healthcare framework among nations, research is wide and one spotlight is on the potential of this innovation to diffuse among the wider society. In this paper, we attempt a preliminary evaluation of the potential of telemedicine through the components of [Diffusion of Innovations (4th ed.). NY: The Free Press] diffusion of innovations theoretical lens, drawing also as relevant from reported literature. We expect that this preliminary evaluation will facilitate further academic research and make other stakeholders, such as regulators, medical practitioners, telecommunication providers and vendors, aware of potential steps to be taken to leverage telemedicine diffusion.

Supporting user evaluation of IT security certification schemes

IT Security Certification is an increasingly important qualification for information technology (IT) professionals seeking employment in IT security. Yet currently there is a lack of rigorously developed approaches to support the evaluation and selection by key stakeholders of the most appropriate IT security certification scheme from among hundreds of vendor-neutral and vendor-specific schemes. This paper develops a framework based on categories, characteristics and criteria to support user evaluation and selection of an (IT) Security Certification scheme that satisfies user priorities and requirements. The paper illustrates the use of the framework to support an experienced IT Professional’s evaluation. Theoretical and practical implications of the framework and trial evaluation are discussed.

A proposed hybrid evaluation methodology for intelligent transport system deployment

The rapid deployment of intelligent transport systems in practice requires serious evaluation of new construction projects. Traditional transportation evaluation methods are insufficient to capture the new features of intelligent transport systems, particularly the applications of recently developed information technologies. This research aims to develop a new conceptual framework to evaluate the social, economic and environmental impacts of intelligent transport systems deployment, with explicit consideration of impacts of information technologies on variant travel behaviours. The research would provide a systematic evaluation methodology to enable decision makers and publics better evaluating and understanding the social, economic and environmental benefits of implementation of intelligent transport systems.

Irish theme pubs : an evaluation of environmental cues

A major phenomenon in hospitality retailing in recent years has been the growth in Irish theme pubs. These pubs consist of a collection of both tangible and intangible attributes referred to as environmental cues or atmospherics. Although environmental cues have been found to have a significant influence on consumer behaviour, little research has been conducted into their effects within Irish theme pubs. Using an established research framework, the aims of this study were to identify and evaluate environmental cues within Irish theme pubs and to establish how the dimensions of this framework influenced customers of Irish theme pubs in Melbourne. Qualitative interviews were conducted with customers of six selected Irish theme pubs and with the owners of these establishments. Whilst confirming the impOliance of environmental cues to Irish theme pub customers, the study also provides information that has application for hospitality retailers.

Evidence of high-skilled VET practitioners and high-performing VET organisations : findings from an evaluation of the long-term impacts of Framing the Future projects conducted in 1999-2000

Framing the Future is a major staff development initiative of the Australian National Training Authority (ANTA), designed to support the implementation of the National Training Framework (NTF). Since 1997 over 20,000 vocational education and training (VET) practitioners have participated in the program. The program was renamed Reframing the Future in 2001.

This study reports on research conducted on the long-term impacts of projects funded by Framing the Future in 1999 and 2000. John Mitchell and Sarah Wood from John Mitchell & Associates conducted the research from May 2000–May 2001.

Using twenty four case studies and the results of interviews and an extensive survey, the report provides evidence of high-skilled VET practitioners and high-performing VET organisations who ensure that their involvement in Framing the Future projects leads to long-term gains, particularly in support of the implementation of the NTF.

The potential for communities of practice to underpin the National Training Framework

This report is an evaluation of the sixteen pilot projects that were funded to form Communities of Practice in the VET sector in 2001, as part of the Reframing the Future program. The major theme emerging from this evaluation study is that Communities of Practice in the vocational education and training (VET) sector in Australia have the potential to accelerate, intensify, enrich and enhance the implementation of the national training system.

Equity-focused health impact assessment framework

Background to the Development of the Equity-Focused HIA Framework
The equity focused health impact assessment (EFHIA) framework arises out of a two year research project funded for the most part by the Australian Government’s Public Health Education Research Program (PHERP) Innovations Grants (Round 2) scheme. This project had as its primary objective the development of a framework for health inequalities impact assessment, subsequently renamed equity focused health impact assessment. A partnership between the University of Newcastle, Deakin University and the University of New South Wales (the Project Management Steering Committee) received the funding and the Australasian Collaboration for Health Equity Impact Assessment (ACHEIA) was formed to undertake appropriate background research and to develop, pilot test, modify and disseminate the framework. The work commenced in September 2002 and concluded in October 2004. Part of the funding included a capacity building workshop in August 2004. ACT Health and the Division of Medicine at the John Hunter Hospital, Newcastle, also provided financial support for the project. The August 2004 Workshop was supported by NSW Health. All participants and organisations involved in the project gave extensive in-kind support.
The aims of the workshop were to bring together an international collaboration of multidisciplinary investigators, public health experts, and key senior health managers working in national, state and local settings, to inform the further development of the framework and to provide training in its application. The initial goals of the project were to work collaboratively to develop a strategic framework to assess the health inequalities of public health-related policies, plans, strategies, decisions, programs and services. The EFHIA framework as presented at the August workshop was developed through:
1. an extensive review of the relevant literature
2. formal and informal consultation with members of ACHEIA (the international
reference group), members of the Project Management Steering Committee and
other relevant experts; and
3. testing of the draft EFHIA framework with the 5 case study partners – who applied the draft framework in a range of health settings (see
Acknowledgements).
The result of this work has been the development of an equity focused health impact assessment framework that can be used to determine the unanticipated and systemic health inequities that may exist within the decision making processes or activities of a range of organisations and sectors. The EFHIA framework provides one approach that can be used to assist decision makers to put equity and health on their agenda in a more obvious and systematic way. The framework represents a ‘moment in time’ rather than a definitive statement or ‘toolkit’ on the best way to proceed. Further practice, refinement and adjustment will be needed over many years to consolidate both HIA and EFHIA. As well as this guide to the framework, additional outputs from the project team include:
- A literature review
- A position paper
- A report on the five case studies
- An evaluation report.
With the consent of the Australian Government, a monograph will be made available to workshop participants at the end of October which contains the framework and the appropriate background papers.

Reflexive and critical views of leadership performance in corporate accomplishment : framework and illustration

Purpose – This paper intends to shed some light on the relationship between leadership performance and corporate accomplishment through the aid of complexity sciences. The objective is to describe leadership performance in corporate accomplishment using different teleological approaches.

Design/methodology/approach – The paper discusses the underlying criteria of the relationship between leadership performance and corporate accomplishment. Case illustration and narrative analogy are also provided.

Findings – The authors believe that the discussion highlights a potential downside of leadership performance in corporate accomplishment and its precision rarely highlighted in practice and literature.

Research limitations/implications – There is a reigning assumption in management practice that is based on the belief that a top-down approach of leadership performance in management and business practices is superior to the bottom-up approach. It proffers the assumed importance of strategic management issues, but neglects the knowledge, experience, competence and awareness inherent among employees at tactical and operational levels of business practices. It also proffers a mechanical view of employee performance and ignores the worth of the generation of ideas from subordinates in management and business practices that contribute to corporate achievements. Furthermore, it neglects the fact that it is not possible to know the future nor it is predictable.

Practical implications – The paper contends that the importance of top management tends to be inflated in respect to corporate achievements in the management/leadership literature. It also contends that it should be questioned as to whether the top management of corporations are largely responsible for the corporate results on which they attempt to justify their salaries and other benefits. Furthermore, the paper contends that it also should be questioned as to what extent corporate accomplishment may be derived from the performance of the top management in organizations.

Originality/value – The paper strives to contribute to the ongoing discussion of leadership performance in corporate accomplishment in various ways. The principal contributions are: a set of teleological sub-processes of leadership performance and a case illustration and narrative analogies of teleological leadership performance patterns, in respect to corporate accomplishment in management and business practices. These contributions provide theoretical and managerial ideas and insights to anticipate and avoid deficient or erroneous grounds of leadership performance evaluation in corporate accomplishment.

IT security certifications : stakeholder evaluation and selection

Information technology (IT) security certifications have proliferated in recent years. However they differ in regards to stakeholder considerations of credibility, accessibility and relevance. Key stakeholders with an interest in selecting an IT security certification (IT security professionals, employers, governments and higher education institutes) lack a systematic approach for differentiating between candidate certifications and selecting the “best” certification to satisfy requirements. The paper focuses on reporting a confirmatory focus group from a recent research project. It provides a framework for supporting stakeholder evaluation and selection of IT security certifications and discusses key implications for the IT security industry, IT security certifications, and the higher education sector.

Evaluation of a decision aid for prenatal testing of fetal abnormalities : a cluster randomised trial [ISRCTN22532458]

Background: By providing information on the relative merits and potential harms of the options available and a framework to clarify preferences, decision aids can improve knowledge and realistic expectations and decrease decisional conflict in individuals facing decisions between alternative forms of action. Decision-making about prenatal testing for fetal abnormalities is often confusing and difficult for women and the effectiveness of decision aids in this field has not been established. This study aims to test whether a decision aid for prenatal testing of fetal abnormalities, when compared to a pamphlet, improves women's informed decision-making and decreases decisional conflict.

Methods/design: A cluster designed randomised controlled trial is being conducted in Victoria, Australia. Fifty General Practitioners (GPs) have been randomised to one of two arms: providing women with either a decision aid or a pamphlet. The two primary outcomes will be measured by comparing the difference in percentages of women identified as making an informed choice and the difference in mean decisional conflict scores between the two groups. Data will be collected from women using questionnaires at 14 weeks and 24 weeks gestation.

The sample size of 159 women in both arms of the trial has been calculated to detect a difference of 18% (50 to 68%) in informed choice between the two groups. The required numbers have been adjusted to accommodate the cluster design, miscarriage and participant lost – to – follow up.

Baseline characteristics of women will be summarised for both arms of the trial. Similarly, characteristics of GPs will be compared between arms.

Differences in the primary outcomes will be analysed using 'intention-to-treat' principles. Appropriate regression techniques will adjust for the effects of clustering and include covariates to adjust for the stratifying variable and major potential confounding factors.

Discussion: The findings from this trial will make a significant contribution to improving women's experience of prenatal testing and will have application to a variety of maternity care settings. The evaluation of a tailored decision aid will also have implications for pregnancy care providers by identifying whether or not such a resource will support their role in providing prenatal testing information.

Evaluation of Australian undergraduate engineering management education for flexible delivery

This thesis examines issues in Australian undergraduate engineering management studies in the context of flexible learning delivery. It is proposed that, within an Australian context: a) the management skills and competencies required by graduate engineers can be determined and classified on a rational basis, permitting an educational focus on those elements most appropriate for graduates; and b) on-line and other computer-based technologies are a practical and effective method for the support of undergraduate engineering management studies. The doctoral project incorporates: • an examination of the nature of engineering management; • a review of the relevant literature establishing the importance of management studies in undergraduate engineering courses; • a review of historical and recent developments in Australian undergraduate engineering management studies; • an investigation of the management skills and competencies required by graduate engineers - based on original research; • an examination of flexible delivery of engineering education - based on professional practice experience; and • an evaluation of case studies of flexible delivery of engineering management education - based on original research and professional practice experience. A framework of ranked classified management skills is developed. Broadly, the ranking framework is generic professional skills, followed by general management skills and technical discipline specific management skills, followed by other professional discipline skills and theoretical skills. This framework provides a rational basis for design of undergraduate engineering management studies. This is supplemented by consideration of the management skills required for the future of engineering practice. It is concluded that undergraduate engineering management education is well suited to delivery and support by on-line and computer-based technology. Recent developments in improved access to the Internet, software systems for on-line collaboration and changes in copyright legislation to create a broad-based right to communication via on-line media have contributed to the facilitation of on-line delivery of teaching and learning. It is noted that though many on-line infrastructure issues have been satisfactorily resolved, higher level issues will emerge as being crucial, including the academic staff development and reward for operating in an online teaching environment and the financial sustainability of on-line development and delivery of courses.

Patient assessment of medication information leaflets and validation of the Evaluative Linguistic Framework (ELF)

Objective : To obtain patient feedback about the structure and quality of medication information leaflets and validate the usefulness of the Evaluative Linguistic Framework (ELF) for improving written communication with patients.
Methods : Triangulated feedback about a set of rheumatoid arthritis (RA) medication leaflets, some developed with knowledge of the ELF, was obtained from 27 people with RA from interviews, focus group discussion and self-administered questionnaires. The principal elements of the framework were investigated: overall generic structure and functions of each stage, interpersonal relationship between writer and reader, technicality of language and density of information.
Results : Participant assessments of the leaflets aligned with the framework in terms of what constituted a good leaflet. While the main purpose of the leaflets was identified as being information provision, participants also wanted clear instructions, benefits to be highlighted and side effects to be comprehensively listed. For comprehensiveness and user-friendliness, leaflets developed with guidance of the ELF were consistently preferred.
Conclusion : According to people with RA, leaflets generated from a linguistic framework are clearer and more effective in communicating information about medications.
Practice Implications : The ELF is a user-friendly, structured analytic system that can assist with the development of effective high quality patient information materials.