An institutional study of the influence of ‘onlineness’ on student evaluation of teaching in a dual mode Australian university

Student evaluation of teaching (SET) is now commonplace in many universities internationally. The most common criticism of SET practices is that they are influenced by a number of non-teaching-related factors. More recently, there has been dramatic growth in online education internationally, but only limited research on the use of SET to evaluate online teaching. This paper presents a large-scale and detailed investigation, using the institutional SET data from an Australian university with a significant offering of wholly online units, and whose institutional SET instrument contains items relating to student perceptions of online technologies in teaching and learning. The relationship between educational technology and SET is not neutral. The mean ratings for the ‗online‘ aspects of SET are influenced by factors in the wider teaching and learning environment, and the overall perception of teaching quality is influenced by whether a unit is offered in wholly online mode or not.

Leading the evaluation of institutional online learning environments for quality enhancement in times of change

This paper reports on findings from a nationally funded project which aims to design and implement a quality management framework for online learning environments (OLEs). Evaluation is a key component of any quality management system and it is this aspect of the framework that is the focus of this paper. In developing the framework initial focus groups were conducted at the five participating institutions. These revealed that, although regarded as important, there did not appear to be a shared understanding of the nature and purpose of evaluation. A second series of focus groups revealed there were multiple perspectives arising from those with a vested interest in online learning. These perspectives will be outlined. Overall, how evaluation was undertaken was highly variable within and across the five institutions reflecting where they were at in relation to the development of their OLE.

Analysis of the potential and problems of new institutional economics for third world development

Purpose – The aim of this paper is to provide a critical evaluation of the potential of new institutional economics (NIE) in third world development.

Design/methodology/approach – The paper reviews various theories under NIE from both conceptual and empirical perspectives. It then reviews the various definitions of institutions and show that institutions are essential to overcome problems of information and uncertainty.

Findings – The review finds that weak institutions can undermine development and hence governments in developing countries should strengthen their institutions to provide greater scope for efficient functioning of markets. Where the market does not work owing to high transactions costs, traditional institutions of collective action and group decision making can work and hence need to be recognised.

Research limitations/implications – The major implications of the paper is that in developing countries, a clear understanding of various institutions such as user groups, inter-linked credit markets, rotational irrigation etc. is needed before they are replaced or modified by other institutions. The main limitations of NIE are that there can be capture by elites of various institutional innovations in rural areas, and that it does not explicitly consider income distribution and uncertainty which are glossed over and hence remain areas for future research.

Originality/value – This paper critically reviews the various institutional environments that developing countries face in addressing development issues.

Evaluation of cervial range of motion and isometric nech muscle strngth: reliability and validity

Objective: To examine the test–retest reliability and construct validity of cervical active range of motion and isometric neck muscle strength as measured by the Multi Cervical Rehabilitation Unit (Hanoun Medical Inc., Ontario).
Design: A cross-sectional study.
Setting: Institutional practice.
Subjects: Twenty-one patients with neck pain and 25 healthy volunteers.
Methods: After a trial-run session, active range of motion (AROM) was measured in the subsequent two sessions, with 2–3 days in between. During each session, three measurements were taken for each direction (flexion, extension, lateral flexions and rotations). The measurement of isometric strength was after a 15-minute break following completion of the measurement of AROM. Three measurements were made for each of the six directions (flexion, extension, lateral flexions, protraction and retraction). The software of the Multi Cervical Rehabilitation Unit automatically recorded and calculated the maximum AROM and isometric strength.
Results: There was a good to high level of reliability in the measurement of AROM for both groups of subjects, with intraclass correlation coefficients (ICCs) ranging from 0.81 to 0.96. Results also demonstrated very good to excellent reliability in isometric strength measurement (ICCs ranged from 0.92 to 0.99). Moreover, there was a significant difference in isometric neck muscle strength (p = 0.001) and in AROM (p = 0.034) between the two groups.
Conclusions: The Multi Cervical Rehabilitation Unit was found to be reliable and valid for testing the cervical active range of motion and isometric neck muscle strength for both normal and patient subjects.

Benchmarking publishing activity of U.S. colleges and universities across the leading journals : a grouped evaluation

This paper examines institutional research productivity across two sets of journals – Hult et al. (1997) and Polonsky and Whitelaw (2006), based on institutions' Carnegie Foundation Classification, funding type (private or public) and whether institutions' offer PhD's in marketing. It is identified that while Research Intensive institutions publish more than other types of organizations across journal types, the type of funding and offering PhD's in Marketing are equally if not more important. As such, institutions seeking to compare themselves to other institutions should choose a 'similar' set of institutions on which to base any comparisons.

Anticipating policy and logics of practice : Australian institutional and academic responses to the globalising "quality research" agenda

Research assessment is now an international trend. This article mobilises a critical policy sociology informed by Bourdieu to unpack the differential effects of research policy shifts in Australia on universities, academics and the field of educational research. It argues in anticipating policy moves - from surveying the logics of practice that have emerged elsewhere from research assessment - that institutional, individual and field responses, while specific to the Australian policy context and mix, have assumed a logic of practice counter productive to "quality" research, education as a field, and equity.

Evaluation of a large group method in an educational Institution : the World café versus Large Group facilitation.

This article reports on the use of a large group method, theWorld Café, compared to large group facilitation in an educational institutional setting in Oman. Change is central to many educational institutions as they aspire to ensure quality procedures and processes are implemented and maintained. One of the difficulties of introducing change within this context is the number of stakeholders involved, as well as the inherent hierarchical nature of these settings. Large group methods (LGM) are one such intervention for change that seeks to involve a maximum number of stakeholders and minimise hierarchies, while at the same time encouraging participation and creating engagement. To evaluate LGM within this setting, the study compared two different workshop techniques: one workshop was run using the LGM theWorld Café, while the other was conducted using large group facilitation (LGF). Results indicated that theWorld Café was superior to LGF in terms of increases in participant knowledge and understanding. Participants who took part in the LGM were also significantly more likely to indicate that the technique was beneficial to their learning, compared to those who took part in the LGF. Qualitative data in the form of comments also provide support for LGM. Further research is needed to assess the applicability of the findings in educational settings in other countries. Moreover, more stringent research is required to assess over time, changes in behaviours occurring following LGM, to provide further evidence as to their value in facilitating change in institutional settings.

Assessing the efficacy and standardization potential of five competing venture capital investment evaluation approaches

The research objective behind this article was to perform a critical evaluation and comparison of five representative business plan evaluation aids (BPEAs) to facilitate constructive discussion of the proposition that greater standardization of venture capital decision-making might be both desirable and possible. The five BPEAs were systematically compared using a structured, taxonomic process. The evidence of this investigation suggests a clear superiority of BPEAs that are based on the researched attributes of successful ventures and use actuarial modeling. Discussion centered on the importance of using BPEAs in a quest for greater consistency during venture capital investment decision-making.

Evaluation of the effectiveness of marine protected areas in temperate waters of Australasia

An evaluation of the effectiveness of Marine Protected Areas (MPAs) in temperate waters of Australasia has been conducted for this thesis. The aim was to identify key elements needed in a strategy for establishment and management of MPA’s in temperate waters of Australasia. This aim was achieved by assessing how effective a sample of MPA’s has been in meeting the conservation objectives for their establishment and by identifying factors that have contributed to success or failure of the MPA’s in meeting these objectives. Particular attention was paid to the objectives of ecological sustainability and biodiversity preservation. A MPA for the purposes of this research was defined as an area of coastal or marine environment, with a substantial subtidal component, set aside by law primarily for conservation purposes. The study region encompassed the coastal zones of Victoria, Tasmania and South Australia (Australia) and New Zealand. The questions posed in order to address the aim of this thesis were; a) Have existing MPA’s been effective in achieving conservation objectives? b) What have been the important influences on effectiveness of existing MPA’s? c) What are the key elements required for implementation of effective MPA’s? The thesis is divided into three parts. Part I is a review of the literature on implementation and effectiveness of MPA’s. Part II presents a detailed evaluation of one MPA: Point Lonsdale Marine Reserve (PLMR), Victoria, Australia. Part in is an evaluation of a number of Australasian temperate MPA’s based on information provided in a survey of people involved in management of MPA’s, and from a variety of published and unpublished documents. The MPA’s are described, evidence about ecological effectiveness is presented and factors that have enhanced and limited the ability of these MPA’s to achieve conservation objectives are derived. A substantial amount of scientific evidence was found for increases in abundance, mean size and size range of fish and invertebrate populations within the boundaries of ‘no-take’ MPA’s, Some evidence was found for ‘spillover’ of adults and juveniles into adjacent fishing grounds. Ecological effects detected within ‘no-take’ MPA’s in Australasia matched those described in the literature. The abundance and mean size of a number of previously exploited species have increased, migration into adjacent fishing grounds has been documented, and species richness has increased in at least one MPA. The PLMR was established primarily to protect the scientifically significant intertidal rock platforms. The results of the case study suggest that this objective has been achieved. Opinions about effectiveness were obtained for 28 MPA’s. Of these 19 were considered to be achieving some objectives and 10 were considered to be performing well in terms of overall ecological effectiveness. Positive effects on biodiversity were generally assumed as a result of reduction of damaging anthropogenic effects on habitat. Many questionnaire respondents noted an increase in community awareness about and support for marine conservation as a result of proclamation of MPA’s, Overall, the results support the value of MPA’s for sustainability of fish stocks and preservation of biodiversity, but there is substantial doubt over whether some of the MPA’s are too small to maintain benefits in the long-term. ‘No-take’ MPA’s, particularly those more remote from the impacts of human activities, have been the most effective in achieving objectives. A number of interacting factors important to eventual success of MPA’s were identified. The most important enhancing factors identified for the PLMR were physical attributes that limit the extent of human use and a strong conservation ethic amongst many of the visitors to this marine reserve. Limiting factors were far more numerous. Of most concern is the inadequacy of at-site management. Almost half of visitors to the area were not aware of its marine reserve status, despite the fact many were frequent visitors. The need for better educational and interpretative material on-site is highlighted by the results of the PLMR visitor survey. A total of 56 factors that have enhanced effectiveness and 46 factors that have limited effectiveness of Australasian temperate MPA’s were identified. A number of factors were important in more than one MPA and this was used to derive a set of critical, or key, factors. For example, a conclusion of all three approaches used in this study is that failure to implement day-to-day management through lack of resources was a major constraint on effectiveness. The importance of MPA’s to marine conservation depends, in part, on how well they are managed. The key factors that influence MPA effectiveness were used as the basis for derivation of the main requirements for implementation of MPA’s that will be capable of meeting the objectives for their establishment. The most important needs are: • that ‘no-take’ areas surrounded by buffer zones form the basis for a system of MPA’s; • that a high level of protection is bestowed by legislation and regulations; • that a minimum size be set for the ‘no-take’ core areas; • that the selection of sites for MPA’s takes into account land-based impacts; • that institutional arrangements are developed specifically for MPA’s; • that funding for MPA’s is increased to enable effective management; • that day-to-day management is implemented in all MPA’s, with enforcement and education programs as priority areas; • that a monitoring program for one or more MPA in each 'State' is established to provide evidence of ecological effects of reservation; • that public and stakeholder involvement in development and implementation of MPA’s is encouraged as this will influence the degree of public support and compliance; • that community-based programs to educate the general public, stakeholders, the media and decision-makers about the value of MPA’s are essential; and • that measures to reduce financial impact on affected stakeholders be implemented.

Linking student evaluations to institutional goals : a change story

For the past 30 years, beginning with the seminal work of Herbert Marsh in Australia and New Zealand, institutions of higher education have developed internal practices and procedures to collect and analyse student evaluations of teaching and learning. However, the question remains: has this development resulted in the achievement of institutional goals that maximise learning across all teaching contexts? As is the case in many other countries, a recent review of Australian national student evaluation data failed to conclude that student evaluations have improved overall teaching. However, these data have made student perceptions of teaching and learning more salient in the minds of tertiary educators. Certainly, teaching staff are aware of the impacts of student evaluations on informed decision-making, such as the continuation or discontinuation of courses, and on their promotion processes. This paper will review student evaluation practices according to criteria used in change theories, such as the transtheoretical change model (TTM). TTM construes organisational change as a process involving progress through a series of five stages: precontemplation, contemplation, preparation, action and maintenance. This paper will focus on organisational and behavioural outcomes that can be linked to the use of student data. It will recommend strategies for better aligning evaluation results to the stages of change.

Leading an evidence-based, multi-stakeholder approach to evaluating the implementation of a new online learning environment : an Australian institutional case study

An Australian institutional case study is presented on the evaluation approach being adopted for the implementation of a new online learning environment. Well conceived and inclusive evaluation is seen as essential for the quality management of online learning environments. The focus is on identifying and reconciling the informational needs of various stakeholder groups in the institution, and developing a plan of action covering the key period of implementation. The significant judgements required to carry out evaluation in a multi-campus environment cannot be under-estimated. This is particularly the case given the more recent move to devolving resources and responsibility for the successful implementation of the environment to faculties in the institution concerned. It calls for a more sophisticated conception and set of practices around distributed leadership, as aided by institutions‘ teaching and learning centres. A set of strategic recommendations are offered to help with the evaluation task.

Student evaluation of teaching : keeping in touch with reality

Student evaluation of teaching is commonplace in many universities and may be the predominant input into the performance evaluation of staff and organisational units. This article used publicly available student evaluation of teaching data to present examples of where institutional responses to evaluation processes appeared to be educationally ineffective and where the pursuit of the ‘right’ student evaluation results appears to have been mistakenly equated with the aim of improved teaching and learning. If the vast resources devoted to student evaluation of teaching are to be effective, then the data produced by student evaluation systems must lead to real and sustainable improvements in teaching quality and student learning, rather than becoming an end in itself.