Evaluation in health outcomes research : linking theories, methodologies and practice in health promotion.

Health outcomes research has developed as a means of evaluating the effectiveness of health care interventions and as an approach to informing resource allocation. The use of a health outcomes approach in health promotion has made increasing demands on evaluation methodologies to demonstrate program effectiveness. However, criticism of the contribution of health promotion to outcomes research has made several assumptions about the use of qualitative methodologies and the content of program objectives largely derived from a biomedical approach. In contrast to the measurement of biomedical interventions in clinical health care, health promotion practice involves social phenomena, wide-reaching cultural, psychological, political and ideological problems and issues. The integration of methodologies of health promotion evaluation will inform further conceptualisation of the health outcomes approach with the differentiation of three types of outcomes: health development outcomes; social health outcomes; and biomedical health outcomes. It is concluded that this differentiation moves away from dualist concepts that advocate the replacement of goals and targets with regional and locally based approaches. Rather, the future direction for health promotion evaluation needs to employ a framework that elaborates multiple methodologies and approaches necessary for establishing what relationships exist between morbidity, mortality, health advancement and equity.

A methodological evaluation of business process compliance management frameworks

Existing compliance management frameworks (CMFs) offer a multitude of compliance management capabilities that makes difficult for enterprises to decide on the suitability of a framework. Making a decision on the suitability requires a deep understanding of the functionalities of a framework. Gaining such an understanding is a difficult task which, in turn, requires specialised tools and methodologies for evaluation. Current compliance research lacks such tools and methodologies for evaluating CMFs. This paper reports a methodological evaluation of existing CMFs based on a pre-defined evaluation criteria. Our evaluation highlights what existing CMFs offer, and what they cannot. Also, it underpins various open questions and discusses the challenges in this direction.

REFRAME : a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

ClusterEval 1.0 : Cluster quality Evaluation software

This report describes the available functionality and use of the ClusterEval evaluation software. It implements novel and standard measures for the evaluation of cluster quality. This software has been used at the INEX XML Mining track and in the MediaEval Social Event Detection task.

Identification and evaluation of service bundles for governmental one-stop portals

Recent advances in the area of ‘Transformational Government’ position the citizen at the centre of focus. This paradigm shift from a department-centric to a citizen-centric focus requires governments to re-think their approach to service delivery, thereby decreasing costs and increasing citizen satisfaction. The introduction of franchises as a virtual business layer between the departments and their citizens is intended to provide a solution. Franchises are structured to address the needs of citizens independent of internal departmental structures. For delivering services online, governments pursue the development of a One-Stop Portal, which structures information and services through those franchises. Thus, each franchise can be mapped to a specific service bundle, which groups together services that are deemed to be of relevance to a specific citizen need. This study focuses on the development and evaluation of these service bundles. In particular, two research questions guide the line of investigation of this study: Research Question 1): What methods can be used by governments to identify service bundles as part of governmental One-Stop Portals? Research Question 2): How can the quality of service bundles in governmental One-Stop Portals be evaluated? The first research question asks about the identification of suitable service bundle identification methods. A literature review was conducted, to, initially, conceptualise the service bundling task, in general. As a consequence, a 4-layer model of service bundling and a morphological box were created, detailing characteristics that are of relevance when identifying service bundles. Furthermore, a literature review of Decision-Support Systems was conducted to identify approaches of relevance in different bundling scenarios. These initial findings were complemented by targeted studies of multiple leading governments in the e-government domain, as well as with a local expert in the field. Here, the aim was to identify the current status of online service delivery and service bundling in practice. These findings led to the conceptualising of two service bundle identification methods, applicable in the context of Queensland Government: On the one hand, a provider-driven approach, based on service description languages, attributes, and relationships between services was conceptualised. As well, a citizen-driven approach, based on analysing the outcomes from content identification and grouping workshops with citizens, was also conceptualised. Both methods were then applied and evaluated in practice. The conceptualisation of the provider-driven method for service bundling required the initial specification of relevant attributes that could be used to identify similarities between services called relationships; these relationships then formed the basis for the identification of service bundles. This study conceptualised and defined seven relationships, namely ‘Co-location’, ‘Resource’, ‘Co-occurrence’, ‘Event’, ‘Consumer’, ‘Provider’, and ‘Type’. The relationships, and the bundling method itself, were applied and refined as part of six Action Research cycles in collaboration with the Queensland Government. The findings show that attributes and relationships can be used effectively as a means for bundle identification, if distinct decision rules are in place to prescribe how services are to be identified. For the conceptualisation of the citizen-driven method, insights from the case studies led to the decision to involve citizens, through card sorting activities. Based on an initial list of services, relevant for a certain franchise, participating citizens grouped services according to their liking. The card sorting activity, as well as the required analysis and aggregation of the individual card sorting results, was analysed in depth as part of this study. A framework was developed that can be used as a decision-support tool to assist with the decision of what card sorting analysis method should be utilised in a given scenario. The characteristic features associated with card sorting in a government context led to the decision to utilise statistical analysis approaches, such as cluster analysis and factor analysis, to aggregate card sorting results. The second research question asks how the quality of service bundles can be assessed. An extensive literature review was conducted focussing on bundle, portal, and e-service quality. It was found that different studies use different constructs, terminology, and units of analysis, which makes comparing these models a difficult task. As a direct result, a framework was conceptualised, that can be used to position past and future studies in this research domain. Complementing the literature review, interviews conducted as part of the case studies with leaders in e-government, indicated that, typically, satisfaction is evaluated for the overall portal once the portal is online, but quality tests are not conducted during the development phase. Consequently, a research model which appropriately defines perceived service bundle quality would need to be developed from scratch. Based on existing theory, such as Theory of Reasoned Action, Expectation Confirmation Theory, and Theory of Affordances, perceived service bundle quality was defined as an inferential belief. Perceived service bundle quality was positioned within the nomological net of services. Based on the literature analysis on quality, and on the subsequent work of a focus group, the hypothesised antecedents (descriptive beliefs) of the construct and the associated question items were defined and the research model conceptualised. The model was then tested, refined, and finally validated during six Action Research cycles. Results show no significant difference in higher quality or higher satisfaction among users for either the provider-driven method or for the citizen-driven method. The decision on which method to choose, it was found, should be based on contextual factors, such as objectives, resources, and the need for visibility. The constructs of the bundle quality model were examined. While the quality of bundles identified through the citizen-centric approach could be explained through the constructs ‘Navigation’, ‘Ease of Understanding’, and ‘Organisation’, bundles identified through the provider-driven approach could be explained solely through the constructs ‘Navigation’ and ‘Ease of Understanding’. An active labelling style for bundles, as part of the provider-driven Information Architecture, had a larger impact on ‘Quality’ than the topical labelling style used in the citizen-centric Information Architecture. However, ‘Organisation’, reflecting the internal, logical structure of the Information Architecture, was a significant factor impacting on ‘Quality’ only in the citizen-driven Information Architecture. Hence, it was concluded that active labelling can compensate for a lack of logical structure. Further studies are needed to further test this conjecture. Such studies may involve building alternative models and conducting additional empirical research (e.g. use of an active labelling style for the citizen-driven Information Architecture). This thesis contributes to the body of knowledge in several ways. Firstly, it presents an empirically validated model of the factors explaining and predicting a citizen’s perception of service bundle quality. Secondly, it provides two alternative methods that can be used by governments to identify service bundles in structuring the content of a One-Stop Portal. Thirdly, this thesis provides a detailed narrative to suggest how the recent paradigm shift in the public domain, towards a citizen-centric focus, can be pursued by governments; the research methodology followed by this study can serve as an exemplar for governments seeking to achieve a citizen-centric approach to service delivery.

Quantitative evaluation of polymer gel dosimeters by broadband ultrasound attenuation

Ultrasound has been examined previously as an alternative readout method for irradiated polymer gel dosimeters, with authors reporting varying dose response to ultrasound transmission measurements. In this current work we extend previous work to measure the broadband ultrasound attenuation (BUA) response of irradiated PAGAT gel dosimeters, using a novel ultrasound computed tomography system.

Quality Assessment Matrix : A Collaborative Evaluation Using Multiple Lines of Evidence

Welcome to the Quality assessment matrix. This matrix is designed for highly qualified discipline experts to evaluate their course, major or unit in a systematic manner. The primary purpose of the Quality assessment matrix is to provide a tool that a group of academic staff at universities can collaboratively review the assessment within a course, major or unit annually. The annual review will result in you being read for an external curricula review at any point in time. This tool is designed for use in a workshop format with one, two or more academic staff, and will lead to an action plan for implementation.

From evidence to action: a shared leadership approach

This paper proposes an organisational effectiveness model that applies the theoretical frameworks of shared leadership, appreciative inquiry, and knowledge creation. Similar to many libraries worldwide, Auraria Library technical services department struggled to establish efficient and effective workflow for electronic resources management. The library purchased an Electronic Resource Management System, as the literature suggests; however, this technology-enabled system did not resolve workflow issues. The Auraria Library case study demonstrates that a technical services division can successfully reorganize personnel, reassign responsibilities, and measure outcomes within an evidencebased shared leadership culture, which invites and enables participants to identify problems and create solutions amidst a dynamically changing electronic resources environment.

Distributing leadership and cultivating dialogue with collaborative EBIP

The purpose of this paper is to demonstrate the efficacy of collaborative evidence based information practice (EBIP) as an organizational effectiveness model. Shared leadership, appreciative inquiry and knowledge creation theoretical frameworks provide the foundation for change toward the implementation of a collaborative EBIP workplace model. Collaborative EBIP reiterates the importance of gathering the best available evidence, but it differs by shifting decision-making authority from "library or employer centric" to "user or employee centric". University of Colorado Denver Auraria Library Technical Services department created a collaborative EBIP environment by flattening workplace hierarchies, distributing problem solving and encouraging reflective dialogue. By doing so, participants are empowered to identify problems, create solutions, and become valued and respected leaders and followers. In an environment where library budgets are in jeopardy, recruitment opportunities are limited and the workplace is in constant flux, the Auraria Library case study offers an approach that maximizes the capability of the current workforce and promotes agile responsiveness to industry and organizational challenges. Collaborative EBIP is an organizational model demonstrating a process focusing first on the individual and moving to the collective to develop a responsive and high performing business unit, and in turn, organization.

Proximal and distal forms of visionary leadership : a stress buffering perspective

Occupational stress research has consistently demonstrated many negative effects of work stressors on employee adjustment (i.e., job-related attitudes and health). Considerable literature also describes potential moderators of this relationship. While research has revealed that leader vision can have significant positive effects on employee adjustment it has neglected investigation of its potential stress buffering effects. Based on leadership and identification theories, it was predicted that stress buffering effects of leader vision would be more evident for distal versus proximal leaders and also when the distal or proximal leader vision was congruent with the employee adjustment outcome type (distal versus proximal). Predictions were tested with an employee sample from one large public sector organization (N = 519). Hierarchical multiple regression analyses revealed that main and moderated effects relating to leader vision supported the notion that occupational stress would be reduced when there was congruence of distal and proximal leader vision and distal and proximal outcome types. However, stress buffering effects were found for high and low perceivers of leader vision that were not in line with hypotheses posing questions for the definitions of distal and proximal identifications. Findings are discussed in terms of theoretical and practical implications.

Enhancing critical analysis and problem-solving skills in undergraduate psychology : An evaluation of a collaborative learning and problem-based learning approach

Critical analysis and problem-solving skills are two graduate attributes that are important in ensuring that graduates are well equipped in working across research and practice settings within the discipline of psychology. Despite the importance of these skills, few psychology undergraduate programmes have undertaken any systematic development, implementation, and evaluation of curriculum activities to foster these graduate skills. The current study reports on the development and implementation of a tutorial programme designed to enhance the critical analysis and problem-solving skills of undergraduate psychology students. Underpinned by collaborative learning and problem-based learning, the tutorial programme was administered to 273 third year undergraduate students in psychology. Latent Growth Curve Modelling revealed that students demonstrated a significant linear increase in self-reported critical analysis and problem-solving skills across the tutorial programme. The findings suggest that the development of inquiry-based curriculum offers important opportunities for psychology undergraduates to develop critical analysis and problem-solving skills.

Leadership of public education : an exploration of executive leaders in education Queensland

Education reform aimed at achieving improved student learning is a demanding challenge for leaders at all levels of education across the globe. In Queensland, Australia, Assistant Regional Directors, School Performance (ARD-SP) of public schools are executive leaders at the forefront of this challenge, working with groups, clusters, or networks of schools and one-on-one with principals, focusing on the performance of their schools. The ARD-SP role was recently established to positively impact student learning across the entire public school system in Queensland. The proposed study aimed to capture how ARDs-SP conceptualise and enact their leadership role. The study utilised a micropolitical perspective of leadership to understand the way in which these leaders talked about their leadership practices, their challenges, and the wider contextual factors impacting upon their work. A case study methodology guided the study and allowed ARDs-SP to share their understandings and enactment of executive leadership. A conceptual framework drawing upon the micropolitical leadership framework of Blase and Anderson (1995) was employed to analyse the research data gathered. Data were collected from Education Queensland (EQ) (i.e. that sector of the Department of Education and Training in Queensland responsible for public schools) policy material and reports and two rounds of semi-structured interviews with 18 ARD-SP participants and two senior EQ executives. The findings of this study were initially presented as four themes: performance, supervision, professional challenge, and system sustainability. They were then considered in the light of the literature and explored through the macro, meso, and micro layers within the conceptual framework. The key findings of this study found that ARDs-SP referred to using two different leadership approaches (i.e. an adversarial approach and/or a facilitative approach) when supervising school principals and the approach employed depended primarily upon the perceived performance of the principal. It was also found that the notion of supervision embedded within the role was perceived by ARDs-SP as problematic. These findings imply opportunities to refine the role and in doing so harness other system improvement strategies for EQ. An important contribution of this study was a reconceptualised conceptual framework that showed leadership approaches used by ARDs-SP as falling upon a continuum.

The design science research roadmap : in progress evaluation

Although Design Science Research (DSR) is now an accepted approach to research in the Information Systems (IS) discipline, consensus on the methodology of DSR has yet to be achieved. Lack of a comprehensive and detailed methodology for Design Science Research (DSR) in the Information System (IS) discipline is a main issue. Prior research (the parent-study) aimed to remedy this situation and resulted in the DSR-Roadmap (Alturki et al., 2011a). Continuing empirical validation and revision of the DSR-Roadmap strives towards a methodology with appropriate levels of detail, integration, and completeness for novice researchers to efficiently and effectively conduct and report DSR in IS. The sub-study reported herein contributes to this larger, ongoing effort. This paper reports results from a formative evaluation effort of the DSR-Roadmap conducted using focus group analysis. Generally, participants endorsed the utility and intuitiveness of the DSR-Roadmap, while also suggesting valuable refinements. Both parent-study and sub-study make methodological contributions. The parent-study is the first attempt of utilizing DSR to develop a research methodology showing an example of how to use DSR in research methodology construction. The sub-study demonstrates the value of the focus group method in DSR for formative product evaluation.

Anthropometry and body composition of Indonesian adults : an evaluation of body image, eating behaviours, and physical activity

This thesis examines the characteristics of anthropometry and body composition in Indonesian adults and some of the risk factors including body image, eating behaviours, and physical activity. Examination on body image, eating behaviours, and physical activity demonstrates significant correlations with anthropometry and body composition in Indonesian adults. The study also identified body image distortion in some of the participants and provides suggestions for intervention development addressed to the groups of participants which have been identified as having a distorted body image.