Validating the is-impact model in the Malaysian public sector

This research is one of several ongoing studies conducted within the IT Professional Services (ITPS) research programme at Queensland University of Technology (QUT). In 2003, ITPS introduced the IS-Impact model, a measurement model for measuring information systems success from the viewpoint of multiple stakeholders. The model, along with its instrument, is robust, simple, yet generalisable, and yields results that are comparable across time, stakeholders, different systems and system contexts. The IS-Impact model is defined as “a measure at a point in time, of the stream of net benefits from the Information System (IS), to date and anticipated, as perceived by all key-user-groups”. The model represents four dimensions, which are ‘Individual Impact’, ‘Organizational Impact’, ‘Information Quality’ and ‘System Quality’. The two Impact dimensions measure the up-to-date impact of the evaluated system, while the remaining two Quality dimensions act as proxies for probable future impacts (Gable, Sedera & Chan, 2008). To fulfil the goal of ITPS, “to develop the most widely employed model” this research re-validates and extends the IS-Impact model in a new context. This method/context-extension research aims to test the generalisability of the model by addressing known limitations of the model. One of the limitations of the model relates to the extent of external validity of the model. In order to gain wide acceptance, a model should be consistent and work well in different contexts. The IS-Impact model, however, was only validated in the Australian context, and packaged software was chosen as the IS understudy. Thus, this study is concerned with whether the model can be applied in another different context. Aiming for a robust and standardised measurement model that can be used across different contexts, this research re-validates and extends the IS-Impact model and its instrument to public sector organisations in Malaysia. The overarching research question (managerial question) of this research is “How can public sector organisations in Malaysia measure the impact of information systems systematically and effectively?” With two main objectives, the managerial question is broken down into two specific research questions. The first research question addresses the applicability (relevance) of the dimensions and measures of the IS-Impact model in the Malaysian context. Moreover, this research question addresses the completeness of the model in the new context. Initially, this research assumes that the dimensions and measures of the IS-Impact model are sufficient for the new context. However, some IS researchers suggest that the selection of measures needs to be done purposely for different contextual settings (DeLone & McLean, 1992, Rai, Lang & Welker, 2002). Thus, the first research question is as follows, “Is the IS-Impact model complete for measuring the impact of IS in Malaysian public sector organisations?” [RQ1]. The IS-Impact model is a multidimensional model that consists of four dimensions or constructs. Each dimension is represented by formative measures or indicators. Formative measures are known as composite variables because these measures make up or form the construct, or, in this case, the dimension in the IS-Impact model. These formative measures define different aspects of the dimension, thus, a measurement model of this kind needs to be tested not just on the structural relationship between the constructs but also the validity of each measure. In a previous study, the IS-Impact model was validated using formative validation techniques, as proposed in the literature (i.e., Diamantopoulos and Winklhofer, 2001, Diamantopoulos and Siguaw, 2006, Petter, Straub and Rai, 2007). However, there is potential for improving the validation testing of the model by adding more criterion or dependent variables. This includes identifying a consequence of the IS-Impact construct for the purpose of validation. Moreover, a different approach is employed in this research, whereby the validity of the model is tested using the Partial Least Squares (PLS) method, a component-based structural equation modelling (SEM) technique. Thus, the second research question addresses the construct validation of the IS-Impact model; “Is the IS-Impact model valid as a multidimensional formative construct?” [RQ2]. This study employs two rounds of surveys, each having a different and specific aim. The first is qualitative and exploratory, aiming to investigate the applicability and sufficiency of the IS-Impact dimensions and measures in the new context. This survey was conducted in a state government in Malaysia. A total of 77 valid responses were received, yielding 278 impact statements. The results from the qualitative analysis demonstrate the applicability of most of the IS-Impact measures. The analysis also shows a significant new measure having emerged from the context. This new measure was added as one of the System Quality measures. The second survey is a quantitative survey that aims to operationalise the measures identified from the qualitative analysis and rigorously validate the model. This survey was conducted in four state governments (including the state government that was involved in the first survey). A total of 254 valid responses were used in the data analysis. Data was analysed using structural equation modelling techniques, following the guidelines for formative construct validation, to test the validity and reliability of the constructs in the model. This study is the first research that extends the complete IS-Impact model in a new context that is different in terms of nationality, language and the type of information system (IS). The main contribution of this research is to present a comprehensive, up-to-date IS-Impact model, which has been validated in the new context. The study has accomplished its purpose of testing the generalisability of the IS-Impact model and continuing the IS evaluation research by extending it in the Malaysian context. A further contribution is a validated Malaysian language IS-Impact measurement instrument. It is hoped that the validated Malaysian IS-Impact instrument will encourage related IS research in Malaysia, and that the demonstrated model validity and generalisability will encourage a cumulative tradition of research previously not possible. The study entailed several methodological improvements on prior work, including: (1) new criterion measures for the overall IS-Impact construct employed in ‘identification through measurement relations’; (2) a stronger, multi-item ‘Satisfaction’ construct, employed in ‘identification through structural relations’; (3) an alternative version of the main survey instrument in which items are randomized (rather than blocked) for comparison with the main survey data, in attention to possible common method variance (no significant differences between these two survey instruments were observed); (4) demonstrates a validation process of formative indexes of a multidimensional, second-order construct (existing examples mostly involved unidimensional constructs); (5) testing the presence of suppressor effects that influence the significance of some measures and dimensions in the model; and (6) demonstrates the effect of an imbalanced number of measures within a construct to the contribution power of each dimension in a multidimensional model.

Is anybody there? Bootstrapping attendance with engagement

The report card for the introductory programming unit at our university has historically been unremarkable in terms of attendance rates, student success rates and student retention in both the unit and the degree course. After a course restructure recently involving a fresh approach to introducing programming, we reported a high retention in the unit, with consistently high attendance and a very low failure rate. Following those encouraging results, we collected student attendance data for several semesters and compared attendance rates to student results. We have found that interesting workshop material which directly relates to course-relevant assessment items and therefore drives the learning, in an engaging collaborative learning environment has improved attendance to an extraordinary extent, with student failure rates plummeting to the lowest in recorded history at our university.

User behaviour modelling in a multi-dimensional environment for personalization and recommendation

Handling information overload online, from the user's point of view is a big challenge, especially when the number of websites is growing rapidly due to growth in e-commerce and other related activities. Personalization based on user needs is the key to solving the problem of information overload. Personalization methods help in identifying relevant information, which may be liked by a user. User profile and object profile are the important elements of a personalization system. When creating user and object profiles, most of the existing methods adopt two-dimensional similarity methods based on vector or matrix models in order to find inter-user and inter-object similarity. Moreover, for recommending similar objects to users, personalization systems use the users-users, items-items and users-items similarity measures. In most cases similarity measures such as Euclidian, Manhattan, cosine and many others based on vector or matrix methods are used to find the similarities. Web logs are high-dimensional datasets, consisting of multiple users, multiple searches with many attributes to each. Two-dimensional data analysis methods may often overlook latent relationships that may exist between users and items. In contrast to other studies, this thesis utilises tensors, the high-dimensional data models, to build user and object profiles and to find the inter-relationships between users-users and users-items. To create an improved personalized Web system, this thesis proposes to build three types of profiles: individual user, group users and object profiles utilising decomposition factors of tensor data models. A hybrid recommendation approach utilising group profiles (forming the basis of a collaborative filtering method) and object profiles (forming the basis of a content-based method) in conjunction with individual user profiles (forming the basis of a model based approach) is proposed for making effective recommendations. A tensor-based clustering method is proposed that utilises the outcomes of popular tensor decomposition techniques such as PARAFAC, Tucker and HOSVD to group similar instances. An individual user profile, showing the user's highest interest, is represented by the top dimension values, extracted from the component matrix obtained after tensor decomposition. A group profile, showing similar users and their highest interest, is built by clustering similar users based on tensor decomposed values. A group profile is represented by the top association rules (containing various unique object combinations) that are derived from the searches made by the users of the cluster. An object profile is created to represent similar objects clustered on the basis of their similarity of features. Depending on the category of a user (known, anonymous or frequent visitor to the website), any of the profiles or their combinations is used for making personalized recommendations. A ranking algorithm is also proposed that utilizes the personalized information to order and rank the recommendations. The proposed methodology is evaluated on data collected from a real life car website. Empirical analysis confirms the effectiveness of recommendations made by the proposed approach over other collaborative filtering and content-based recommendation approaches based on two-dimensional data analysis methods.

Associations between young children's emotion attributions and prediction of outcome in differing social situations

Associations between young children's attributions of emotion at different points in a story, and with regard to their own prediction about the story's outcome, were investigated using two hypothetical scenarios of social and emotional challenge (social entry and negative event). First grade children (N = 250) showed an understanding that emotions are tied to situational cues by varying the emotions they attributed both between and within scenarios. Furthermore, emotions attributed to the main protagonist at the beginning of the scenarios were differentially associated with children's prediction of a positive or negative outcome and with the valence of the emotion attributed at the end of the scenario. Gender differences in responses to some items were also found. © 2010 The British Psychological Society.

Understanding graphicacy : students’ making sense of graphics in mathematics assessment tasks

The ability to decode graphics is an increasingly important component of mathematics assessment and curricula. This study examined 50, 9- to 10-year-old students (23 male, 27 female), as they solved items from six distinct graphical languages (e.g., maps) that are commonly used to convey mathematical information. The results of the study revealed: 1) factors which contribute to success or hinder performance on tasks with various graphical representations; and 2) how the literacy and graphical demands of tasks influence the mathematical sense making of students. The outcomes of this study highlight the changing nature of assessment in school mathematics and identify the function and influence of graphics in the design of assessment tasks.

Confirmatory factor analysis of the Behaviour of Young Novice Drivers Scale (BYNDS)

The greatly increased risk of being killed or injured in a car crash for the young novice driver has been recognised in the road safety and injury prevention literature for decades. Risky driving behaviour has consistently been found to contribute to traffic crashes. Researchers have devised a number of instruments to measure this risky driving behaviour. One tool developed specifically to measure the risky behaviour of young novice drivers is the Behaviour of Young Novice Drivers Scale (BYNDS) (Scott-Parker et al., 2010). The BYNDS consists of 44 items comprising five subscales for transient violations, fixed violations, misjudgement, risky driving exposure, and driving in response to their mood. The factor structure of the BYNDS has not been examined since its development in a matched sample of 476 novice drivers aged 17-25 years. Method: The current research attempted to refine the BYNDS and explore its relationship with the self-reported crash and offence involvement and driving intentions of 390 drivers aged 17-25 years (M = 18.23, SD = 1.58) in Queensland, Australia, during their first six months of independent driving with a Provisional (intermediate) driver’s licence. A confirmatory factor analysis was undertaken examining the fit of the originally proposed BYNDS measurement model. Results: The model was not a good fit to the data. A number of iterations removed items with low factor loadings, resulting in a 36-item revised BYNDS which was a good fit to the data. The revised BYNDS was highly internally consistent. Crashes were associated with fixed violations, risky driving exposure, and misjudgement; offences were moderately associated with risky driving exposure and transient violations; and road-rule compliance intentions were highly associated with transient violations. Conclusions: Applications of the BYNDS in other young novice driver populations will further explore the factor structure of both the original and revised BYNDS. The relationships between BYNDS subscales and self-reported risky behaviour and attitudes can also inform countermeasure development, such as targeting young novice driver non-compliance through enforcement and education initiatives.

Trust-based automated recommendation making

Recommender systems are one of the recent inventions to deal with ever growing information overload in relation to the selection of goods and services in a global economy. Collaborative Filtering (CF) is one of the most popular techniques in recommender systems. The CF recommends items to a target user based on the preferences of a set of similar users known as the neighbours, generated from a database made up of the preferences of past users. With sufficient background information of item ratings, its performance is promising enough but research shows that it performs very poorly in a cold start situation where there is not enough previous rating data. As an alternative to ratings, trust between the users could be used to choose the neighbour for recommendation making. Better recommendations can be achieved using an inferred trust network which mimics the real world "friend of a friend" recommendations. To extend the boundaries of the neighbour, an effective trust inference technique is required. This thesis proposes a trust interference technique called Directed Series Parallel Graph (DSPG) which performs better than other popular trust inference algorithms such as TidalTrust and MoleTrust. Another problem is that reliable explicit trust data is not always available. In real life, people trust "word of mouth" recommendations made by people with similar interests. This is often assumed in the recommender system. By conducting a survey, we can confirm that interest similarity has a positive relationship with trust and this can be used to generate a trust network for recommendation. In this research, we also propose a new method called SimTrust for developing trust networks based on user's interest similarity in the absence of explicit trust data. To identify the interest similarity, we use user's personalised tagging information. However, we are interested in what resources the user chooses to tag, rather than the text of the tag applied. The commonalities of the resources being tagged by the users can be used to form the neighbours used in the automated recommender system. Our experimental results show that our proposed tag-similarity based method outperforms the traditional collaborative filtering approach which usually uses rating data.

Communication, Cultural and Media Studies : The Key Concepts (4th Edition)

This fourth edition of Communication, Cultural and Media Studies: The Key Concepts is an indispensible guide to the most important terms in the field. It offers clear explanations of the key concepts, exploring their origins, what they’re used for and why they provoke discussion. The author provides a multi-disciplinary explanation and assessment of the key concepts, from ‘authorship’ to ‘censorship’; ‘creative industries’ to ‘network theory’; ‘complexity’ to ‘visual culture’. The new edition of this classic text includes: * Over 200 entries including 50 new entries * All entries revised, rewritten and updated * Coverage of recent developments in the field * Insight into interactive media and the knowledge-based economy * A fully updated bibliography with 400 items and suggestions for further reading throughout the text

The relation between work motivation and project management success : an empirical investigation

The content and context of work significantly influences an employees’ satisfaction. While managers see work motivation as a tool to engage the employees so that they perform better, academicians value work motivation for its contribution to human behaviour. Though the relationship between employee motivation and project success has been extensively covered in the literature, more research focusing on the nature of job design on project success may have been wanting. We address this gap through this study. The present study contributes to the extant literature by suggesting an operational framework of work motivation for project—based organizations. We are also advancing the conceptual understanding of this variable by understanding how the different facets of work motivation have a differing impact of the various parameters of project performance. A survey instrument using standardized scales of work motivation and project success was used. 199 project workers from various industries completed the survey. We first ‘operationalized’ the definition of work motivation for the purpose of our study through a principal component analysis of work motivation items. We obtained a five factor structure that had items pertaining to employee development, work climate, goal clarity, and job security. We then performed a Pearson’s correlation analysis which revealed moderate to significant relationship between project outcomes ad work climate; project outcomes & employee development. In order to establish a causality between work motivation and project management success, we employed linear regression analysis. The results show that work climate is a significant predictor of client satisfaction, while it moderately influences the project quality. Further, bringing in objectivity to project work is important for a successful implementation.

Using technology to facilitate grading consistency in large classes

University classes in marketing are often large, and therefore require teams of teachers to cover all of the necessary activities. A major problem with teaching teams is the inconsistency that results from myriad individuals offering subjective opinions. This innovation uses the latest moderation techniques along with Audience Response Technology (ART) to enhance the learning experience by providing more consistent and reliable grading in large classes. Assessment items are moderated before they are graded in meetings that employ ART. Results show the process is effective when the teaching team is very large, or there is a diverse range of experienced and inexperienced teachers. This “behind the scenes” innovation is not immediately apparent to students, but results in more consistent grades, more useful feedback for students, and more confident graders.

The development of the graphics-decoding proficiency instrument

The Graphics-Decoding Proficiency (G-DP) instrument was developed as a screening test for the purpose of measuring students’ (aged 8-11 years) capacity to solve graphics-based mathematics tasks. These tasks include number lines, column graphs, maps and pie charts. The instrument was developed within a theoretical framework which highlights the various types of information graphics commonly presented to students in large-scale national and international assessments. The instrument provides researchers, classroom teachers and test designers with an assessment tool which measures students’ graphics decoding proficiency across and within five broad categories of information graphics. The instrument has implications for a number of stakeholders in an era where graphics have become an increasingly important way of representing information.

Submission to discussion paper : Queensland retirement villages

This submission is directed to addressing Issue 4 only – Entering a village and closure – implications for residents; and one question only – What particular items do you consider should be particularly addressed as part of the disclosure requirements for prospective retirement village residents? The recommendations below are premised on the basis that information to enable informed choice needs to be clearly presented, easy to understand and, while legal advice and assistance remains essential for any form of ‘conveyancing’ process, should enable decision making by prospective residents independent of that legal advice.

"Participation" in participatory design research

This thesis is about defining participation in the context of fostering research cohesion in the field of Participatory Design. The systematic and incremental building of new knowledge is the process by which science and research is advanced. This process requires a certain type of cohesion in the way research is undertaken for new knowledge to be built from the knowledge provided by previous projects and research. To support this process and to foster research cohesion three conditions are necessary. These conditions are: common ground between practitioners, problem-space positioning, and adherence to clear research criteria. The challenge of fostering research cohesion in Participatory Design is apparent in at least four themes raised in the literature: the role of politics within Participatory Design epistemology, the role of participation, design with users, and the ability to translate theory into practice. These four thematic challenges frame the context which the research gap is situated. These themes are also further investigated and the research gap – a general lack of research cohesion – along with one avenue for addressing this gap – a clear and operationalizable definition for participation – are identified. The intended contribution of this thesis is to develop a framework and visual tool to address this research gap. In particular, an initial approximation for a clear and operationalizable definition for participation will be proposed such that it can be used within the field of Participatory Design to run projects and foster research cohesion. In pursuit of this contribution, a critical lens is developed and used to analyse some of the principles and practices of Participatory Design that are regarded as foundational. This lens addresses how to define participation in a way that adheres to basic principles of scientific rigour – namely, ensuring that the elements of a theory are operationalizable, falsifiable, generalizable, and useful, and it also treats participation as a construct rather than treating the notion of participation as a variable. A systematic analysis is performed using this lens on the principles and practices that are considered foundational within the field. From this analysis, three components of the participation construct – impact, influence, and agency – are identified. These components are then broken down into two constituent variables each (six in all) and represented visually. Impact is described as the relationship between the quality and use of information. Influence is described as the relationship between the amount and scope of decision making. Agency is described as the relationship between the motivation of the participant and the solidarity of the group. Thus, as a construct, participation is described as the relationship between a participant’s impact, influence, and agency. In the concluding section, the value of this participation construct is explored for its utility in enhancing project work and fostering research cohesion. Three items of potential value that emerge are: the creation of a visual tool through the representation of these six constituent variables in one image; the elaboration of a common language for researchers based on the six constituent variables identified; and the ability to systematically identify and remedy participation gaps throughout the life of the project. While future research exploring the applicability of the participation construct in real world projects is necessary, it is intended that this initial approximation of a participation construct in the form of the visual tool will serve as the basis for a cohesive and rigorous discussion about participation in Participatory Design.

Visual support for work assignment in process-aware information systems : framework formalisation and implementation

Process-aware information systems, ranging from generic workflow systems to dedicated enterprise information systems, use work-lists to offer so-called work items to users. In real scenarios, users can be confronted with a very large number of work items that stem from multiple cases of different processes. In this jungle of work items, users may find it hard to choose the right item to work on next. The system cannot autonomously decide which is the right work item, since the decision is also dependent on conditions that are somehow outside the system. For instance, what is “best” for an organisation should be mediated with what is “best” for its employees. Current work-list handlers show work items as a simple sorted list and therefore do not provide much decision support for choosing the right work item. Since the work-list handler is the dominant interface between the system and its users, it is worthwhile to provide an intuitive graphical interface that uses contextual information about work items and users to provide suggestions about prioritisation of work items. This paper uses the so-called map metaphor to visualise work items and resources (e.g., users) in a sophisticated manner. Moreover, based on distance notions, the work-list handler can suggest the next work item by considering different perspectives. For example, urgent work items of a type that suits the user may be highlighted. The underlying map and distance notions may be of a geographical nature (e.g., a map of a city or office building), but may also be based on process designs, organisational structures, social networks, due dates, calendars, etc. The framework proposed in this paper is generic and can be applied to any process-aware information system. Moreover, in order to show its practical feasibility, the paper discusses a full-fledged implementation developed in the context of the open-source workflow environment YAWL, together with two real examples stemming from two very different scenarios. The results of an initial usability evaluation of the implementation are also presented, which provide a first indication of the validity of the approach.

Job satisfaction and intention to leave among critical care nurses in Saudi Arabia

Aim The purpose of this study was to examine the relationship between registered nurses’ (RN) job satisfaction and their intention to leave critical care nursing in Saudi Arabia. Background Many studies have identified critical care areas as stressful work environments for nurses and have identified factors contributing to job satisfaction and staff retention. However, very little research has examined these relationships in the Saudi context. Design and Methods This study utilised an exploratory, cross-sectional survey design to examine the relationship between RN job satisfaction and intention to leave at King Abdul-Aziz University Hospital, Saudi Arabia. Respondents completed a self-administered survey including demographic items and validated measures of job satisfaction and intention to leave. A convenience sample of 182 RNs working in critical care areas during the data collection period were included. Results Regression analysis predicting RN intention to leave found that demographic variables including age, parental status and length of ICU experience, and three of the job satisfaction subscales including perceived workload, professional support and pay and prospects for promotion, were significantly associated with the outcome variable. Conclusion This study adds to the existing literature on the relationship between job satisfaction and intention to leave critical care areas among RNs working in Saudi Arabia. These findings point to the need for management and policy interventions targeting nurses’ workloads, professional support and pay and promotion in order to improve nurse retention.