Performance measurement and feedback in a public sector program

Establishing a framework for measuring the performance of public sector programs is fraught with dangers. Many public sector organisations are satisfied with measuring activity in programs and fail to see the need for establishing a framework that will meet the needs of participants and measure outcomes as well as activities. This paper explores how a government department in Queensland went about establishing a performance management framework to measure the outcomes and activities in a program that was a public private partnership. Findings indicate that using an iterative consultative approach performance measure can be put in place that are meaningful and assist the participants to review the program.

Measuring internal market orientation

Internal marketing has been discussed in the management and academic literature for over three decades, yet it remains ill defined and poorly operationalized. This paper responds to calls for research to develop a single clear understanding of the construct, for the development of a suitable instrument to measure it, and for empirical evidence of its impact. Existing, divergent conceptualization of internal marketing are explored, and a new, multidimensional construct, describing the managerial behaviors associated with internal marketing is developed, and termed internal market orientation (IMO). IMO represents the adaptation of market orientation to the context of employer-employee exchanges in the internal market. The paper describes the development of a valid and reliable measure of IMO in a retail services context. Five dimensions of IMO are identified and confirmed. These are 1) formal written information generation, 2) formal face-to-face information generation, 3) informal information generation, 4) communication and dissemination of information, and 5) responding to this internal market information. The impact of IMO on important organizational factors is also explored. Results indicate positive consequences for customer satisfaction, relative competitive position, staff attitudes, staff retention and staff compliance.

The impact of internal and external market orientations on firm performance.

The role of internal marketing in developing organisational competencies is identified as a key area for continued research (Rafiq and Ahmed, 2003). One competence of particular interest to marketers is market orientation. This paper examines the impact of internal marketing, operationalised as a set of internal market orientated behaviours (IMO) on market orientation (MO), and consequently organisational performance, and provides the first quantitative evidence to support the long held assumption that internal marketing has an impact on marketing success. Data from UK retail managers were analysed using structural equations modelling employing LISREL software. These data indicate significant relationships between internal market orientation, employee motivation and external marketing success (market orientation, financial performance and customer satisfaction). Our results also support previous findings indicating a positive impact of external market orientation on customer satisfaction and financial performance. For marketing practitioners, the role of internal market orientation is developing marketing strategies is discussed.

Implementation of an integrated, technology-based, discovery mode assessment item involving an incubation period to enhance learning outcomes for engineering maths students

In this paper we discuss our current efforts to develop and implement an exploratory, discovery mode assessment item into the total learning and assessment profile for a target group of about 100 second level engineering mathematics students. The assessment item under development is composed of 2 parts, namely, a set of "pre-lab" homework problems (which focus on relevant prior mathematical knowledge, concepts and skills), and complementary computing laboratory exercises which are undertaken within a fixed (1 hour) time frame. In particular, the computing exercises exploit the algebraic manipulation and visualisation capabilities of the symbolic algebra package MAPLE, with the aim of promoting understanding of certain mathematical concepts and skills via visual and intuitive reasoning, rather than a formal or rigorous approach. The assessment task we are developing is aimed at providing students with a significant learning experience, in addition to providing feedback on their individual knowledge and skills. To this end, a noteworthy feature of the scheme is that marks awarded for the laboratory work are primarily based on the extent to which reflective, critical thinking is demonstrated, rather than the amount of CBE-style tasks completed by the student within the allowed time. With regard to student learning outcomes, a novel and potentially critical feature of our scheme is that the assessment task is designed to be intimately linked to the overall course content, in that it aims to introduce important concepts and skills (via individual student exploration) which will be revisited somewhat later in the pedagogically more restrictive formal lecture component of the course (typically a large group plenary format). Furthermore, the time delay involved, or "incubation period", is also a deliberate design feature: it is intended to allow students the opportunity to undergo potentially important internal re-adjustments in their understanding, before being exposed to lectures on related course content which are invariably delivered in a more condensed, formal and mathematically rigorous manner. In our presentation, we will discuss in more detail our motivation and rationale for trailing such a scheme for the targeted student group. Some of the advantages and disadvantages of our approach (as we perceived them at the initial stages) will also be enumerated. In a companion paper, the theoretical framework for our approach will be more fully elaborated, and measures of student learning outcomes (as obtained from eg. student provided feedback) will be discussed.

Project team’s internal and external social networks and their influence on project performance

The research on project learning has recognised the significance of knowledge transfer in project based organisations (PBOs). Effective knowledge transfer across projects avoids reinventions, enhances knowledge creation and saves lots of time that is crucial in project environment. In order to facilitate knowledge transfer, many PBOs have invested lots of financial and human resources to implement IT-based knowledge repository. However, some empirical studies found that employees would rather turn for knowledge to colleagues despite their ready access to IT-based knowledge repository. Therefore, it is apparent that social networks play a pivotal role in the knowledge transfer across projects. Some scholars attempt to explore the effect of network structure on knowledge transfer and performance, however, focused only on egocentric networks and the groups’ internal social networks. It has been found that the project’s external social network is also critical, in that the team members can not handle critical situations and accomplish the projects on time without the assistance and knowledge from external sources. To date, the influence of the structure of a project team’s internal and external social networks on project performance, and the interrelation between both networks are barely known. In order to obtain such knowledge, this paper explores the interrelation between the structure of a project team’s internal and external social networks, and their effect on the project team’s performance. Data is gathered through survey questionnaire distributed online to respondents. Collected data is analysed applying social network analysis (SNA) tools and SPSS. The theoretical contribution of this paper is the knowledge of the interrelation between the structure of a project team’s internal and external social networks and their influence on the project team’s performance. The practical contribution lies in the guideline to be proposed for constructing the structure of project team’s internal and external social networks.

Harnessing assessment and feedback in the first year to support learning success, engagement and retention

It has been argued that intentional first year curriculum design has a critical role to play in enhancing first year student engagement, success and retention (Kift, 2008). A fundamental first year curriculum objective should be to assist students to make the successful transition to assessment in higher education. Scott (2006) has identified that ‘relevant, consistent and integrated assessment … [with] prompt and constructive feedback’ are particularly relevant to student retention generally; while Nicol (2007) suggests that ‘lack of clarity regarding expectations in the first year, low levels of teacher feedback and poor motivation’ are key issues in the first year. At the very minimum, if we expect first year students to become independent and self-managing learners, they need to be supported in their early development and acquisition of tertiary assessment literacies (Orrell, 2005). Critical to this attainment is the necessity to alleviate early anxieties around assessment information, instructions, guidance, and performance. This includes, for example:  inducting students thoroughly into the academic languages and assessment genres they will encounter as the vehicles for evidencing learning success; and  making expectations about the quality of this evidence clear. Most importantly, students should receive regular formative feedback of their work early in their program of study to aid their learning and to provide information to both students and teachers on progress and achievement. Leveraging research conducted under an ALTC Senior Fellowship that has sought to articulate a research-based 'transition pedagogy' (Kift & Nelson, 2005) – a guiding philosophy for intentional first year curriculum design and support that carefully scaffolds and mediates the first year learning experience for contemporary heterogeneous cohorts – this paper will discuss theoretical and practical strategies and examples that should be of assistance in implementing good assessment and feedback practices across a range of disciplines in the first year.

Internal audit quality, audit committee independence, growth opportunities and firm performance

This study explores whether the relation between internal audit quality and firm performance is associated with firm characteristics of information asymmetry and uncertainty (growth opportunities) and certain governance controls (audit committee effectiveness). The results from this preliminary study of 60 Malaysian companies show that the association between internal audit quality and firm performance is stronger for firms with high growth opportunities and that this positive association is weakened by increasing audit committee independence. These findings demonstrate the internal auditors conflicting roles and question the governance recommendations that require all members of the audit committee to be non-executive directors.

Mining Negative Relevance Feedback for Information Filtering

It is a big challenge to clearly identify the boundary between positive and negative streams. Several attempts have used negative feedback to solve this challenge; however, there are two issues for using negative relevance feedback to improve the effectiveness of information filtering. The first one is how to select constructive negative samples in order to reduce the space of negative documents. The second issue is how to decide noisy extracted features that should be updated based on the selected negative samples. This paper proposes a pattern mining based approach to select some offenders from the negative documents, where an offender can be used to reduce the side effects of noisy features. It also classifies extracted features (i.e., terms) into three categories: positive specific terms, general terms, and negative specific terms. In this way, multiple revising strategies can be used to update extracted features. An iterative learning algorithm is also proposed to implement this approach on RCV1, and substantial experiments show that the proposed approach achieves encouraging performance.

An effective model of using negative relevance feedback for information filtering

Over the years, people have often held the hypothesis that negative feedback should be very useful for largely improving the performance of information filtering systems; however, we have not obtained very effective models to support this hypothesis. This paper, proposes an effective model that use negative relevance feedback based on a pattern mining approach to improve extracted features. This study focuses on two main issues of using negative relevance feedback: the selection of constructive negative examples to reduce the space of negative examples; and the revision of existing features based on the selected negative examples. The former selects some offender documents, where offender documents are negative documents that are most likely to be classified in the positive group. The later groups the extracted features into three groups: the positive specific category, general category and negative specific category to easily update the weight. An iterative algorithm is also proposed to implement this approach on RCV1 data collections, and substantial experiments show that the proposed approach achieves encouraging performance.

State-of-the-Art Review on Opinion Mining from Online Customers’ Feedback

Dealing with the ever-growing information overload in the Internet, Recommender Systems are widely used online to suggest potential customers item they may like or find useful. Collaborative Filtering is the most popular techniques for Recommender Systems which collects opinions from customers in the form of ratings on items, services or service providers. In addition to the customer rating about a service provider, there is also a good number of online customer feedback information available over the Internet as customer reviews, comments, newsgroups post, discussion forums or blogs which is collectively called user generated contents. This information can be used to generate the public reputation of the service providers’. To do this, data mining techniques, specially recently emerged opinion mining could be a useful tool. In this paper we present a state of the art review of Opinion Mining from online customer feedback. We critically evaluate the existing work and expose cutting edge area of interest in opinion mining. We also classify the approaches taken by different researchers into several categories and sub-categories. Each of those steps is analyzed with their strength and limitations in this paper.

Measuring factors that influence the utilisation of preventive care services provided by general practitioners in Australia

Background: Relatively little research attention has been given to the development of standardised and psychometrically sound scales for measuring influences relevant to the utilisation of health services. This study aims to describe the development, validation and internal reliability of some existing and new scales to measure factors that are likely to influence utilisation of preventive care services provided by general practitioners in Australia.----- Methods: Relevant domains of influence were first identified from a literature review and formative research. Items were then generated by using and adapting previously developed scales and published findings from these. The new items and scales were pre-tested and qualitative feedback was obtained from a convenience sample of citizens from the community and a panel of experts. Principal Components Analyses (PCA) and internal reliability testing (Cronbach's alpha) were then conducted for all of the newly adapted or developed scales utilising data collected from a self-administered mailed survey sent to a randomly selected population-based sample of 381 individuals (response rate 65.6 per cent).----- Results: The PCA identified five scales with acceptable levels of internal consistency were: (1) social support (ten items), alpha 0.86; (2) perceived interpersonal care (five items), alpha 0.87, (3) concerns about availability of health care and accessibility to health care (eight items), alpha 0.80, (4) value of good health (five items), alpha 0.79, and (5) attitudes towards health care (three items), alpha 0.75.----- Conclusion The five scales are suitable for further development and more widespread use in research aimed at understanding the determinants of preventive health services utilisation among adults in the general population.