A construção de indicadores de qualidade no campo da avaliação educacional : um enfoque epistemológico

Este artigo trata do processo de construção de indicadores de qualidade no campo da avaliação educacional e, mais especificamente, da avaliação de políticas e programas educacionais. O objetivo é promover uma discussão sobre duas questões articuladas: num primeiro momento, tratar os aspectos epistemológicos e teórico-metodológicos envolvidos no processo de construção de indicadores nas ciências sociais; e, num segundo, refletir sobre os limites e possibilidades da construção de indicadores de qualidade no campo da avaliação educacional com base nas abordagens teóricas mais recentes sobre qualidade em educação e sobre avaliação. Para proceder à discussão adotamos uma perspectiva epistemológica que busca focalizar a questão a partir da relação entre a definição de modelos de avaliação educacional e os problemas de validez que permeia o processo de construção do conhecimento, os obstáculos envolvidos na relação que estabelece entre o pesquisador social (no caso específico, o avaliador) e o conhecimento a ser produzido, bem como a questão do seu uso social e político que remete ao sentido atribuído para o termo qualidade em educação. Na conclusão o artigo apresenta uma proposição provisória de um conjunto de estratégias que podem ser considerados como norteadores para a construção de modelos de avaliação de políticas e programas educacionais e de indicadores de qualidade.

Benchmarking or business opportunity: Using milk recording data to expose your clients' weaknesses

Routine milk recording data, often covering many years, are available for approximately half the dairy herds of England and Wales. In addition to milk yield and quality, these data include production events that can be used to derive objective Key Performance Indicators (KPI) describing a herd's fertility and production. Recent developments in information systems give veterinarians and other technical advisers access to these KPIs on-line. In addition to reviewing individual herd performance, advisers can establish local benchmark groups to demonstrate the relative performance of similar herds in the vicinity. The use of existing milk recording data places no additional demands on farmer's time or resources. These developments could also readily be exploited by universities to introduce veterinary undergraduates to the realities of commercial dairy production.

Mid-term evaluation of African Technology Policy Studies Network Phase VI Strategic Plan: for the period January 1, 2009 to December 31, 2010

The African Technology Policy Studies Network (ATPS) is a multidisciplinary network of researchers, private sector actors, policymakers and civil society. ATPS has the vision to become the leading international centre of excellence and reference in science, technology and innovation (STI) systems research, training and capacity building, communication and sensitization, knowledge brokerage, policy advocacy and outreach in Africa. It has a Regional Secretariat in Nairobi Kenya, and operates through national chapters in 29 countries (including 27 in Africa and two Chapters in the United Kingdom and USA for Africans in the Diaspora) with an expansion plan to cover the entire continent by 2015. The ATPS Phase VI Strategic Plan aims to improve the understanding and functioning of STI processes and systems to strengthen the learning capacity, social responses, and governance of STI for addressing Africa's development challenges, with a specific focus on the Millennium Development Goals (MDGs). A team of external evaluators carried out a midterm review to assess the effectiveness and efficiency of the implementation of the Strategic Plan for the period January 1, 2009 to December 31, 2010. The evaluation methodology involved multiple quantitative and qualitative methods to assess the qualitative and quantitative inputs (human resources, financial resources, time, etc.) into ATPS activities (both thematic and facilitative) and their tangible and intangible outputs, outcomes and impacts. Methods included a questionnaire survey of ATPS members and stakeholders, key informant interviews, and focus group discussions (FGDs) with members in six countries. Effectiveness of Programmes Under all six strategic goals, very good progress has been made towards planned outputs and outcomes. This is evidenced by key performance indicators (KPIs) generated from desk review, ratings from the survey respondents, and the themes that run through the FGDs. Institutional and Programme Cost Effectiveness Institutional Effectiveness: assessment of institutional effectiveness suggests that adequate management frameworks are in place and are being used effectively and transparently. Also technical and financial accounting mechanisms are being followed in accordance with grant agreements and with global good practice. This is evidenced by KPIs generated from desk review. Programme Cost Effectiveness: assessment of cost-effectiveness of execution of programmes shows that organisational structure is efficient, delivering high quality, relevant research at relatively low cost by international standards. The evidence includes KPIs from desk review: administrative costs to programme cost ratio has fallen steadily, to around 10%; average size of research grants is modest, without compromising quality. There is high level of pro bono input by ATPS members. ATPS Programmes Strategic Evaluation ATPS research and STI related activities are indeed unique and well aligned with STI issues and needs facing Africa and globally. The multi-disciplinary and trans-boundary nature of the research activities are creating a unique group of research scientists. The ATPS approach to research and STI issues is paving the way for the so called Third Generation University (3GU). Understanding this unique positioning, an increasing number of international multilateral agencies are seeking partnership with ATPS. ATPS is seeing an increasing level of funding commitments by Donor Partners. Recommendations for ATPS Continued Growth and Effectiveness On-going reform of ATPS administrative structure to continue The on-going reforms that have taken place within the Board, Regional Secretariat, and at the National Chapter coordination levels are welcomed. Such reform should continue until fully functional corporate governance policy and practices are fully established and implemented across the ATPS governance structures. This will further strengthen ATPS to achieve the vision of being the leading STI policy brokerage organization in Africa. Although training in corporate governance has been carried out for all sectors of ATPS leadership structure in recent time, there is some evidence that these systems have not yet been fully implemented effectively within all the governance structures of the organization, especially at the Board and National chapter levels. Future training should emphasize practical application with exercises relevant to ATPS leadership structure from the Board to the National Chapter levels. Training on Transformational Leadership - Leading a Change Though a subject of intense debate amongst economists and social scientists, it is generally agreed that cultural mindsets and attitudes could enhance and/or hinder organizational progress. ATPS’s vision demands transformational leadership skills amongst its leaders from the Board members to the National Chapter Coordinators. To lead such a change, ATPS leaders must understand and avoid personal and cultural mindsets and value systems that hinder change, while embracing those that enhance it. It requires deliberate assessment of cultural, behavioural patterns that could hinder progress and the willingness to be recast into cultural and personal habits that make for progress. Improvement of relationship amongst the Board, Secretariat, and National Chapters A large number of ATPS members and stakeholders feel they do not have effective communications and/or access to Board, National Chapter Coordinators and Regional Secretariat activities. Effort should be made to improve the implementation of ATPS communication strategy to improve on information flows amongst the ATPS management and the members. The results of the survey and the FGDs suggest that progress has been made during the past two years in this direction, but more could be done to ensure effective flow of pertinent information to members following ATPS communications channels. Strategies for Increased Funding for National Chapters There is a big gap between the fundraising skills of the Regional Secretariat and those of the National Coordinators. In some cases, funds successfully raised by the Secretariat and disbursed to national chapters were not followed up with timely progress and financial reports by some national chapters. Adequate training in relevant skills required for effective interactions with STI key policy players should be conducted regularly for National Chapter coordinators and ATPS members. The ongoing training in grant writing should continue and be made continent-wide if funding permits. Funding of National Chapters should be strategic such that capacity in a specific area of research is built which, with time, will not only lead to a strong research capacity in that area, but also strengthen academic programmes. For example, a strong climate change programme is emerging at University of Nigeria Nsukka (UNN), with strong collaborations with Universities from neighbouring States. Strategies to Increase National Government buy-in and support for STI Translating STI research outcomes into policies requires a great deal of emotional intelligence, skills which are often lacking in the first and second generation universities. In the epoch of the science-based or 2GUs, governments were content with universities carrying out scientific research and providing scientific education. Now they desire to see universities as incubators of new science- or technology-based commercial activities, whether by existing firms or start-ups. Hence, governments demand that universities take an active and leading role in the exploitation of their knowledge and they are willing to make funds available to support such activities. Thus, for universities to gain the attention of national leadership they must become centres of excellence and explicit instruments of economic development in the knowledge-based economy. The universities must do this while working collaboratively with government departments, parastatals, and institutions and dedicated research establishments. ATPS should anticipate these shifting changes and devise programmes to assist both government and universities to relate effectively. New administrative structures in member organizations to sustain and manage the emerging STI multidisciplinary teams Second Generation universities (2GUs) tend to focus on pure science and often do not regard the application of their know-how as their task. In contrast, Third Generation Universities (3GUs) objectively stimulate techno-starters – students or academics – to pursue the exploitation or commercialisation of the knowledge they generate. They view this as being equal in importance to the objectives of scientific research and education. Administratively, research in the 2GU era was mainly monodisciplinary and departments were structured along disciplines. The emerging interdisciplinary scientific teams with focus on specific research areas functionally work against the current mono-disciplinary faculty-based, administrative structure of 2GUs. For interdisciplinary teams, the current faculty system is an obstacle. There is a need for new organisational forms for university management that can create responsibilities for the task of know-how exploitation. ATPS must anticipate this and begin to strategize solutions for their member institutions to transition to 3Gus administrative structure, otherwise ATPS growth will plateau, and progress achieved so far may be stunted.

KPIs and their properties defined with the EXTREME method

Key Performance Indicators (KPIs) are the main instruments of Business Performance Management. KPIs are the measures that are translated to both the strategy and the business process. These measures are often designed for an industry sector with the assumptions about business processes in organizations. However, the assumptions can be too incomplete to guarantee the required properties of KPIs. This raises the need to validate the properties of KPIs prior to their application to performance measurement. This paper applies the method called EXecutable Requirements Engineering Management and Evolution (EXTREME) for validation of the KPI definitions. EXTREME semantically relates the goal modeling, conceptual modeling and protocol modeling techniques into one methodology. The synchronous composition built into protocol modeling enables raceability of goals in protocol models and constructive definitions of a KPI. The application of the method clarifies the meaning of KPI properties and procedures of their assessment and validation.

Evaluation of a thermally driven heat pump for solar heating and cooling applications

Exploiting solar energy technology for both heating and cooling purposes has the potential of meeting an appreciable portion of the energy demand in buildings throughout the year. By developing an integrated, multi-purpose solar energy system, that can operate all twelve months of the year, a high utilisation factor can be achieved which translates to more economical systems. However, there are still some techno-economic barriers to the general commercialisation and market penetration of such technologies. These are associated with high system and installation costs, significant system complexity, and lack of knowledge of system implementation and expected performance. A sorption heat pump module that can be integrated directly into a solar thermal collector has thus been developed in order to tackle the aforementioned market barriers. This has been designed for the development of cost-effective pre-engineered solar energy system kits that can provide both heating and cooling. This thesis summarises the characterisation studies of the operation of individual sorption modules, sorption module integrated solar collectors and a full solar heating and cooling system employing sorption module integrated collectors. Key performance indicators for the individual sorption modules showed cooling delivery for 6 hours at an average power of 40 W and a temperature lift of 21°C. Upon integration of the sorption modules into a solar collector, measured solar radiation energy to cooling energy conversion efficiencies (solar cooling COP) were between 0.10 and 0.25 with average cooling powers between 90 and 200 W/m2 collector aperture area. Further investigations of the sorption module integrated collectors implementation in a full solar heating and cooling system yielded electrical cooling COP ranging from 1.7 to 12.6 with an average of 10.6 for the test period. Additionally, simulations were performed to determine system energy and cost saving potential for various system sizes over a full year of operation for a 140 m2 single-family dwelling located in Madrid, Spain. Simulations yielded an annual solar fraction of 42% and potential cost savings of €386 per annum for a solar heating and cooling installation employing 20m2 of sorption integrated collectors.

Financial reporting for environmental and social responsibility: a normative strategic concept

Corporate responsibility demands that firms address environmental and social values in their firms' policy and key performance indicators. These are integrated through strategic planning and require firms to merge the long-term environmental and social values with short-term economic objectives and performance measures. Each firm's strategy will differ. This paper provides a normative reporting concept to connect the financial implications associated with long-term planning for environmental and social values, with short-term accounting reports. Reporting variants adapted from total cost assessment, life cycle costing and variable costing are integrated to offer upstream information based on a product segment view.

Environmental and social responsibility: a normative financial reporting concept

The inclusion of environmental and social values in a firm’s policy and key performance indicators can enhance its reputation and create wealth for both the firm and its investors. Social values and associated activities are initially integrated with financial resources through the strategic plan, which requires firms to merge the longer term environmental and social values with short term economic objectives and performance measures. Strategies will differ between individual corporations. This paper provides a normative reporting concept which connects the financial implications associated with longer term planning for environmental and social values, with short term accounting reports. Reporting variants adapted from total cost assessment, life cycle costing, variable costing are integrated to offer opportunities to present both past and predicted information based on a product segment view.

Reflexive capability maturity assessment for firm management of multipartner international projects

This paper reports the findings of a research project which was aimed at developing and promoting a greater understanding of the uniqueness of successful Australian-Malaysian partnerships and joint ventures undertaken on complex multi-partner mega construction projects. The aim of the investigation was to identify the critical success factors specific to such partnerships. The theoretical framework was grounded in a previous empirical study on design firm internationalisation which demonstrated that long-term success in international markets is enhanced by the management of integration of transformation of social, cultural and intellectual capital within the firm. A generic adaptive performance framework mapping the key performance indicators for each of these dimensions was developed which can be modified to unique contexts related to specific geographical localities. In the Australian Malaysian Institute study the framework is tested within the context of firm and shared partnership capital. The broad interpretive framework of adaptive performance was refined as a result of the findings of the four case studies of Malaysian organisations to develop a partnerships capacity performance measurement framework for Malaysian firms working on international projects. Six in-depth interviews were conducted with ten participants across the four Malaysian firms and a thematic analysis was undertaken of the transcripts. A reflexive capability maturity assessment tool was developed from insights gained into the underlying causes of key barriers coupled with common themes emerging in relation to the successful approaches used by case study firms to overcome those barriers. Specific examples of the case study firms' demonstration of reflexive capability in the areas of market knowledge, design management and market entry are presented.

Age and productivity of academics: A case study of a public university in Malaysia

This study examines the relationship between age and productivity measured based on key performance indicators (KPI) amongst academic staff at Universiti Sains Malaysia (USM). Three models were used in the analysis: linear, quadratic and piece-wise spline. The linear model indicates that age is negatively related to KPI. The quadratic model shows an inverted-U shaped relationship where KPI peaks at age 41 years. The piece-wise spline model indicates academic staff reach the peak of their productivity between ages 46-50 years with another productive age interval between 36-40 years implying 10 golden years when KPI could be harvested fruitfully. There is a significant downtrend in the KPI after 50 years of age. Other factors that have significant influence on KPI are gender, academic rank and discipline. The sub-models show that the influence of age on KPI is more significant amongst academic staff in the arts compared to the science stream. Age influence on KPI is significant amongst female staff but not male staff. We conclude that assessing performance in the workplace with regard to age requires complex methodological engagement and also needs to be based on a wider lens which recognises and includes within the discussion, the intangible and social dimensions of performance.

Intensive case management for high-risk patients with first-episode psychosis: service model and outcomes

Background : The first episode of psychosis is a crucial period when early intervention can alter the trajectory of the young person's ongoing mental health and general functioning. After an investigation into completed suicides in the Early Psychosis Prevention and Intervention Centre (EPPIC) programme, the intensive case management subprogramme was developed in 2003 to provide assertive outreach to young people having a first episode of psychosis who are at high risk owing to risk to self or others, disengagement, or suboptimal recovery. We report intensive case management model development, characterise the target cohort, and report on outcomes compared with EPPIC treatment as usual.

Methods : Inclusion criteria, staff support, referral pathways, clinical review processes, models of engagement and care, and risk management protocols are described. We compared 120 consecutive referrals with 50 EPPIC treatment as usual patients (age 15–24 years) in a naturalistic stratified quasi-experimental real-world design. Key performance indicators of service use plus engagement and suicide attempts were compared between EPPIC treatment as usual and intensive case management, and psychosocial and clinical measures were compared between intensive case management referral and discharge.

Findings : Referrals were predominately unemployed males with low levels of functioning and educational attainment. They were characterised by a family history of mental illness, migration and early separation, with substantial trauma, history of violence, and forensic attention. Intensive case management improved psychopathology and psychosocial outcomes in high-risk patients and reduced risk ratings, admissions, bed days, and crisis contacts.

Interpretation : Characterisation of intensive case management patients validated the clinical research focus and identified a first episode of psychosis high-risk subgroup. In a real-world study, implementation of an intensive case management stream within a well-established first episode of psychosis service showed significant improvement in key service outcomes. Further analysis is needed to determine cost savings and effects on psychosocial outcomes. Targeting intensive case management services to high-risk patients with unmet needs should reduce the distress associated with pathways to care for patients, their families, and the community.

Uma investigação empírico-exploratória acerca da causalidade dos indicadores operacionais de call center com a satisfação dos clientes

Year after year the call center industry comes presenting high growth, either in Brazil or the world, using increasing contingent of people and receiving large amount of financial resources. Supporting all this industry, exists a theoretical reference that privileges the search of the improvement of diverse operational performance indicators. This study, empirically examines the relationship between caller satisfaction and diverse operational performance indicators currently used for call center management in Brazil. For in such a way, a group of telecommunications segment call centers will be used searching to determine which indicators are really more significantly correlated with the customer satisfaction. Of this form, it is aimed at to add new evidences and explanations to existing literature, in way that companies of call center industry can give a more adequate service to their customer, opposite to solely focus its efforts in the improvement of a great set of performance pointers that, effectively, can not become related with the customer satisfaction.

Responsabilidade social corporativa e gaps de percepção: um estudo de caso

A disseminação do termo Responsabilidade Social Corporativa e de suas respectivas ações demonstra o grande potencial que as empresas possuem para atenuar ou agravar problemas sociais e ambientais. Hoje, existem diversas métricas que permitem medir a efetividade das ações de Responsabilidade Social praticadas pelas organizações e assim divulgar os feitos em RSC tanto em relatórios para investidores e quanto em outros meios. Esta pesquisa propõe analisar o desempenho em Responsabilidade Social de uma empresa do segmento de logística, pela percepção dos seus funcionários, confrontando-a com o discurso da empresa procurando identificar gaps de percepção. À luz de teorias da Escola de Frankfurt e de estudos em Responsabilidade Social, são analisadas possíveis causas para a existência de gaps entre a percepção dos funcionários e o discurso da empresa. Para delimitar o escopo da pesquisa, dada a amplitude do tema, foram utilizados os Indicadores Ethos de Responsabilidade Social Empresarial para identificar o desempenho em RSC da empresa analisada.

Reflexões sobre a utilização do balanced scorecard na gestão da cadeia de suprimentos: um estudo de caso na unidade operacional de uma grande distribuidora de combustíveis

As organizações contemporâneas apresentam elevado grau de complexidade. A extensa cadeia de processos deve ser gerenciada de forma integrada e requer descentralização da tomada de decisões para assegurar respostas ágeis aos estímulos do ambiente competitivo. Este contexto demanda a criação de ferramentas de gestão que apoiem os gestores no atingimento dos objetivos estratégicos globais das empresas, como o modelo de referência analisado neste trabalho, que alia as métricas de gestão da cadeia de suprimentos com o conceito do Balanced Scorecard. Este trabalho objetivou, por um lado, avaliar a atual arquitetura de indicadores de desempenho da unidade operacional (UO) de uma grande distribuidora de combustíveis à luz deste modelo de referência e, por outro, testar o modelo de referência através do estudo de caso quanto à sua universalidade de aplicação por diferentes empresas. Dentre os resultados obtidos, destacam-se: a arquitetura de indicadores atual da UO está desequilibrada nas dimensões do BSC – comparada à dimensão financeira que apresenta 7 indicadores, a dimensão clientes apresenta somente 2 indicadores; identificou-se uma lacuna de 50% na relação de indicadores de desempenho da UO quando a confrontamos com o modelo de referência; para as lacunas identificadas, desenvolveu-se um plano de ação para sua incorporação à arquitetura da UO; constatou-se que 24% das métricas do modelo de referência não se aplicam à realidade da UO e; o modelo de referência não abarca itens relevantes como treinamento, engajamento e Saúde, Meio Ambiente e Segurança (SMS), primordial para a realidade da empresa, que acompanha atualmente 6 indicadores de SMS. Verificou-se, também, que o uso de um modelo de referência genérico não é o mais adequado por causa das complexidades intrínsecas de cada uma das organizações. Entretanto, o mesmo pode ser usado como um guia para verificação de suficiência dos indicadores pré-selecionados por uma determinada empresa porque indica lacunas que podem não ser percebidas no processo de seleção.

Medição de desempenho na cadeia de suprimentos da energia eólica: proposta de um conjunto de indicadores de desempenho

Performance measurement in highly competitive markets is a necessary measure for those who aim the top positions. The business performance measurement approach have reached relevant results in the literature, however, a different approach has recently appeared that broadened the perception of competition, where companies do not seek the competition among companies only, but also among supply chains. Brazilian Wind energy supply chain is living a structuring and expanding moment, with the major global players in the industry making investments in the country. This research aims to answer which are the key performance indicators that must be considered by the Brazilian wind energy sector companies, which are part in a broad perspective of supply chain competition. The research was executed in two steps: exploratory (literature review and a field research in the companies) and later a survey was conducted with the Brazilian Wind energy companies workers with the purpose to validate the performance indicators found in the exploratory step. The survey evaluated 40 performance indicators distributed among five major activities: Project prospection, building/execution, operation and maintenance, logistics and transverse processes, which summarize the performance of the entire supply chain, pointing the sinergy and the competitive level of the supply chain. The selected performance indicators reflect the high relevance of the costs dimension in the Brazilian energy companies performance, acting as a key performance indicator, is also indicates the limited performance management integration throughout the Brazilian wind energy supply chain