External Evaluation for the BCE-CT Project

Strand one: Effectiveness of project activities and processes

External school evaluation : teachers’ perspectives : two case studies from Northern Portugal

The external evaluation of non-higher education schools in Portugal has been developed by the General Inspectorate of Education since 2006. A first cycle of evaluation was completed, covering all educational units in continental Portugal up to 2011. The model of evaluation has since been subject to alterations, and a second cycle of evaluation is now coming to an end. The current model of evaluation is based on documental analysis, analysis of students’ results, and panel interviews with a variety of representatives of the school community, and addresses three domains: results, provision of educational service and management. This paper is part of an ongoing research project, developed by 6 universities and supported by the Portuguese Foundation for Science and Technology (PTDC/CPE-CED/116674/2010) which intends to analyse the impacts and effects this process of external evaluation has had on Portuguese schools. This project includes a variety of perspectives and methodologies. In particular, we will focus on two case studies undertaken in two schools from the northern region of Portugal, and more specifically on the perspectives expressed by the teachers of those schools. These particular schools were chosen because they have been evaluated twice and represent different educational levels (basic and secondary), contexts and results. These case studies included the analysis of documental data, interviews to key informants and a questionnaire directed to teachers (n = 141) – the latter will be the main focus of this paper. Teachers are essential elements of the school community when considering the impacts of external evaluation, as any changes directed at teaching practices, student evaluation, among others are only possible through their direct action and implication. Therefore, their perceptions on the process and its impacts are crucial to the understanding of what does and does not change in schools as a consequence of external evaluation. Although teachers’ opinions are not homogenous and each school reveals a number of differences when it comes to teachers’ perceptions of School Evaluation, it was possible to stress some areas as the most and as the least consensual. Teachers in both schools agree External School Evaluation (ESE) is useful for the identification of the schools’ strengths and weaknesses, values students’ external evaluation results, imposes a model for schools internal evaluation (and in fact contributes to the very existence of internal evaluation practices), and contributes to schools improvement. However teachers in both schools do not believe ESE contributes to teachers’ autonomy produces changes in how curriculum is managed, or leads to innovative teaching practices. These results point to a greater emphasis on change at the levels of school management, self-evaluation and particularly internal evaluation, but little impact on the teaching practices. We believe the classroom is at the core of school practices and teaching processes are essential to any measure of school quality and to their impacts on student learning.

Women’s health Goulburn North East community crime prevention plan – regional preventing violence against women and children strategy: summative evaluation report from the external evaluator, February 2016

This report presents the external evaluation of the Hume Regional Preventing Violence Against Women Strategy. This is one of 12 projects funded by the Department of Justice and Regulation in Victoria under its initiative to support primary prevention and early intervention-focused partnership projects that seek to prevent violence before it occurs or address the key contributing factors of violence against women and their children. The focus is on changing behaviours and attitudes that allow violence against women and children to continue. The lead agency in the project was Women’s Health Goulburn North East (WHGNE).The Hume regional strategy The focus of the project was to develop and implement a coordinated regional strategy that addresses the determinants of violence against women and builds the capacity of communities to take action to prevent such violence. At the outset it was envisaged that a key component of the strategy would be to undertake preventive activities that respond to identified gaps in the region.The first version of the strategy was completed in September 2013 and formally launched in November 2013. In October 2013, Courageous Conversations was identified as the brand to be used for activities in the strategy, including a charter and other resources. As the project evolved, the strategy was revised to reflect differing levels of engagement and progress with the different parts of the work and to identity explicitly the activities associated with the brand. A revised version of the regional strategy was produced in September 2014, with four aims:• promoting equal and respectful relationships between men and women;• working across local government, workplaces and sporting settings to coordinate a region-wide approach to preventing violence against women;• bringing about structural and systemic organisational change to promote gender equitable and non-violent cultures;• build the capacity of leaders in preventing violence against women.Different components of the activities carried out under the strategy included: partnership and capacity building; building gender equity in organisations; gender equity and masculinities training; bystander training; knowledge dissemination and the Courageous Conversations website.

Wishful Thinking: Lessons from the internal and external evaluations of an innovatory education project in Northern Ireland

From March 1999 to August 2000, the authors were involved in simultaneous internal and external evaluations of the social civic and political education (SCaPE) project in Northern Ireland. This project was a major initiative established by the Citizenship Foundation, the Northern Ireland Council for the Curriculum, Examination and Assessment (CCEA), and the School of Education at the University of Ulster at Coleraine. It was a 2-year project in 25 secondary schools established to design, develop, pilot and evaluate a new programme of social, civic and political education for Northern Ireland. It also aimed to serve as a model for future Citizenship curriculum developments throughout Northern Ireland and elsewhere. This paper describes the background to the project, the design and conduct of the two evaluations, and the links between them. It outlines the main conclusions of each evaluation and describes the way SCaPE has since evolved into a mainstream curriculum development project. The final part of the paper analyses the key opportunities, tensions and challenges involved in running such evaluations at a critical time in the history of Northern Ireland – a time when innovation is both necessary and controversial. It argues that, especially in such circumstances, evaluation cannot be conducted from a neutral, objective standpoint, and that it is incumbent on evaluators to recognise the emotional, personal and political commitment they make to the projects in which they are engaged.

Evaluation of the environments for health framework

Environments for Health, Five Years On…
Since its release in 2001, Victoria’s Environments for Health Municipal Public Heath Planning Framework, has represented a leading edge approach to supporting quality public health planning at the local government level.

In 2006 the Department of Human Services appointed an external evaluation team from Deakin and Melbourne Universities to assess the introduction of the Environments for Health framework to municipal public health planning and to make recommendations for its future direction.

The evaluation aimed to determine the extent to which Environments for Health and related implementation activities had:

* been incorporated by local governments in their policies and practices
* created greater opportunities for health gain, and
* been supported effectively by the Department of Human Services and other stakeholders

The evaluation project involved participation from a broad range of key stakeholders, including all local Councils within Victoria. Data collection covered four distinct stages - document analysis, key informant interviews, state-wide survey of practitioners, and council and stakeholder forums, all with opportunities for input to the evaluation.

Grouping people in social networks using a weighted multi-constraints clustering method

Grouping users in social networks is an important process that improves matching and recommendation activities in social networks. The data mining methods of clustering can be used in grouping the users in social networks. However, the existing general purpose clustering algorithms perform poorly on the social network data due to the special nature of users' data in social networks. One main reason is the constraints that need to be considered in grouping users in social networks. Another reason is the need of capturing large amount of information about users which imposes computational complexity to an algorithm. In this paper, we propose a scalable and effective constraint-based clustering algorithm based on a global similarity measure that takes into consideration the users' constraints and their importance in social networks. Each constraint's importance is calculated based on the occurrence of this constraint in the dataset. Performance of the algorithm is demonstrated on a dataset obtained from an online dating website using internal and external evaluation measures. Results show that the proposed algorithm is able to increases the accuracy of matching users in social networks by 10% in comparison to other algorithms.

The role of transformative thinking and practice in a property entrepreneur's response to the funding crisis since 2008

This portfolio of exploration explores the role of transformative thinking and practice in a property entrepreneur’s response to the financial crisis which swept over Ireland from 2008. The complexity of this challenge and the mental capacity to meet its demands is at the core of the exploration. This inquiry emerged from the challenges the financial crisis presented to my values, beliefs, assumptions, and theories, i.e. the interpretive lens through which I make meaning of my experiences. Given the issue identified, this inquiry is grounded in aspects of theories of constructive developmental psychology, applied developmental science, and philosophy. Integrating and linking these elements to business practice is the applied element of the Portfolio. As the 2008 crisis unfolded I realised I was at the limits of my way of knowing. I came to understand that the underlying structure of a way of knowing is the ‘subject-object relationship’ i.e. what a way of knowing can reflect upon, look at, have perspective on, in other words, make object, as against what is it embedded in, attached to, identified within, or subject to. My goal became enhancing my awareness of how I made meaning and how new insights, which would transform a way of knowing, are created. The focus was on enhancing my practice. This Portfolio is structured into three essays. Essay One reported on my self-reflection and external evaluation out of which emerged my developmental goals. In Essay Two I undertake a reading for change programme in which different meaning making systems were confronted in order to challenge me as a meaning maker. Essay Three reported on my experiment which concerned the question whether it was possible for me as a property entrepreneur, and for others alike, to retain bank finance in the face of the overwhelming objective of the bank to deleverage their balance sheet of property loans. The output of my research can be grouped into General Developmental and Specific Business Implications. Firstly I address those who are interested in a transformational-based response to the challenges of operating in the property sector in Ireland during a crisis. I outline the apparatus of thought that I used to create insight, and thus transform how I thought, these are Awareness, Subject-object separation, Exploring other’s perspectives from the position of incompleteness, Dialectical thinking and Collingwood’s Questioning activity. Secondly I set out my learnings from the crisis and their impact on entrepreneurial behaviour and the business of property development. I identify ten key insights that have emerged from leading a property company through the crisis. Many of these are grounded in common sense, however, in my experience these were, to borrow Shakespeare’s words, “More honor'd in the breach than the observance” in pre Crisis Ireland. Finally I set out a four-step approach for forging a strategy. This requires my peer practitioners to identify (i) what they are subject to, (ii) Assess the Opportunity or challenge in a Systemic Context, (iii) Explore Multiple Perspectives on the opportunity or Challenge with an Orientation to change how you know and (iv) Using the Questioning Activity to create Knowledge. Based on my experience I conclude that transformative thinking and practice is a key enabler for a property entrepreneur, in responding to a major collapse of traditional (bank debt) funding.

QA and Enhancement Marketplace for HEIs – An Erasmus+ project

Improving European education and training system quality has been set as a key target in Europe’s strategy to become a smart, sustainable and inclusive economy by 2020 (European Commission, 2010). These objectives are more specifically defined in the so called Modernisation Agenda (European Commission, 2011). More specifically it sets a goal to improve the quality and relevance of higher education. In this process external evaluation and
Proceedings of the 11th International CDIO Conference, Chengdu University of Information Technology,
Chengdu, Sichuan, P.R. China, June 8-11, 2015.
self-assessment are seen in a key role! In the CDIO approach the 12 CDIO standards provide a framework for continuous improvement. Each institution/institutional department are encouraged to regularly do the self-evaluation using the CDIO Standards. Eight European universities identified a need for further enhancement of the self-evaluations and creation of processes with peers to reduce the inertia of heavy accreditations/evaluations in HEIs. In September 2014 these universities started an Erasmus+ project (QAEMarketPlace4HEI) aiming at
1. Developing a collaborative, comprehensive and accessible evaluation process model, methods and tools for HEIs to complement the accreditation systems.
2. Promoting, increasing and exploiting further the European collaboration in the evaluation processes and the exchange of best practices.
3. Disseminating the model, best practices and widen the cooperation to new HEIs in Europe through the partner networks.

Avaliação da Escola - Resultados Escolares e a procura da Qualidade

Trabalho de Projeto apresentado para a obtenção do grau de Mestre na especialidade em Administração Educacional. Orientador:Prof. Doutor João Sebastião

Papel da assessoria na auto-avaliação das escolas

Esta tese apresenta uma investigação que procura problematizar o papel da assessoria na auto-avaliação das escolas. Reflecte-se sobre o contributo de assessores externos nas unidades de gestão para encontrar fórmulas que permitam a construção e sustentabilidade da sua auto-avaliação através da concepção, monitorização e avaliação de planos de acção. A investigação consubstanciou-se na análise dos relatórios das 100 unidades orgânicas que participaram na avaliação externa em 2006/2007 promovida pela Inspecção-Geral de Educação (IGE) especificamente no domínio da Capacidade de Auto-Regulação e Melhoria da Escola (CARME), no Questionário a 90 participantes em equipas de auto-avaliação e ainda em todo o trabalho de campo realizado numa escola secundária através de observação participante. Durante a investigação percebeu-se a adaptação de alguns modelos de auto-avaliação de carácter gerencialista à realidade das unidades orgânicas, identificam-se alguns obstáculos que se apresentam à aplicação dos referidos modelos, as dificuldades que as referidas unidades de gestão sentem quando autonomamente se querem auto-avaliar e identificam-se mecanismos utilizados para a concretização da auto-avaliação e sua manutenção ao longo do tempo. Por fim, constata-se que a assessoria externa é um pilar que pode contribuir para a introdução efectiva dos referidos mecanismos de auto-avaliação e sua sustentabilidade.

Impactos do processo RVCC de nível básico : aprender compensou?

Trabalho de projecto de mestrado, Ciências da Educação (Formação de Adultos), Universidade de Lisboa, Instituto de Educação, 2011

Perceções da equipa de avaliação extrerna relativamente às suas práticas de avaliação

Relatório de estágio de mestrado, Ciências da Educação (Administração Educacional), Universidade de Lisboa, Instituto de Educação, 2013

Do currículo de física e química instituído ao aplicado nas escolas : das metodologias de ensino à avaliação

Tese de doutoramento, Educação (Avaliação em Educação), Universidade de Lisboa, Instituto de Educação, 2015

As representações das equipas de avaliação externa da IGEC na dimensão relativa à autoavaliação das escolas

Relatório de estágio de mestrado, Ciências da Educação (Especialização em Administração Educacional), Universidade de Lisboa, Instituto de Educação, 2014

Gestão Escolar e dinâmicas de avaliação externa num agrupamento de escolas

Relatório de estágio de mestrado, Ciências da Educação (Área de especialização em Administração Educacional), Universidade de Lisboa, Instituto de Educação, 2014