Rakentaako Chen evaluaatiotutkimukselle sen tuolista puuttuvat kaksi jalkaa - vai vahvistaako hän jo olemassa olevaa? Entä selkänoja? Tarvitseeko evaluaatiotutkimus laajemmasta näkökulmasta oman irtotuolin sijasta penkin vai torin, jolla ei ole edes penkkejä?

Abstract:Does evaluation need a three-legged stool? Should there be even more legs in the stool for evaluation or should evaluation have a bench? Or would it be best or more practical for evaluation if it was a square or marketplace without any kind of stools or bench? What is Huey-Tsyh Chen doing for evaluation in his new "Practical evaluation"?

Seurantatiedon hyödyntäminen ohjelma-arvioinnissa : esimerkkinä Euroopan sosiaalirahaston ohjelma-arvioinnit ja seurantajärjestelmä

Abstract: Application of monitoring information in program evaluation: experiences from European Social Fund monitoring and evaluations

Turun kaupungin ympäristönsuojelutoimiston projektit 1995–2000. Yhteiskunnallisen vaikuttavuuden arviointi.

Tässä pro gradu -tutkielmassa arvioin Turun kaupungin ympäristönsuojelutoimiston vuosina 1995–2000 toteuttamien projektien yhteiskunnallista vaikuttavuutta. Projektit ovat: muovijätteen kierrättämiseen keskittynyt KAPULA, ilmanlaadun seurannan kehittämiseen luotu City Simulator, eri sektorien väliseen ympäristöyhteistyöhön kannustanut Green Know-How Turku, Hirvensalon kaavoituksen tulevaisuutta valottanut ICTULA sekä kansainvälinen sähköautoprojekti EVD-POST. Kaikki viisi projektia on toteutettu yhteistyönä, ja osallistujina niissä on ollut ympäristönsuojelutoimiston lisäksi niin Turun alueen kuntia, yrityksiä kuin eurooppalaisia kaupunkejakin. Tutkielmassa arvioin projektien yhteiskunnallista vaikuttavuutta julkishallinnollisesta näkökulmasta. Tässä tapauksessa yhteiskunnallinen vaikuttavuus merkitsee muun muassa sitä, miten ympäristönsuojelutoimisto on vastannut sille asetettuihin tavoitteisiin ja miten toiminta on vastannut siihen kohdistuvia yhteiskunnallisia tarpeita. Tutkimukseni perusteella tarpeet ovat muodostuneet 1990-luvun Suomen talouslaman, 1980- ja 1990-lukujen globaalin ja EU:n ympäristöpolitiikan sekä 1990-luvun kaupunkistrategioiden ja ympäristönsuojeluohjelman kautta. Projektien arviointiin hyödynnän tavoitelähtöistä metodia Evert Vedungin Public Policy and Program Evaluation sekä Huey-Tsyh Chenin Theory-Driven Evaluations -teoksia. Metodilla tarkennan tavoitteiden, toimenpiteiden ja lopputulemien suhdetta. Sen lisäksi käytän Ray Pawsonin ja Nick Tilleyn Realistic Evaluation -kirjan realistista arviointitapaa, jolla tutkin kontekstin ja tehtyjen toimenpiteiden vaikutusta projekteissa. Projektien arvioinnin jälkeen kokoan yhteen päätelmiä projektien yhteiskunnallisesta vaikuttavuudesta ja pohdin syitä siihen, miksi projektit eroavat vaikuttavuudeltaan. Tutkielman johtopäätöksinä on, että projektit eroavat yhteiskunnalliselta vaikuttavuudeltaan lähtökohtien ja hyväksyttävyyden johdosta esimerkiksi lainsäädännön ja ympäristöpolitiikan vuoksi. Kyse on osaksi myös siitä, että joissain projekteissa tavoitteet on asetettu epärealistisiksi suhteessa niiden toteutumiseen. Projekteihin kohdistuvat ulkoiset tarpeet, toimintaympäristö ja realistiset tavoitteet eivät kuitenkaan ehdottomasti takaa yhteiskunnallista vaikuttavuutta. Myös yhteistyöprojektien toteuttajien ominaisuudet eli sisäiset tekijät vaikuttavat yhteiskunnallisen vaikuttavuuden ilmenemiseen.

TURUN KAUPUNGIN YMPÄRISTÖNSUOJELUTOIMISTON PROJEKTIT 1995–2000 Yhteiskunnallisen vaikuttavuuden arviointi

Tässä pro gradu -tutkielmassa arvioin Turun kaupungin ympäristönsuojelutoimiston vuosina 1995–2000 toteuttamien projektien yhteiskunnallista vaikuttavuutta. Projektit ovat: muovijätteen kierrättämiseen keskittynyt KAPULA, ilmanlaadun seurannan kehittämiseen luotu City Simulator, eri sektorien väliseen ympäristöyhteistyöhön kannustanut Green Know-How Turku, Hirvensalon kaavoituksen tulevaisuutta valottanut ICTULA sekä kansainvälinen sähköautoprojekti EVD-POST. Kaikki viisi projektia on toteutettu yhteistyönä, ja osallistujina niissä on ollut ympäristönsuojelutoimiston lisäksi niin Turun alueen kuntia, yrityksiä kuin eurooppalaisia kaupunkejakin. Tutkielmassa arvioin projektien yhteiskunnallista vaikuttavuutta julkishallinnollisesta näkökulmasta. Tässä tapauksessa yhteiskunnallinen vaikuttavuus merkitsee muun muassa sitä, miten ympäristönsuojelutoimisto on vastannut sille asetettuihin tavoitteisiin ja miten toiminta on vastannut siihen kohdistuvia yhteiskunnallisia tarpeita. Tutkimukseni perusteella tarpeet ovat muodostuneet 1990-luvun Suomen talouslaman, 1980- ja 1990-lukujen globaalin ja EU:n ympäristöpolitiikan sekä 1990-luvun kaupunkistrategioiden ja ympäristönsuojeluohjelman kautta. Projektien arviointiin hyödynnän tavoitelähtöistä metodia Evert Vedungin Public Policy and Program Evaluation sekä Huey-Tsyh Chenin Theory-Driven Evaluations -teoksia. Metodilla tarkennan tavoitteiden, toimenpiteiden ja lopputulemien suhdetta. Sen lisäksi käytän Ray Pawsonin ja Nick Tilleyn Realistic Evaluation -kirjan realistista arviointitapaa, jolla tutkin kontekstin ja tehtyjen toimenpiteiden vaikutusta projekteissa. Projektien arvioinnin jälkeen kokoan yhteen päätelmiä projektien yhteiskunnallisesta vaikuttavuudesta ja pohdin syitä siihen, miksi projektit eroavat vaikuttavuudeltaan. Tutkielman johtopäätöksinä on, että projektit eroavat yhteiskunnalliselta vaikuttavuudeltaan lähtökohtien ja hyväksyttävyyden johdosta esimerkiksi lainsäädännön ja ympäristöpolitiikan vuoksi. Kyse on osaksi myös siitä, että joissain projekteissa tavoitteet on asetettu epärealistisiksi suhteessa niiden toteutumiseen. Projekteihin kohdistuvat ulkoiset tarpeet, toimintaympäristö ja realistiset tavoitteet eivät kuitenkaan ehdottomasti takaa yhteiskunnallista vaikuttavuutta. Myös yhteistyöprojektien toteuttajien ominaisuudet eli sisäiset tekijät vaikuttavat yhteiskunnallisen vaikuttavuuden ilmenemiseen.

Project Success through Effective Decisions: Case Studies on Project Goal Setting, Success Evaluation and Managerial Decision Making

The objective of the thesis is to structure and model the factors that contribute to and can be used in evaluating project success. The purpose of this thesis is to enhance the understanding of three research topics. The goal setting process, success evaluation and decision-making process are studied in the context of a project, business unitand its business environment. To achieve the objective three research questionsare posed. These are 1) how to set measurable project goals, 2) how to evaluateproject success and 3) how to affect project success with managerial decisions.The main theoretical contribution comes from deriving a synthesis of these research topics which have mostly been discussed apart from each other in prior research. The research strategy of the study has features from at least the constructive, nomothetical, and decision-oriented research approaches. This strategy guides the theoretical and empirical part of the study. Relevant concepts and a framework are composed on the basis of the prior research contributions within the problem area. A literature review is used to derive constructs of factors withinthe framework. They are related to project goal setting, success evaluation, and decision making. On the basis of this, the case study method is applied to complement the framework. The empirical data includes one product development program, three construction projects, as well as one organization development, hardware/software, and marketing project in their contexts. In two of the case studiesthe analytic hierarchy process is used to formulate a hierarchical model that returns a numerical evaluation of the degree of project success. It has its origin in the solution idea which in turn has its foundation in the notion of projectsuccess. The achieved results are condensed in the form of a process model thatintegrates project goal setting, success evaluation and decision making. The process of project goal setting is analysed as a part of an open system that includes a project, the business unit and its competitive environment. Four main constructs of factors are suggested. First, the project characteristics and requirements are clarified. The second and the third construct comprise the components of client/market segment attractiveness and sources of competitive advantage. Together they determine the competitive position of a business unit. Fourth, the relevant goals and the situation of a business unit are clarified to stress their contribution to the project goals. Empirical evidence is gained on the exploitation of increased knowledge and on the reaction to changes in the business environment during a project to ensure project success. The relevance of a successful project to a company or a business unit tends to increase the higher the reference level of project goals is set. However, normal performance or sometimes performance below this normal level is intentionally accepted. Success measures make project success quantifiable. There are result-oriented, process-oriented and resource-oriented success measures. The study also links result measurements to enablers that portray the key processes. The success measures can be classified into success domains determining the areas on which success is assessed. Empiricalevidence is gained on six success domains: strategy, project implementation, product, stakeholder relationships, learning situation and company functions. However, some project goals, like safety, can be assessed using success measures that belong to two success domains. For example a safety index is used for assessing occupational safety during a project, which is related to project implementation. Product safety requirements, in turn, are connected to the product characteristics and thus to the product-related success domain. Strategic success measures can be used to weave the project phases together. Empirical evidence on their static nature is gained. In order-oriented projects the project phases are oftencontractually divided into different suppliers or contractors. A project from the supplier's perspective can represent only a part of the ¿whole project¿ viewed from the client's perspective. Therefore static success measures are mostly used within the contractually agreed project scope and duration. Proof is also acquired on the dynamic use of operational success measures. They help to focus on the key issues during each project phase. Furthermore, it is shown that the original success domains and success measures, their weights and target values can change dynamically. New success measures can replace the old ones to correspond better with the emphasis of the particular project phase. This adjustment concentrates on the key decision milestones. As a conclusion, the study suggests a combination of static and dynamic success measures. Their linkage to an incentive system can make the project management proactive, enable fast feedback and enhancethe motivation of the personnel. It is argued that the sequence of effective decisions is closely linked to the dynamic control of project success. According to the used definition, effective decisions aim at adequate decision quality and decision implementation. The findings support that project managers construct and use a chain of key decision milestones to evaluate and affect success during aproject. These milestones can be seen as a part of the business processes. Different managers prioritise the key decision milestones to a varying degree. Divergent managerial perspectives, power, responsibilities and involvement during a project offer some explanation for this. Finally, the study introduces the use ofHard Gate and Soft Gate decision milestones. The managers may use the former milestones to provide decision support on result measurements and ad hoc critical conditions. In the latter milestones they may make intermediate success evaluation also on the basis of other types of success measures, like process and resource measures.

Evaluation of Teacher Education Programs by Students and Graduates

The purpose of this research was to do a repeated cross-sectional research on class teachers who study in the 4th year and also graduated at the Faculty of Education, University of Turku between the years of 2000 through 2004. Specifically, seven research questions were addressed to target the main purpose of the study: How do class teacher education masters’ degree senior students and graduates rate “importance; effectiveness; and quality” of training they have received at the Faculty of Education? Are there significant differences between overall ratings of importance; effectiveness and quality of training by year of graduation, sex, and age (for graduates) and sex and age (for senior students)? Is there significant relationship between respondents’ overall ratings of importance; effectiveness and their overall ratings of the quality of training and preparation they have received? Are there significant differences between graduates and senior students about importance, effectiveness, and quality of teacher education programs? And what do teachers’ [Graduates] believe about how increasing work experience has changed their opinions of their preservice training? Moreover the following concepts related to the instructional activities were studied: critical thinking skills, communication skills, attention to ethics, curriculum and instruction (planning), role of teacher and teaching knowledge, assessment skills, attention to continuous professional development, subject matters knowledge, knowledge of learning environment, and using educational technology. Researcher also tried to find influence of some moderator variables e.g. year of graduation, sex, and age on the dependent and independent variables. This study consisted of two questionnaires (a structured likert-scale and an open ended questionnaire). The population in study 1 was all senior students and 2000-2004 class teacher education masters’ degree from the departments of Teacher Education Faculty of Education at University of Turku. Of the 1020 students and graduates the researcher was able to find current addresses of 675 of the subjects and of the 675 graduates contacted, 439 or 66.2 percent responded to the survey. The population in study 2 was all class teachers who graduated from Turku University and now work in the few basic schools (59 Schools) in South- West Finland. 257 teachers answered to the open ended web-based questions. SPSS was used to produce standard deviations; Analysis of Variance; Pearson Product Moment Correlation (r); T-test; ANOVA, Bonferroni post-hoc test; and Polynomial Contrast tests meant to analyze linear trend. An alpha level of .05 was used to determine statistical significance. The results of the study showed that: A majority of the respondents (graduates and students) rated the overall importance, effectiveness and quality of the teacher education programs as important, effective and good. Generally speaking there were only a few significant differences between the cohorts and groups related to the background variables (gender, age). The different cohorts were rating the quality of the programs very similarly but some differences between the cohorts were found in the importance and effectiveness ratings. Graduates of 2001 and 2002 rated the importance of the program significantly higher than 2000 graduates. The effectiveness of the programs was rated significantly higher by 2001 and 2003 graduates than other groups. In spite of these individual differences between cohorts there were no linear trends among the year cohorts in any measure. In respondents’ ratings of the effectiveness of teacher education programs there was significant difference between males and females; females rated it higher than males. There were no significant differences between males’ and females’ ratings of the importance and quality of programs. In the ratings there was only one difference between age groups. Older graduates (35 years or older) rated the importance of the teacher training significantly higher that 25-35 years old graduates. In graduates’ ratings there were positive but relatively low correlations between all variables related to importance, effectiveness and quality of Teacher Education Programs. Generally speaking students’ ratings about importance, effectiveness and quality of teacher education program were very positive. There was only one significant difference related to the background variables. Females rated higher the effectiveness of the program. The comparison of students’ and graduates’ perception about importance, effectiveness, and quality of teacher education programs showed that there were no significant differences between graduates and students in the overall ratings. However there were differences in some individual variables. Students rated higher in importance of “Continuous Professional Development”, effectiveness of “Critical Thinking Skills” and “Using Educational Technology” and quality of “Advice received from the advisor”. Graduates rated higher in importance of “Knowledge of Learning Environment” and effectiveness of “Continuous Professional Development”. According to the qualitative data of study 2 some graduates expressed that their perceptions have not changed about the importance, effectiveness, and quality of training that they received during their study time. They pointed out that teacher education programs have provided them the basic theoretical/formal knowledge and some training of practical routines. However, a majority of the teachers seems to have somewhat critical opinions about the teacher education. These teachers were not satisfied with teacher education programs because they argued that the programs failed to meet their practical demands in different everyday situations of the classroom e.g. in coping with students’ learning difficulties, multiprofessional communication with parents and other professional groups (psychologists and social workers), and classroom management problems. Participants also emphasized more practice oriented knowledge of subject matter, evaluation methods and teachers’ rights and responsibilities. Therefore, they (54.1% of participants) suggested that teacher education departments should provide more practice-based courses and programs as well as closer collaboration between regular schools and teacher education departments in order to fill gap between theory and practice.

Program hosting in communications environment with Reactor and thread pool approach

In this thesis concurrent communication event handling is implemented using thread pool approach. Concurrent events are handled with a Reactor design pattern and multithreading is implemented using a Leader/Followers design pattern. Main focus is to evaluate behaviour of implemented model by different numbers of concurrent connections and amount of used threads. Furthermore, model feasibility in a PeerHood middleware is evaluated. Implemented model is evaluated with created test environment which enables concurrent message sending from multiple connections to the system under test. Messages round trip times are measured in the tester application. In the evaluation processing delay into system is simulated and influence of delay to the average round trip time is analysed.

Proactive and Preventive Student Welfare Activities in Finnish Preschool and Elementary School: Handling of Transition to Formal Schooling and a National Anti-Bullying Program as Examples

Prerequisites and effects of proactive and preventive psycho-social student welfare activities in Finnish preschool and elementary school were of interest in the present thesis. So far, Finnish student welfare work has mainly focused on interventions and individuals, and the voluminous possibilities to enhance well-being of all students as a part of everyday school work have not been fully exploited. Consequently, in this thesis three goals were set: (1) To present concrete examples of proactive and preventive psycho-social student welfare activities in Finnish basic education; (2) To investigate measurable positive effects of proactive and preventive activities; and (3) To investigate implementation of proactive and preventive activities in ecological contexts. Two prominent phenomena in preschool and elementary school years—transition to formal schooling and school bullying—were chosen as examples of critical situations that are appropriate targets for proactive and preventive psycho-social student welfare activities. Until lately, the procedures concerning both school transitions and school bullying have been rather problem-focused and reactive in nature. Theoretically, we lean on the bioecological model of development by Bronfenbrenner and Morris with concentric micro-, meso-, exo- and macrosystems. Data were drawn from two large-scale research projects, the longitudinal First Steps Study: Interactive Learning in the Child–Parent– Teacher Triangle, and the Evaluation Study of the National Antibullying Program KiVa. In Study I, we found that the academic skills of children from preschool–elementary school pairs that implemented several supportive activities during the preschool year developed more quickly from preschool to Grade 1 compared with the skills of children from pairs that used fewer practices. In Study II, we focused on possible effects of proactive and preventive actions on teachers and found that participation in the KiVa antibullying program influenced teachers‘ self-evaluated competence to tackle bullying. In Studies III and IV, we investigated factors that affect implementation rate of these proactive and preventive actions. In Study III, we found that principal‘s commitment and support for antibullying work has a clear-cut positive effect on implementation adherence of student lessons of the KiVa antibullying program. The more teachers experience support for and commitment to anti-bullying work from their principal, the more they report having covered KiVa student lessons and topics. In Study IV, we wanted to find out why some schools implement several useful and inexpensive transition practices, whereas other schools use only a few of them. We were interested in broadening the scope and looking at local-level (exosystem) qualities, and, in fact, the local-level activities and guidelines, along with teacherreported importance of the transition practices, were the only factors significantly associated with the implementation rate of transition practices between elementary schools and partner preschools. Teacher- and school-level factors available in this study turned out to be mostly not significant. To summarize, the results confirm that school-based promotion and prevention activities may have beneficial effects not only on students but also on teachers. Second, various top-down processes, such as engagement at the level of elementary school principals or local administration may enhance implementation of these beneficial activities. The main message is that when aiming to support the lives of children the primary focus should be on adults. In future, promotion of psychosocial well-being and the intrinsic value of inter- and intrapersonal skills need to be strengthened in the Finnish educational systems. Future research efforts in student welfare and school psychology, as well as focused training for psychologists in educational contexts, should be encouraged in the departments of psychology and education in Finnish universities. Moreover, a specific research centre for school health and well-being should be established.

Evaluation of novel programs in health counseling – their effectiveness in the oral health of young children, and their acceptance by parents and professionals in public dental service.

Aim and design: To evaluate family-based health counseling for young children, and to study the significance of adding parental self-care or the training of professionals to the programs. The effectiveness and acceptability of the programs were evaluated by comparing two new programs with an earlier one. Subjects and methods: The study was carried out in Vantaa, which was divided into three study areas. The subjects consisted of children born in 2008, particularly fi rstborn children, while children born in 2006 formed the historical control. The fi rst of the new programs emphasized oral hygiene and use of fl uoride, and the second program focused on proper diet and use of xylitol. The main outcome measure was mutansstreptococci (MS) in the dental biofi lm of two-year-olds, and the opinions of parents and dental professionals were evaluated using questionnaires. Results: The programs found wide acceptance among dental professionals. There were no group-related differences found in the MS scores of the two-year-olds. However, all groups combined, father’s advanced level of education and child’s proper use of xylitol were associated with negative MS scores. In the opinion of parents, the oral healthcare guidance at least somewhat met their expectations. Conclusions: The present fi ndings suggest that providing training and support for professionals in health education is important. The addition of parental self-care to supplement programs aimed at young children does not improve the program, although it may improve parental readiness to change their own health habits. Counseling for families might be best carried out through a routine patient-centered program.