Sources of selection bias in evaluating social programs: An interpretation of conventional measures and evidence on the effectiveness of matching as a program evaluation method

This paper decomposes the conventional measure of selection bias in observational studies into three components. The first two components are due to differences in the distributions of characteristics between participant and nonparticipant (comparison) group members: the first arises from differences in the supports, and the second from differences in densities over the region of common support. The third component arises from selection bias precisely defined. Using data from a recent social experiment, we find that the component due to selection bias, precisely defined, is smaller than the first two components. However, selection bias still represents a substantial fraction of the experimental impact estimate. The empirical performance of matching methods of program evaluation is also examined. We find that matching based on the propensity score eliminates some but not all of the measured selection bias, with the remaining bias still a substantial fraction of the estimated impact. We find that the support of the distribution of propensity scores for the comparison group is typically only a small portion of the support for the participant group. For values outside the common support, it is impossible to reliably estimate the effect of program participation using matching methods. If the impact of participation depends on the propensity score, as we find in our data, the failure of the common support condition severely limits matching compared with random assignment as an evaluation estimator.

Access, equity and social cohesion: evaluation of intersectoral strategies for people experiencing homelessness

Objective To understand and evaluate the work of intersectoral assistance on the insertion and the flow of people in situation of street with severe mental illness in public services of Mental Health. Method A case study developed from ten visits to a night shelter between March and April 2012. For data collection, the participant observation and semi-structured interviews were carried out with four sheltered individuals, as well as non-directive group interviews with five technicians of the social-assistance services. Results Were analyzed using Content Analysis and developing a Logic Model validated with the professionals involved. Conclusion The social assistance services are the main entry of this clientele in the public network of assistance services, and the Mental Health services have difficulty in responding to the specificities of the same clientele and in establishing intersectoral work.


Cap a on van els programes de garantia social

En el present estudi es descriu l'experiència de la implantació dels PGS en la comunitat de Castella-La Manxa, partint d'un marc conceptual en el que es descriuen les modalitats i característiques de programes. Es detallen les dades obtingudes mitjançant d'un estudi empíric que ofereix informació sobre els diferents agents (professorat, alumnes, famílies) que concurreixen en tals Programes de Garantía Social.

Rationalizing social policy: evaluation and viability

Includes bibliography

Avaliação do impacto dos programas de transferência monetária condicionada na taxa de frequência escolar e labor infantil: o caso do Programa JUNTOS no Peru

Na presente dissertação tem como objetivo avaliar o impacto do programa JUNTOS sobre a taxa de freqüência escolar e sobre o trabalho infantil nas crianças de 6 a 14 anos. Estas duas variáveis foram selecionadas para o seu estudo, pois ao nosso entender estas são as principais variáveis que são influenciadas pelo programa JUNTOS e que tem uma influencia direta sobre o capital humano das crianças e assim sobre a diminuição da pobreza futura. As principais hipóteses derivadas das teorias de capital humano e de transferências de rendas condicionadas foram corroboradas pela nossa avaliação: (1) o programa JUNTOS tem um efeito positivo sobre o incremento da freqüência escolar, (2) o programa JUNTOS é efetivo na redução do trabalho infantil, (3) quando o chefe de família é de sexo feminino, a renda familiar é utilizada em bens e serviços em favor das crianças, e (4) o efeito do programa JUNTOS é maior nas crianças com piores características socioeconômicas (ex: menor renda familiar per capita, chefe de família com poucos anos de estudo, idioma do chefe de família, etc.) Outra conclusão importante da dissertação foi que o programa JUNTOS provoca uma realocação na oferta de trabalho intra familiar.

Herramienta de evaluación para programas de inclusión en actividad física, recreación y deporte para personas con discapacidad en Colombia

Introducción: La evaluación de programas sociales, ha sido, desde sus inicios, de gran importancia para fortalecer los procesos de toma de decisiones de los hacedores de los programas, ya que retroalimenta lo ejecutado y ayuda a aplicar correctivos necesarios para lograr el impacto deseado en la población objetivo, que generalmente son comunidades vulnerables como las personas con discapacidad. En los últimos años, esta población ha sido foco de múltiples programas y proyectos en diferentes temáticas como la actividad física, la recreación, el deporte y la educación física. Por esto, en el año 2013, Coldeportes en aras de poder parametrizar y permitir un impacto adecuado en esta población, diseñó un grupo de lineamientos los cuales son los encargados de generar las directrices de los programas que se diseñen y ejecuten en los entes territoriales. Para poder realizar el seguimiento y resultado de su aplicación, es necesario hacerlo por intermedio de una herramienta de evaluación. De acuerdo con las últimas tendencias en la evaluación de programas sociales, debe hacerse por intermedio de indicadores, los cuales permiten realizarla de una manera más eficiente. Por tal motivo, la herramienta de evaluación para los lineamientos será a través de indicadores. Objetivo: Generar una herramienta de evaluación para los programas y actividades en actividad física, recreación y deporte para la población con discapacidad en Colombia, a partir de los lineamientos de inclusión definidos por Coldeportes. Metodología: Se realizó una revisión bibliográfica acerca de la evaluación de programas sociales, indicadores de evaluación y población con discapacidad, seguido a esto se diseñaron los indicadores de evaluación con base en las variables de los lineamientos propuestos por Coldeportes, posteriormente, los indicadores fueron sometidos a una validación en un grupo nominal, conformado por expertos en programas sociales y trabajo en población con discapacidad, por último, se sistematizaron y analizaron los resultados. iii Resultados: La herramienta de evaluación fue validada por parte de los expertos, hubo ajustes en los indicadores de evaluación de los lineamientos de accesibilidad a la comunicación e información, accesibilidad a la formación académica y accesibilidad al entorno físico. Conclusiones: La herramienta de evaluación diseñada, es un primer paso para mejorar los métodos actuales de evaluación, encaminada a fortalecer los datos existentes de la población con discapacidad y tener una visión del impacto de los planes, programas y proyectos en actividad física, recreación y deporte ejecutados en la actualidad, por lo cual queda continuar con la investigación en nuevas alternativas de evaluación que permitan mejorar los procesos de toma de decisiones y así garantizar planes, programas y proyectos sociales adecuados para la población, sin distinguir características particulares.

Method for the evaluation of social projects

This instrument was part of the research project "Research on Evaluation of Health and Education Plans and Programs in the Province of Buenos Aires", developed by the of Chair Preventive Psychology of the Psychology course of studies at the School of Humanities and Educational Sciences, National Univer- sity of La Plata (Argentina). The basis for proposing an assessment instrument is the need for a method enabling analysis, systematization of knowledge and the assignment of values distributed into scales and organized in general charts, on social programs. Its main concern is the analysis of health and education programs and projects, restricted to certain specific areas or regions, in search for theoretic trustworthi-ness, methodological accuracy as well as pragmatic operability. This is the result of four years of researching said programs at system, service and community levels.

Method for the evaluation of social projects

This instrument was part of the research project "Research on Evaluation of Health and Education Plans and Programs in the Province of Buenos Aires", developed by the of Chair Preventive Psychology of the Psychology course of studies at the School of Humanities and Educational Sciences, National Univer- sity of La Plata (Argentina). The basis for proposing an assessment instrument is the need for a method enabling analysis, systematization of knowledge and the assignment of values distributed into scales and organized in general charts, on social programs. Its main concern is the analysis of health and education programs and projects, restricted to certain specific areas or regions, in search for theoretic trustworthi-ness, methodological accuracy as well as pragmatic operability. This is the result of four years of researching said programs at system, service and community levels.

Method for the evaluation of social projects

This instrument was part of the research project "Research on Evaluation of Health and Education Plans and Programs in the Province of Buenos Aires", developed by the of Chair Preventive Psychology of the Psychology course of studies at the School of Humanities and Educational Sciences, National Univer- sity of La Plata (Argentina). The basis for proposing an assessment instrument is the need for a method enabling analysis, systematization of knowledge and the assignment of values distributed into scales and organized in general charts, on social programs. Its main concern is the analysis of health and education programs and projects, restricted to certain specific areas or regions, in search for theoretic trustworthi-ness, methodological accuracy as well as pragmatic operability. This is the result of four years of researching said programs at system, service and community levels.

Los desaf??os de las pol??ticas y la gesti??n social en Am??rica Latina

La pobreza est?? aumentando en t??rminos absolutos de manera sistem??tica desde 1990 y, desde comienzos del nuevo siglo, tambi??n en valores relativos. Hay, sin embargo, una buena nueva: el gasto p??blico social ha aumentado tambi??n de manera sistem??tica desde entonces. ??Qu?? se ha hecho con la enorme magnitud de recursos destinada a financiar las pol??ticas sociales? Hay, por un lado, limitaciones que surgen de la forma en que ellas se dise??an e implementan. La gesti??n social tradicional asume que el impacto perseguido se producir?? autom??ticamente como resultado de la mera entrega de los bienes o servicios a la poblaci??n destinataria. Aparece, adem??s, la masiva introducci??n de mecanismos de mercado en las pol??ticas sociales, bajo el supuesto que permitir??an mejorar su eficiencia e impacto. Teniendo en cuenta las limitaciones precedentes, se sugieren en este art??culo tres senderos, estos son: la superaci??n del estilo tradicional de gestionar los programas y proyectos sociales; el aprendizaje por medio de la experiencia de la evaluaci??n de impacto y del monitoreo; y la generaci??n de un estilo m??s participativo en el proceso de dise??o, gesti??n y evaluaci??n de las pol??ticas sociales.

Prospectiva en torno a las grandes dimensiones de los programas de Garantía Social.

El artículo se divide en cuatro apartados. En el primero, se presentan los antecedentes y el marco legal de los Programas de Garantía Social (PGS). En el segundo, se concretan los objetivos y modalidades de estos programas, las características del alumnado que los cursa y el perfil del profesorado que los imparte. El tercer apartado se centra en el contexto socio-educativo de la Comunidad de Castilla La Mancha y el estudio de los PGS que en ella se desarrollan. Además, se presentan las dimensiones de los cuestionarios elaborados para realizarlo. Por último, en el cuarto apartado, se exponen las conclusiones derivadas de su aplicación y análisis en el estudio de la implantación de los PGS en la Comunidad de Castilla La Mancha, y se hacen unas propuestas de optimizaciön que pueden generalizarse a otras comunidades autónomas.

Educator Evaluation of Academic and Social Competence in Students with Acquired Brain Injury (ABI) Relative to Assessed Performance and Sense of Belonging

Acquired brain injury (ABI) is the leading cause of death and disability amongst children and adolescents andpresents itself with challenges associated in cognitive, social, emotional, and behavioural domains. These changes may interfere with academic performance and social inclusion, influencing self-esteem and personal success. The current study examined a subset of data to capture the sense of academic and social belonging for students with ABI as a function of the classroom teachers’ subjective perception of ability, their ABI knowledge, and student identification. Overall, a discrepancy was found between educators’ subjective ratings of student performance and students’ neurocognitive capacity. Educator knowledge and identification of ABI influenced student success in academic and social domains independent of teaching approach. This research has implications for the identification of ABI in the classroom and related challenges students experience. Educators are underprepared for the reintegration of students returning to school and lack appropriate knowledge and strategies to accommodate individual needs.

O novo federalismo social e o Rio: desenho de programas complementares de transferência de renda condicionada

Brazilian public policy entered in the so-called new social federalism through its conditional cash transfers. States and municipalities can operate together through the nationwide platform of the Bolsa Familia Program (BFP), complementing federal actions with local innovations. The state and the city of Rio de Janeiro have created programs named, respectively, Renda Melhor (RM) and Família Carioca (FC). These programs make use of the operational structure of the BFP, which facilitates locating beneficiaries, issuing cards, synchronizing payment dates and access passwords and introducing new conditionalities. The payment system of the two programs complements the estimated permanent household income up to the poverty line established, giving more to those who have less. Similar income complementation system was subsequently adopted in the BFP and the Chilean Ingreso Ético Familiar, which also follow the principle of estimation of income used in the FC and in the RM. Instead of using the declared income, the value of the Rio cash transfers are set using the extensive collection of information obtained from the Single Registry of Social Programs (Cadastro Único): physical configuration of housing, access to public services, education and work conditions for all family members, presence of vulnerable groups, disabilities, pregnant or lactating women, children and benefits from other official transfers such as the BFP. With this multitude of assets and limitations, the permanent income of each individual is estimated. The basic benefit is defined by the poverty gap and priority is given to the poorest. These subnational programs use international benchmarks as a neutral ground between different government levels and mandates. Their poverty line is the highest of the first millennium goal of the United Nations (UN): US$ 2 per person per day adjusted for the cost of living. The other poverty line of the UN, US$ 1.25, was implicitly adopted as the national extreme poverty line in 2011. The exchange of methodologies between federal entities has happened both ways. The FC began with the 575,000 individuals living in the city of Rio de Janeiro who were on the payroll of the BFP. Its system of impact evaluation benefited from bi-monthly standardized examinations. In the educational conditionalities, the two programs reward students' progress, a potential advantage for those who most need to advance. The municipal program requires greater school attendance than that of the BFP and the presence of students’ parents at the bimonthly meetings held on Saturdays. Students must achieve a grade of 8 or improve at least 20% in each exam to receive a bi-monthly premium of R$50. In early childhood, priority is given to the poor children in the program Single Administrative Register (CadÚnico) to enroll in kindergarten, preschools and complementary activities. The state program reaches more than one million people with a payment system similar to the municipal one. Moreover, it innovates in that it transfers awards given to high school students to savings accounts. The prize increases and is paid to the student, who can withdraw up to 30% annually. The total can reach R$3,800 per low-income student. The State and the city rewarded already education professionals according to student performance, now completing the chain of demand incentives on poor students and their parents. Increased performance is higher among beneficiaries and the presence of their guardians at meetings is twice compared to non beneficiaries; The Houston program, also focuses on aligning the incentives to teachers, parents and students. In general, the plan is to explore strategic complementarities, where the whole is greater than the sum of its parts. The objective is to stimulate, through targets and incentives, synergies between social actors (teachers, parents, students), between areas (education, assistance, work) and different levels of government. The cited programs sum their efforts and divide labor so as to multiply interactions and make a difference in the lives of the poor.

Social policies and socioeconomic indicators for transitional economies

Includes bibliography

O Serviço Social e a avaliação de impacto na gestão de programas e projetos empresariais

Pós-graduação em Serviço Social - FCHS