Sources of selection bias in evaluating social programs: An interpretation of conventional measures and evidence on the effectiveness of matching as a program evaluation method

This paper decomposes the conventional measure of selection bias in observational studies into three components. The first two components are due to differences in the distributions of characteristics between participant and nonparticipant (comparison) group members: the first arises from differences in the supports, and the second from differences in densities over the region of common support. The third component arises from selection bias precisely defined. Using data from a recent social experiment, we find that the component due to selection bias, precisely defined, is smaller than the first two components. However, selection bias still represents a substantial fraction of the experimental impact estimate. The empirical performance of matching methods of program evaluation is also examined. We find that matching based on the propensity score eliminates some but not all of the measured selection bias, with the remaining bias still a substantial fraction of the estimated impact. We find that the support of the distribution of propensity scores for the comparison group is typically only a small portion of the support for the participant group. For values outside the common support, it is impossible to reliably estimate the effect of program participation using matching methods. If the impact of participation depends on the propensity score, as we find in our data, the failure of the common support condition severely limits matching compared with random assignment as an evaluation estimator.

Low-status monkeys “play dumb” when learning in mixed social groups

Many primates, including humans, live in complex hierarchical societies where social context and status affect daily life. Nevertheless, primate learning studies typically test single animals in limited laboratory settings where the important effects of social interactions and relationships cannot be studied. To investigate the impact of sociality on associative learning, we compared the individual performances of group-tested rhesus monkeys (Macaca mulatta) across various social contexts. We used a traditional discrimination paradigm that measures an animal’s ability to form associations between cues and the obtaining of food in choice situations; but we adapted the task for group testing. After training a 55-member colony to separate on command into two subgroups, composed of either high- or low-status families, we exposed animals to two color discrimination problems, one with all monkeys present (combined condition), the other in their “dominant” and “subordinate” cohorts (split condition). Next, we manipulated learning history by testing animals on the same problems, but with the social contexts reversed. Monkeys from dominant families excelled in all conditions, but subordinates performed well in the split condition only, regardless of learning history. Subordinate animals had learned the associations, but expressed their knowledge only when segregated from higher-ranking animals. Because aggressive behavior was rare, performance deficits probably reflected voluntary inhibition. This experimental evidence of rank-related, social modulation of performance calls for greater consideration of social factors when assessing learning and may also have relevance for the evaluation of human scholastic achievement.