Eliciting and Aggregating Forecasts When Information is Shared

Using the wisdom of crowds---combining many individual forecasts to obtain an aggregate estimate---can be an effective technique for improving forecast accuracy. When individual forecasts are drawn from independent and identical information sources, a simple average provides the optimal crowd forecast. However, correlated forecast errors greatly limit the ability of the wisdom of crowds to recover the truth. In practice, this dependence often emerges because information is shared: forecasters may to a large extent draw on the same data when formulating their responses.

To address this problem, I propose an elicitation procedure in which each respondent is asked to provide both their own best forecast and a guess of the average forecast that will be given by all other respondents. I study optimal responses in a stylized information setting and develop an aggregation method, called pivoting, which separates individual forecasts into shared and private information and then recombines these results in the optimal manner. I develop a tailored pivoting procedure for each of three information models, and introduce a simple and robust variant that outperforms the simple average across a variety of settings.

In three experiments, I investigate the method and the accuracy of the crowd forecasts. In the first study, I vary the shared and private information in a controlled environment, while the latter two studies examine forecasts in real-world contexts. Overall, the data suggest that a simple minimal pivoting procedure provides an effective aggregation technique that can significantly outperform the crowd average.

How Vladimir Putin’s Divorce Story Was Constructed and Received, or When the President Divorced His Wife and Married the Country Instead

When the Russian president divorced his wife in 2013, social media crowds coined plenty of the Internet memes to interpret the news. Anastasia Denisova, Doctoral Researcher at CAMRI, examined the framing of the story in traditional and new media and came to the surprising findings.

Temor, silencio y deliberación: la inhibición de la opinión en Tucídides

Tucídides es una de nuestras mejores fuentes de información para conocer la práctica argumentativa de la deliberación democrática. En este trabajo se analiza uno de los vicios que, según el historiador, haría su aparición en la escena política ateniense a la muerte de Pericles: la instrumentalización del miedo para obtener la victoria momentánea en la asamblea. El temor prudente, que fuera una arma periclea para conducir la deliberación racional en aras del bien común, habría desaparecido siendo sustituido por el amedrentamiento del rival, la calumnia, el obstruccionismo y la parálisis de la confrontación dialéctica. Instauradas en la ciudad la desconfianza y la sospecha de ocultación, los golpistas del 411 hallaron el terreno abonado para callar las voces contrarias y, gracias al silencio, instaurar el terror.

The Dynamics of Quadropoly: League Position in English Football between 1888 and 2010

The paper explores competitive balance in top tier English league football from its inception in 1888. It examines the extent to which finishing in the top four positions in successive seasons is the preserve of a small number of clubs. Using a range of statistical measures, the analysis shows that the current high levels of competitive imbalance are not new phenomena. The overall pattern approximates a ‘U curve’: current patterns parallel those in the 1890s. In the early years of English league football, differences in resources between clubs soon became apparent. Clubs from the larger conurbations generated consistently larger revenues than their counterparts in the smaller industrial towns. This was primarily the result of the larger crowds that they could attract to their home games. This enabled them to entice the best players to their clubs away from their smaller rivals. The introduction of the maximum wage in 1901 and the transfer system helped to stem these increasing inequalities between clubs. This coincided with a massive wave of new stadia construction which enabled all the clubs to compete on an increasingly level playing field. These conjunctural changes to English football before 1915 produced the era of relatively competitive football during the inter-war years. This continued until the abolition of the maximum wage in 1961. Since that time, competitive balance has reversed and become increasingly restricted. English top-tier football has re-entered an era of extreme competitive imbalance.

Crime and Punishment.

The study of Victorian crime and punishment is a rich area of research that has attracted the interest not only of literary scholars but also of social historians, legal historians, and criminologists. Related scholarship therefore often situates itself at the intersection of traditional disciplinary boundaries, facilitating interdisciplinary conversation. Crime and punishment was a pressing issue for the Victorians and provoked a wealth of responses from contemporaneous commentators in literature, culture, and science. As a new phase of industrialization brought immense wealth for some and abject poverty for others, Victorian urban centers in particular were afflicted by crime. Without an effective system of social welfare in place, social inequality and deprivation drove women, men, and children into petty crime and more serious offenses, resulting in severe punishment ranging from incarceration via penal transportation to hanging. Public executions, not abolished until 1868, attracted huge crowds of spectators, including authors such as Charles Dickens and William Thackeray, who wrote about these experiences. A forerunner of the popular press, street literature conveyed and illustrated these events for a broad audience. Execution broadsides of famous cases, printing the alleged last lamentations of convicts on the scaffold in verse, are estimated to have sold by the million. As the legal system was undergoing reform (comprising changes in legal evidence procedure, divorce law, women’s property rights, and punishment for sexual offenses, for example), sensational trials caused furor and stimulated commentary in literature and the media. Crime and punishment was discussed in a range of literary and popular genres, poetry, and reformist writing. The “Newgate School” of fiction was accused of glamorizing crime, and the popular penny dreadfuls were feared to corrupt public morals. Sensational fiction in the 1860s, which often drew on real-life criminal cases and newspaper reports, depicted the supposedly respectable middle-class family home as a center of transgression. Similarly, detective fiction typically focused on crime in the world of the middle classes. For the student new to the subject of crime and punishment, this area’s interdisciplinary nature can pose an initial challenge.

Security and Trust in Distributed Computation

We propose three research problems to explore the relations between trust and security in the setting of distributed computation. In the first problem, we study trust-based adversary detection in distributed consensus computation. The adversaries we consider behave arbitrarily disobeying the consensus protocol. We propose a trust-based consensus algorithm with local and global trust evaluations. The algorithm can be abstracted using a two-layer structure with the top layer running a trust-based consensus algorithm and the bottom layer as a subroutine executing a global trust update scheme. We utilize a set of pre-trusted nodes, headers, to propagate local trust opinions throughout the network. This two-layer framework is flexible in that it can be easily extensible to contain more complicated decision rules, and global trust schemes. The first problem assumes that normal nodes are homogeneous, i.e. it is guaranteed that a normal node always behaves as it is programmed. In the second and third problems however, we assume that nodes are heterogeneous, i.e, given a task, the probability that a node generates a correct answer varies from node to node. The adversaries considered in these two problems are workers from the open crowd who are either investing little efforts in the tasks assigned to them or intentionally give wrong answers to questions. In the second part of the thesis, we consider a typical crowdsourcing task that aggregates input from multiple workers as a problem in information fusion. To cope with the issue of noisy and sometimes malicious input from workers, trust is used to model workers' expertise. In a multi-domain knowledge learning task, however, using scalar-valued trust to model a worker's performance is not sufficient to reflect the worker's trustworthiness in each of the domains. To address this issue, we propose a probabilistic model to jointly infer multi-dimensional trust of workers, multi-domain properties of questions, and true labels of questions. Our model is very flexible and extensible to incorporate metadata associated with questions. To show that, we further propose two extended models, one of which handles input tasks with real-valued features and the other handles tasks with text features by incorporating topic models. Our models can effectively recover trust vectors of workers, which can be very useful in task assignment adaptive to workers' trust in the future. These results can be applied for fusion of information from multiple data sources like sensors, human input, machine learning results, or a hybrid of them. In the second subproblem, we address crowdsourcing with adversaries under logical constraints. We observe that questions are often not independent in real life applications. Instead, there are logical relations between them. Similarly, workers that provide answers are not independent of each other either. Answers given by workers with similar attributes tend to be correlated. Therefore, we propose a novel unified graphical model consisting of two layers. The top layer encodes domain knowledge which allows users to express logical relations using first-order logic rules and the bottom layer encodes a traditional crowdsourcing graphical model. Our model can be seen as a generalized probabilistic soft logic framework that encodes both logical relations and probabilistic dependencies. To solve the collective inference problem efficiently, we have devised a scalable joint inference algorithm based on the alternating direction method of multipliers. The third part of the thesis considers the problem of optimal assignment under budget constraints when workers are unreliable and sometimes malicious. In a real crowdsourcing market, each answer obtained from a worker incurs cost. The cost is associated with both the level of trustworthiness of workers and the difficulty of tasks. Typically, access to expert-level (more trustworthy) workers is more expensive than to average crowd and completion of a challenging task is more costly than a click-away question. In this problem, we address the problem of optimal assignment of heterogeneous tasks to workers of varying trust levels with budget constraints. Specifically, we design a trust-aware task allocation algorithm that takes as inputs the estimated trust of workers and pre-set budget, and outputs the optimal assignment of tasks to workers. We derive the bound of total error probability that relates to budget, trustworthiness of crowds, and costs of obtaining labels from crowds naturally. Higher budget, more trustworthy crowds, and less costly jobs result in a lower theoretical bound. Our allocation scheme does not depend on the specific design of the trust evaluation component. Therefore, it can be combined with generic trust evaluation algorithms.

THE PHYSICS OF IDEAS: INFERRING THE MECHANICS OF OPINION FORMATION FROM MACROSCOPIC STATISTICAL PATTERNS

In a microscopic setting, humans behave in rich and unexpected ways. In a macroscopic setting, however, distinctive patterns of group behavior emerge, leading statistical physicists to search for an underlying mechanism. The aim of this dissertation is to analyze the macroscopic patterns of competing ideas in order to discern the mechanics of how group opinions form at the microscopic level. First, we explore the competition of answers in online Q&A (question and answer) boards. We find that a simple individual-level model can capture important features of user behavior, especially as the number of answers to a question grows. Our model further suggests that the wisdom of crowds may be constrained by information overload, in which users are unable to thoroughly evaluate each answer and therefore tend to use heuristics to pick what they believe is the best answer. Next, we explore models of opinion spread among voters to explain observed universal statistical patterns such as rescaled vote distributions and logarithmic vote correlations. We introduce a simple model that can explain both properties, as well as why it takes so long for large groups to reach consensus. An important feature of the model that facilitates agreement with data is that individuals become more stubborn (unwilling to change their opinion) over time. Finally, we explore potential underlying mechanisms for opinion formation in juries, by comparing data to various types of models. We find that different null hypotheses in which jurors do not interact when reaching a decision are in strong disagreement with data compared to a simple interaction model. These findings provide conceptual and mechanistic support for previous work that has found mutual influence can play a large role in group decisions. In addition, by matching our models to data, we are able to infer the time scales over which individuals change their opinions for different jury contexts. We find that these values increase as a function of the trial time, suggesting that jurors and judicial panels exhibit a kind of stubbornness similar to what we include in our model of voting behavior.

Economics for Substantive Democracy

In this paper, I sustain that the discourse of economics has become one of the most substantial contributors to the erosion of the democratic ideal. The first argument used in this case against economics concerns its attempt to be considered a neo- naturalistic science; the second, the fact that economics considers democracy contradictory to the expression of its scientific rationality and; the third, the fact that economics crowds out people from decision-making processes by pushing them into the hands of experts. Because partisan political programmes have essentially become economic programmes, economics should contribute to substantive democracy. In order to do so, an alternative discourse to mainstream economics must be proposed. An economics favourable to substantive democracy should, thereby, be political rather than naturalistic, pluralist rather than monist and, instead of crowding out people from decisions processes, should aim at the co-production of economic knowledge with those concerned by the outcome of economic decisions.

East Harbor State Park

East Harbor State Park attracted huge crowds on warm summer weekends, as this 1950s lineup of vehicles attests. Automobile travel expanded quickly in the 1950s as the economy grew and roads improved. The cities that provided the region with many of its visitors, like Cleveland, Detroit, and Toledo were at their peak in the 1950s and 1960s.

EACS: Effective Avoidance Combination Strategy

International audience