Adaptive Crowd Algorithms for Open-Ended Problems

Thesis (Ph.D.)--University of Washington, 2016-06

Decomposing problems is fundamental to solving them for both people and computers. When it comes to problem solving, people and computers have complementary approaches. Computer algorithms can methodically solve large problems that require lots of state, organization, and memory, but they can only solve problems that are well-defined and have explicit steps. People are not quite as methodical, but can solve problems that are ill-defined and open-ended. This dissertation contributes concepts and techniques, embodied in software artifacts, to answer the following research question: How we can combine the complementary skills of people and computers to solve open-ended problems systematically? From the literature on human problem solving, design, and sensemaking, we know that the process people use to solve open-ended problems is not linear but iterative. People start with the concrete context of the situation to generate ideas, then they dynamically discover the parameters of the problem and adapt to them. To combine people's and computers' abilities, we must decompose the process of solving open-ended problems into explicit steps like an algorithm, but integrate human intelligence for the steps that computers cannot yet do. My dissertation shows how to systematically solve open-ended problems with adaptive crowd algorithms. Adaptive crowd algorithms use crowdsourced microtasks to explore a solution space in incremental steps and test solutions until the goal is met. Because the problems are often large, workers are given only partial information about the problem and respond with proposals for partial solutions. Partial solutions can be tested against the goal and built upon by future microtasks to explore more of the solution space. To arrive at a cohesive output, adaptive mechanisms use partial solution to iterate towards the goal by exploring multiple ideas, testing ideas, adapting to feedback and nudging the output into a tested solution. To demonstrate my thesis, I introduce three systems that systematically solve open-ended problems with adaptive crowd algorithms: * Cascade crowdsources the open-ended problem of taxonomy creation. * Frenzy coordinates a crowd of experts to meet the constraint of organizing accepted conference papers into thematic sessions. * HumorTools decomposes the creative task for writing humorous news satire in the style of My evaluation shows that with adaptive crowd algorithms, we can solve open-ended problems too big for one person, too ill-defined to automate and that require creativity.