1. Delphi Opinion Cycle (Forecasting method)
The Delphi forecasting method was invented in the 1950s by foresight pioneers Olaf Helmer, Norman Dalkey, and Nicholas Rescher at America’s first foresight think tank, RAND. For an exceptional early foresight work, see Helmer’s Looking Forward: A Guide to Futures Research (1983). Delphi is one of the basic methods of modern foresight. It represents any iterative (cyclic) systematic effort at collecting expert or group opinion, promoting group interaction and research, and mapping the convergence (or not) of opinion that ensues. Named after the Oracle of Delphi (1400BCE to 395) it seeks to find better versions of both the consensus future (if one exists) and of alternative futures, so these can be better described and used as inputs to decision-making. The method can be used with experts or any lay group of stakeholders.
Pythia, Priestess of Delphi
(John Collier, 1891)
Delphi is a three stage cyclic process of:
- Opinion Collection
- Clarification of Opinions
- Giving Feedback to the Opinions
This cycle is repeated until the marginal group learning is outweighed by the time and resources involved in turning through any more cycles. The best Delphi exercises often begin with 1) Anonymous Opinion Collection, ideally about probable, possible, and preferable futures, as different individuals are often interested in giving opinions on each of these three core future types.
The anonymity in the early rounds strips away egos and political influence, and focuses the group on considering the evidence and arguments for the positions themselves. After 1) happens, everyone gets to make 2) Anonymous Requests for Clarification and Additional Data from those who have proposed opinions that they find controversial, don’t understand, disagree with, or agree with and want to see further supported. After 2), everyone then has an opportunity and is encouraged to do additional research, online or otherwise, to respond to these clarification requests, and then post whatever new evidence or argument they can. The group is next asked to 3) Anonymously Re-Rank the original offered opinions, in quantifiable ways. For example, they may be given 100 virtual coins or points to distribute among each opinion, to generate a quantitative preference, or to determine their estimate of the likelihood of a proposed future.
Including anonymous rounds at the beginning of a Delphi cycle is often unpopular in low-trust, traditional, groupthink-dominated, or feedback-averse firms, but it should be insisted upon, as this allows true opinions to more easily surface, and shows what people are really thinking, not what they politely say they are thinking but actually aren’t. Running anonymous turns first also allows people who would otherwise be dogmatic or inflexible to change or moderate their first opinions and to learn from the group without social cost. Establishing adequate trust in the facilitators and the process is necessary for the anonymity to have maximum productive effect. See Covey’s The Speed of Trust (2008) for more on trust. Using an anonymous remailer to accept feedback and encouraging folks to use non-identifying language in their anonymous feedback can help with their candor as well. Until trust is established, initial anonymous rounds may not improve internal feedback, but they will improve feedback from external participants who have no fear of repercussions to honesty. Coaching anonymous participants to be civil and constructive, or their comments will be deleted, may also be necessary to keep the feedback honest but helpful.
After anonymous turns of the cycle are concluded, a few public turns of the cycle may then follow, and eventually opinions will stop converging. In their classic 1962 paper introducing the method (PDF), Dalkey and Helmer at RAND used iterated anonymous Delphi to reduce estimation uncertainty in a “number of nuclear bombs needed” scenario, presented to experts, from a 10,000 percent range to a 200 percent range, an impressive fifty-fold reduction in uncertainty. In a more foresighted world, the number needed should have been zero, but that was the Cold War, and different thinking prevailed. Progress occurs in steps.
Cone of Uncertainty in Project Management
Scrum is a very powerful workflow ruleset based on small teams of five to nine people, working in two week sprints, each of which are quick-planned every two weeks. A simple version of Delphi is used in scrum to quickly converge on estimations of task difficulty or cost. Good quick estimation is one of the hardest problems in workflow planning. In Scrum (2014), Jeff Sutherland describes how the cone of uncertainty (picture below) in estimating project cost or completion difficulty, which is large even for estimating just two weeks ahead, can be greatly reduced by having the team that will work together rank their planned tasks by days, person-hours or dollars expected to be needed, using Fibonacci sequence cards (1,3,5,8,13). Any task expected to take 13 days is too close to the project deadline, and should be split into smaller subtasks, then reestimated by the group. Each team member does a quick simultaneous vote with their cards to estimate the difficulty of each proposed task, and the results are averaged. If any team members vote is more than three cards away from any other, all members who voted with the extreme cards must explain their opinions, and then the estimate is done again. Such a quick Delphi process allows reduction of the cone of uncertainty from the classic uncertainty in the earliest stages of many projects, with estimates commonly ranging from 0.25x to 4X the actual outcome, to a much smaller range, allowing the team to better prioritize and stage tasks. Sutherland offers it as one of the tools allowing scrum to deliver 200 to 400 percent more work, and far more accurate estimates of work, than conventional small-team workflow techniques. Skeptical? Try it and see, for your team.
Besides converging opinions, Delphi will also clarify them enough that one can do Argument and Assumption Mapping for the competing Schools of Thought within the group, if there is sufficient time and resources to do this work. Simple software to aid in this mapping, such as Araucaria, is now available. Different schools of thought may exist due to different values (normative differences), different assumptions, or different dominant models among those within the group. This mapping can be followed by a directed search by the group for Data to Resolve Disagreements, or Proposals for Experiments that would generate data that might resolve disagreements.
On controversial and complex issues (climate change, social policy, etc.) the best one may be able to do with these maps is to reduce and clarify the various schools of thought into a manageably small number (see example below, for economic thought), and then to imagine what kinds of data or experiments would tell you which school of thought you should agree with. Getting the map is the first great challenge. Developing a well articulated and referenced Schools of Thought map of probable, possible, and preferable futures, with differing values, assumptions, and models clarified, is in my estimation one of the end goals of a good Delphi.
A Map of Common Schools of Economic Thought, Associated Here With
Normative Differences in Political Philosophy (Bert Wheeler, Cedarville U.)
The learning that occurs from aggregating the opinions of any fixed group of Delphi participants will be on an S-curve, like other types of learning, with rapid increases in understanding in the first few cycles, but then usually quickly tailing off, unless you greatly increase the diversity or intellectual capacity of the group. Given the value of cognitive diversity, we can imagine doing a Delphi where 1) anyone on the web may participate, 2) it is marketed widely, both to experts and lay audiences, 3) incentives are provided to participate, 4) interaction is designed for ease of use, and 5) iteration continues until new learning plateaus would be likely to provide the greatest improvement in group foresight prior to saturation (reaching a learning plateau). A Delphi with these five characteristics might be called an Open Delphi. In 2012 Venessa Miemis, Alvis Brigis and I wrote a brief article proposing doing for this for all our better foresight methods, an approach we call Open Foresight (PDF). Few Delphis are conducted in an open manner today, but as web intelligence and access grows we should see many more in the future.
Adler & Ziglio (1995)
Publications that present complex, uncertain, and controversial topics in a Schools of Thought manner, showing the classic literature and evidence cited in support of each school are hard to write, and rare to find, but I find them particularly valuable for learning the current competitive and cooperative landscape of any complex system. One good example is the Hunger Project’s Ending Hunger, 1985, which explores the many blocking factors (business, politics, ideology, etc.) that have so far prevented us from ending malnutrition and starvation globally, even as the economic and technical means for doing so have existed since the Green Revolution of the 1940s-1960s. Multiple competing schools of thought and their justification are presented for each of the major uncertainties, claims, models, and hypotheses in the book.
Detailed schools of thought treatments belong best as online works, to be updated annually as new data come in to help us resolve competing claims and models. Eventually, as online tools emerge, we hope this Guide, leading textbooks and Wikipedia pages will all be written in this manner, and use open Delphis as inputs. For an older but still good introduction to Delphi techniques, try Adler and Ziglio’s Gazing into the Oracle, 1995. Garson’s The Delphi Method in Quantitative Research, 2013 is at the frontier of computer and quantitative approaches.
