Working In Uncertainty

A practical planning approach to military planning workshops

When Russ Vane III first explained Hypergame Expected Utility to me I must admit I was a bit distracted by the details of his calculations about uncertainty. Some years later he tried again, now able to explain what he had learned from conducting some 200 life-and-death military workshops using his technique over 48 months. Aha! Now it makes a lot of practical sense.

Many thanks to Russ for persisting and for helping me write this article about his approach.

This approach is a combination of a number of techniques and, depending on your problem, you might use some or all of these. In particular, there are two low technology techniques for getting people to think about what they don't know that I think can be used quite widely and are easy to understand. There are also some refinements that are harder to grasp and so I will mention them but not explain them fully.

One of the keys to Russ's approach is a summary table that holds nearly all your thinking in a structured way and (with some spreadsheet power) does useful calculations as you work. With this table in place your thinking remains organized even though the order in which ideas emerge is more flexible and opportunistic. I'll explain what the summary table looks like later on, once I've introduced the elements of it.

The end result is much more than a plan. The result that really counts is that the team develops a greater understanding of what they don't know, how much it matters, what they might see in future that indicates they are facing one of the possible situations they considered, and what they can do in response. The team is mentally prepared.

This approach evolved out of military war gaming but has a lot of features of scenario planning. However, don't imagine that it only applies to the big picture. Clearly, mental preparation is more important when events can move quickly and involve the individuals who participated in the planning meeting. Russ has done this where the scope is narrow and short term as well as where the scope is bigger and more long term.

Games with an adversary

Russ Vane has been playing strategy games since his tenth birthday in 1963 and his approach today has its origins in the most basic ideas of game theory. This is the sort of diagram you will see in every game theory textbook and it is called the 'normal form' of a game between two adversaries, called player A and player B:

In this example player A has two actions available, A1 and A2, and player B has three actions available, B1, B2, and B3. The idea is that in the battle they will each have to choose an action for themselves without knowing what the other player is going to choose. When the battle starts they will each make their move, still not knowing until it is perhaps too late, what their opponent has chosen to do.

The pairs of numbers in the cells of the table, separated with commas, represent the value of the outcome of the battle to each player (player A, then player B). For example, if player A does A1 and player B does B1 then the payoff to player A will be 1 and the payoff to player B will be 2.

Non-adversarial situations

Similar tables can also be drawn even if there is no enemy. For example, you could replace the enemy's actions by alternative situations, as yet unknown, such as how much customers will like a new product, or whether a lover really wants to get married. There is no need to show the payoff to an 'enemy' in this case because there isn't one. However, we can add information about how likely each situation is.

For example, suppose we can launch a new product with cheap advertising or expensive advertising, and customers could like it a lot, a bit, or not at all. The numbers in the cells represent estimated profit for each case.

Improving on the basic normal form

Russ's ideas enhance this simple analysis in a number of ways, including these:

• Developing the situations and actions in a way that challenges planners to think widely about what might happen and what they could do.

• Estimating the outcomes and their value.

• Putting thinking behind the situations that makes it easier to come up with probabilities for each situation and relate these back to relevant evidence, including evidence that might appear later.

• Prompting a search for more information.

• Choosing between alternative actions when it's not obvious which is the best.

'Descending the diagonal' to identify situations and actions

The first core phase explores situations and courses of action. It is called 'descending the diagonal' because that's what the documentation looks like. Later in the process the documentation will be a table where:

• the row headings are brief statements of alternative courses of action;
• the column headings are brief statements of situations that might turn out to be true; and
• the remaining cells record how effective each course of action would be in each situation.

However, before we can get to that stage we need to generate the ideas for courses of action and for situations, and that's what 'descending the diagonal' is for.

Russ, as lead facilitator, usually starts with one situation and one course of action, typed into the first column heading and the first row heading respectively. The starting situation is usually just the current best guess. In a military planning meeting that might be a combination of best guesses about the strength of the enemy, their location, the level of civilian support they have, and so on. The course of action might be the current favourite or nothing more than a straw man to get things started.

An initial assessment of the outcome that would be expected from using that course of action in that situation is entered on the table in a simple way. It might be with plus and minus signs, or with a score from +5 (complete victory) to -5 (complete defeat).

Now something interesting happens. The facilitator (or perhaps someone appointed to act as opposition) describes a new situation and writes it into the next column heading. This situation needs to be different enough to stimulate a new course of action. The situation can be better or worse, and it can be extreme or not.

The outcome of using the first course of action in the new situation is estimated and written on the table. Usually the outcome is worse than before because the situation has changed.

The team now thinks of at least one new course of action that will work in the second situation. (It's ok to have more than one answer.) This is written as the second row heading and the estimated outcome is again written into the matrix. Typically, this estimated outcome will be better than the outcome for the first course of action in the second situation because, of course, it has been designed to do better in that situation.

The facilitator keeps on injecting new situations until the team begins to run out of new ideas for courses of action and a situation is suggested to which the suggested course of action is the first one suggested, at the start of the meeting. This seems to provide a natural closure point.

Where do the ideas for new situations come from? In practice these usually come from the planning team because, perhaps unwittingly, they usually give away assumptions they have made. For example, "Assuming they don't have any heavy weapons in that area..." is an invitation for the facilitator to propose a situation where the enemy does have heavy weapons in the area.

An important requirement on situations is that they should not overlap with each other. Another is that, collectively, they should cover all feasible possibilities, or very nearly.

When this first core phase is over the table looks something like this:

There is no guarantee that every important variation in situations will be explored, or that the courses of action will include the best possible. Nevertheless, this procedure usually does quite well and can be used to get people to think about more possible futures than they might otherwise have coped with considering.

Estimating payoffs

Having made rough estimates of the payoff for some combinations of action and situation during the 'descending the diagonal' exercise, the next task is to estimate the payoffs for all combinations of situation and action, with a bit more care.

To do this the planning team establishes a set of 'performance characteristics', which are measures of different aspects of outcomes, and a mapping from these to an overall utility scale, ranging from +5 for a great victory down to -5 for a heavy defeat.

If this sounds like a lot of thinking, bear in mind that since 2007 Russ Vane has been employed by IBM to facilitate these meetings with clients who care about their outcomes. They care because one of the main performance characteristics in the analyses is usually the number of people killed.

The performance criteria and utility mapping are then applied to each combination of situation and action. Sometimes, doing this prompts people to think of additional performance characteristics. This might happen because, for example, two courses of action lead to outcomes people see as different but the existing performance characteristics do not capture the difference.

Having worked through every cell in the table it might look something like this:

You may have noticed that CoA 7 has disappeared. That's because it is materially similar to CoA 1 so there's no need to repeat it.

Using contexts to put thinking behind situations

The approach Russ uses involves assigning a probability to each situation, but in two stages. It recognizes that the situations that emerge in planning meetings can be understood more easily by noticing that they are driven by broader contexts. For example, the situation 'the enemy has twice as many fighters in the area as currently estimated' is more likely if the context is that 'the enemy sees controlling the area as its main priority'.

So, his approach is to list mutually exclusive contexts. For each context, a probability is assigned to each situation assuming that the context is true. These probabilities, for each context, add up to one.

Planners are encouraged to consider if the context suggests additional situations that have not previously been identified. If so, these need to be added to the summary tables.

Each context is then assigned a probability of being true, and these too should add up to one, provided the contexts are mutually exclusive and cover all feasible alternatives.

The probability of each situation can then be calculated. It is the sum of the products of the context probability times the probability of each situation given each context. On Excel this is easily done with the sumproduct( ) function.

At this point the summary table has all the main elements of the hypergame normal form:

In this illustration the bold black text is just part of the template. The blue numbers are calculated automatically within the table. Everything else is either typed in directly or pulled from somewhere else. 'NC' means not calculated in this illustration, but if needed it would be. Here's an explanation of the format, chunk by chunk:

• The table of actions and situations shown earlier now appears in the bottom right of this full format.

• Above it sits another table, this time with contexts as row headings, where the numbers are conditional probabilities of each situation given each context. The conditional probabilities for each context should sum to one, consistent with the situations being exhaustive and mutually exclusive.

• The numbers under the heading 'Belief' represent the probability of each context being the true one. Again, these should sum to one.

• The top row of the table is the probability of each situation being the true one, calculated from the beliefs in each context and the conditional probabilities of each context.

• The item immediately below C1 labelled C0(GT) is a refinement that may be helpful in adversarial situations. It is a context that represents the game theoretic solution to the game and is usually a 'mixed strategy', meaning that it is a probability weighted selection from the other contexts.

• The numbers under 'Expected Utility' are the expected utility of each course of action, calculated from the probabilities of each situation and the payoffs in the table to the right. Again, Excel's sumproduct( ) function is your best friend.

• The numbers under the headings 'Best Case' and 'Worst Case' are just the best and worst payoffs for each course of action.

Warnings and indicators – guiding the search for more information

When thinking about situations and contexts the planning team is encouraged to think of evidence that might exist, perhaps in the future, that would suggest the situation or context is more or less likely to be true. Russ uses the word 'indicator' for evidence on situations, and the word 'warning' for evidence on contexts.

Choosing the best course of action when it's not obvious

At this stage we have a table showing an estimate of the value of the outcome for each course of action in each situation. This is a classic 'decision making under uncertainty' problem and there are several well known approaches to choosing a best course of action. Some of these look at the best and/or worst outcomes in any situation for each course of action, regardless of how likely each situation is. Others assign a probability to each situation and use that in some way to weight the situations differently.

Russ creates a graph of the expected value, worst case, and best case, for each course of action and allows people to choose. This reflects the fact that the probabilities used are of unknown quality. It may be that the context is one that the planners think is very unlikely.

Final comments

This has been just an overview of the less technical aspects of the approach involved. In particular, the 'descending the diagonal' exercise and the exercise of thinking of contexts and relating them to situations are useful tricks to know and do not require a deep understanding of game theory.

(Sources of information: A presentation by Russ Vane in June 2011 at Tom Gilb's conference in London, and personal communications with Russ. Also the following papers:

Russell R. Vane III, (2000), Hypergame theory for DTGT agents, Papers from 2001 AAAI Spring Symposium. Available here: http://www.aaai.org/Papers/Symposia/Spring/2001/SS-01-03/SS01-03-024.pdf

Russell R. Vane III, (2005), Planning for Terrorist-Caused Emergencies, Proceedings of the 2005 Winter Simulation Conference, M. E. Kuhl, N. M. Steiger, F. B. Armstrong, and J. A. Joines, eds.

Russell R. Vane III, Advances in Hypergame Theory available at http://www.sci.brooklyn.cuny.edu/~parsons/events/gtdt/gtdt06/vane.pdf

Made in England