Working In Uncertainty

Optimism, pessimism, and open-minded realism

Contents

There is an idea floating around that being an optimist is good for you. In her book, ‘Smile or Die’, Barbara Ehrenreich traces the origins of this idea, from early religious eccentrics to modern day psychology and motivational gurus. Related notions are that self belief is important to achievement, that confidence is needed, and you get what you expect. I freely admit that there was a time when I also thought that expectations influenced results, and sometimes they do.

However, since then I have examined this much more closely and changed my views. I now realise that there are downsides to optimism too, and that optimism and pessimism are not the only options. If you are interested in exploring this in more detail, especially if you have been uncomfortable with the self-delusion that positive thinking usually involves, then this article may provide a clearer understanding of a reasonable alternative that works and makes sense.

What is the choice?

Research has fairly consistently shown two biases in our predictions about the net benefits of actions we plan to take: (1) on average our predictions are usually too high, and (2) when asked to give ranges for our predictions our ranges tend to be too narrow.

This gives a useful insight into some familiar mental outlooks that we might try to adopt:

  • Pessimism: Expecting unrealistically low net benefits from courses of action (i.e. poor results achieved, if at all, after a struggle), and being unrealistically sure that those poor benefits will be achieved.

  • Optimism: Expecting unrealistically high net benefits from courses of action (i.e. great results easily achieved), and being unrealistically sure that those great benefits will be achieved.

Both of these involve overly narrow predictions. If we correct that by being more open-minded then other possible outlooks emerge. The one I will focus on in this article is this:

  • Open-minded realism: Having a view of the results of courses of action that is not biased towards high or low net benefits, and is open to a range of possibilities in a way that is, again, rational and unbiased, rather than overly narrow.

Here's a picture that shows each of these outlooks. The horizontal axis represents the outcome of some course of action, ranging from terrible on the left to great on the right. The height of the graphs represents the person's belief that each level of outcome will occur. Pessimism is narrow and negative, represented by the red distribution. Optimism is narrow and positive, represented by the green distribution. Open-minded realism, represented by the black curve, is unbiased and more spread, reflecting an open mind about what might happen.


Which is best?

A big problem with pessimism is that you don't want to do anything. Why bother? If forced to act, a person with a pessimistic outlook sees little point in trying to work out solutions to problems (because there won't be any solutions) and will be unprepared if things go well.

In contrast, a person with an optimistic outlook is keen to get started and has plans for what to do if things go well. That's a good thing. The problem is that the optimistic outlook does not encourage considering the possibility that things might go less well. If things start to go wrong a person with an optimistic outlook is often unprepared. Although habitual optimists may not be worried by this (because they always think things will turn out well) everyone else is suffering. Working for a persistently optimistic person can be very stressful.

So, although optimism seems to have some advantages over pessimism, it still has a worrying weakness.

People whose outlook is one of open-minded realism are also willing to act, when justified, because they can see that things could go well, perhaps very well. However, they are also willing to consider other possibilities and take precautions. By being open-minded (i.e. not making overly narrow predictions about results) we can solve the problem of optimism and remove the need to maintain unrealistic views.

Open-minded realism is a better outlook.

Another way of looking at this is to focus on the biases. There are two potential biases. One is a bias towards overly high or low predictions of results. The other is a bias towards overly narrow or overly wide predictions of results. If you think bias leads to wrong decisions, and I do, then the outlook with the least bias is the one to adopt.


What this analysis reveals is that it is not necessary to adopt an optimistic outlook to have enthusiasm, and it is not necessary to adopt pessimism to achieve a responsible level of control.

(Incidentally, if you want to explain this idea to a friend do not make the mistake of using an example where results are just divided into ‘success’ or ‘failure’. When results are simplified to just these two categories you cannot separate out the two biases and the explanation does not work. This is not a problem for open-minded realism, but just another consequence of seeing the world in black and white terms.)

Admittedly, biases like this are hard to notice and correct in yourself but (1) we can aspire to open-minded realism and at least avoid deliberately cultivating a resolutely biased view, (2) there are some techniques that can reduce or correct for biases, and (3) we can decide to act as if we face more problems and more uncertainty than we perceive, knowing that this is usually true.

Open-minded realism in action

You may be wondering ‘Is open-minded realism enough?’ or perhaps ‘Doesn't optimism provide stronger motivation?’ The following example illustrates the Achilles heel of optimism and the inherent strength of open-minded realism.

Imagine a tennis player starting a tough match. Does she need to believe she certainly will win in order to put in a full effort and play well? No. Logically, all that is necessary is to believe that the chance and value of victory justify the costs of continued effort. If the player remembers that anything can happen, that the gains of victory are large, and that the costs of making an effort are just pain and fatigue that will pass in a day or two, then high performance can continue.

What will happen if she does believe confidently that she will win, and also believes that this is necessary for her to perform well? Provided things go according to plan all will be well. However, if the other player gets ahead, or has a period of great play, this evidence will challenge our player's confident expectation of victory. To maintain this certainty she will have to do something to explain away the evidence, which will be distracting and create the internal conflict of deliberate self-delusion. This is the penalty for trying to hold on to an unrealistic belief.

In summary, open-minded realism provides reasonable motivation and is not so challenged by incoming evidence. Optimism requires repeated and distracting efforts to maintain unrealistic beliefs in the face of incoming evidence from experience. If Serena Williams needs to write reminders that she will win to be read in rest breaks, what hope is there for players who, objectively, have less chance of victory than her?

Robust confidence

This line of thinking can also be used to understand confidence.

To perform well it helps to be confident because this avoids distracting worries about what will happen, how well we are doing, and if we have the ability or talent to succeed in whatever way is of interest.

But confident of what? We can choose what we want to be confident of. If we choose to be confident of victory then evidence showing victory is not certain will cause distracting worries. If instead we choose to be confident that the result is not yet decided, that our opponents may still stumble, that we may be lucky, and that the costs of making an effort are not great, then this confidence will be easier to maintain, because it is the truth.

In a similar way, if we choose to be confident that our technique is as good as it can be then we will struggle to maintain that confidence because there's always something you can do to improve, in the longer run. What is more realistic is to be confident that there is nothing better we can do on this occasion, knowing that later we will examine our skills again and make more improvements.

I have noticed while watching televised sport that commentators often seem to think that competitors need to believe they will win, but the sports stars themselves usually talk differently.

To find an example of this I Googled ‘Lewis Hamilton’ (former Formula 1 world champion) and the first item to come up was an article in a newspaper building up to the 2010 Grand Prix in Monaco. The journalist wrote ‘...the 25-year-old is convinced with the McLaren he has beneath him, an elusive triumph to boost his championship chances is just over the horizon.’ What Lewis actually said was ‘I'm not going to get my hopes up because we know how competitive everyone is. We've just got to hope, keep our fingers crossed and do the best job we can. But a win here would be the beginning of hopefully something very special, although I won't apply pressure on myself and say it will come this weekend. My performances have been pretty good this year, so I just have to build on that, and eventually it has to hit a peak with a win.’ Note the word ‘eventually’. Objectively, this was a reasonable perspective at that stage of the season.

(Hamilton actually finished 5th at Monaco, but won the following two races in Turkey and Canada.)

Finally

Having a mental outlook free of both biases isn't just to avoid making bad decisions. With open-minded realism we have the motivations we need and can maintain confidence in beliefs that are reasonable, and won't wobble and distract us when events unfold.

These are good reasons for aspiring to open-minded realism in ourselves and for looking for it in others. If your life savings were invested in a company, what outlook would you like its chief executive to have?



Thank you

Grateful thanks to early readers of this article whose comments helped me improve it: Stuart Hartley of Focus ROI, Richard Karpinski who describes himself as Nitpicker Extraordinaire but is so much more than that, the well read Felix Kloman of Seawrack Press Inc., creative powerhouse Professor Michael Mainelli of Z/Yen, project expert Niels Malotaux, Dr Alasdair Marshall at the School of Management of the University of Southampton, Peter Sandman the risk communication guru (www.psandman.com), Jon Sharp the Head of Internal Audit UK at AgustaWestland, Clifford Shelley of Oxford Software Engineering, the inexplicably unpublished Martin Ternouth, Russell R. Vane III from a competitive games perspective, and Simon Walker of Detica. Obviously, all faults are mine.






Made in England

 

Words © 2010 Matthew Leitch. First published 14 May 2010.