Working In Uncertainty

Results of a survey on 'risk management'

Many thanks to everyone who participated in this survey. Once again, without your generous donations of time and thought this would have been impossible.

Summary

Is it right to define the phrase 'risk management' in risk-listing terms? In other words, do most people agree that 'risk management' should mean, by definition, the method of managing risk that involves making lists of 'risks' and managing the 'risks' on the lists (Leitch, 2012a)?

No, they do not. The results of this survey show that a clear majority of people think that the phrase 'risk management' should just mean managing risk and should not imply a particular method. They do not agree that 'risk management' should mean specifically the risk-listing approach. This was true for the respondents as a whole but there was a clear difference between respondents whose first language is English and other respondents, with the native English speakers most often wanting the more open definition of 'risk management'.

This finding may come as a surprise to many readers because most respondents thought other people define 'risk management' specifically as risk-listing. Interestingly, people who thought that 'risk management' should mean just risk-listing were much more likely to realise that most other people think differently. Also, people whose knowledge on this point was informed by systematic research were more likely to think that most people prefer the open definition.

Most people also thought that publications tended to define 'risk management' as risk-listing and those whose knowledge was based on systematic research were more likely to think this.

These results have profound implications for anyone producing guidance, standards, or regulations relating to 'risk management' because there is a danger of defining 'risk management' in risk-listing terms without realising that there are alternatives, including better, more comprehensive alternatives. Risk-listing is not a generic approach to risk management.

A note on the statistics and graphs

Most of the graphs in this report look like a dark red bar chart with a grey overlay at the end. This overlay represents uncertainty about the true value in the population, based on the sample evidence, using Bayesian methods.

The dark red part of the bar shows the range up to the 10th percentile of a probability distribution representing belief about the true percentage or average of the entire population (i.e. people with an interest in management). The pink part shows the range from the 10th to the 50th percentile. The light grey part shows the range from the 50th to the 90th percentile. In effect, the graph says that, given various assumptions (as usual in statistics), we can be 80% certain that the truth lies within the grey overlay area (i.e. pink and grey segments combined).

In all cases the initial prior distribution was uniform. For questions requiring a choice between two answers the prior distribution was the beta distribution. For questions requiring a choice between more than two answers the prior distribution was the Dirichlet distribution.

Where percentages do not add up to exactly 100% this is, of course, due to rounding.

The survey questions and answers

In this section the text in italics is the text used in the survey itself. Other text discusses the results and their implications.

The first two questions in the survey were about the first language and skills of respondents. These are given and discussed in a later section. The survey then continued as follows.

3. First, which of the the following best expresses what you think the phrase 'risk management' should mean?

Implications: Clearly, most people think 'risk management' should mean just managing risk, without implying any particular method.

To explore the factors that might be linked to this view the job roles and whether English was a first language were transformed into binary variables and regressed against the answers to question 3. This showed that having English as a first language was the strongest predictor, by far, and was associated with preference for the open definition that does not imply a particular method. A secondary finding was that being an auditor was associated with thinking that 'risk management' should imply risk-listing by definition.

Perhaps a strong grasp of the principles of English language provides protection against publications advocating a risk-listing definition.

4. Now, which of the following best expresses what you think the phrase 'risk management' most often means to other people?

Implications: In contrast to the results of the previous question, most people think that other people think 'risk management' refers specifically to risk-listing. Either we don't realise that others are, mostly, as enlightened as we are, or we distinguish between what we think the phrase should mean and what it usually means.

However, respondents who thought 'risk management' should imply risk-listing were much less likely to think that others agreed. Only 30% of respondents who thought 'risk management' should imply risk-listing thought that others used this definition, compared to 78% for respondents who did not think 'risk management' should imply risk-listing.

Perhaps people who argue for risk-listing provoke a reaction that other people rarely see.

5. How confident were you in answering the previous question?

Implications: The results from the previous two questions suggest that most people do not realise that other people have the same preference for the open definition that they have. This should be, in some way, related to the level of information informing their judgements.

As can be seen from the above table, respondents with all levels of evidence tended to misread the views of others. (Remember that 72% of respondents in this survey thought 'risk management' should mean managing risk with no method implied.) Those using survey evidence underestimated least often but having some knowledge from conversations and reading seemed to create the false impression that others are thinking of risk-listing - a false impression that people who said they just guessed did not suffer from.

6. And finally, which of the following best expresses how 'risk management' is most often defined in publications?

Implications: Again, most respondents thought that publications tended to favour the risk-listing definition.

7. How confident were you in answering the previous question?

Implications: Again, the answers should be related to the level of evidence used by the respondent.

Here the pattern seems to be that the more evidence the respondent had the more often they thought publications tended to define 'risk management' as risk-listing.

What is the truth? What do publications that define 'risk management' usually say it means? The truth is hard to know because the most obvious method of research, which is to search the internet, provides a strong bias towards documents published in the last twenty years or so and where the phrase 'risk management' appears in the title, which is typical of risk-listing documents.

However, to give you an indication of the extent to which influential documents currently define 'risk management' in risk-listing terms despite the common view of English speaking people that this is wrong, here are some examples:

Also, on 6 January 2014 I used Google to search for: definition "risk management". All the first 10 documents presented included a definition of 'risk management' and 8 of these were as risk-listing. This is a situation that should be corrected.

The wider situation, including influential documents that do not come up on Google searches for 'risk management' is rather different, as documented in Leitch (2012b).

8. This box is for any comments, explanations, or suggestions you would like to make. Include your email address if you would like me to respond.

Comments received (other than friendly greetings) were as follows:

"I wouldn't be surprised if most people answer Q3 and Q4 the same way as I do. But what I observe is that when companies create risk management processes they are almost invariably about making lists and (perhaps) managing them. In other words, there is a distinction between risk management as a concept and risk management as an organised activity." (The respondent was right that others would tend to give the same answers to questions 3 and 4 i.e. open and risk-listing respectively.)

"Definitions of risk management are generally too specific. Work by Mike Power and also Felix Kloman may be of interest, if you have not come across them before."

"In my experience and reading, risk itself is often poorly defined. Although 'management' often incorporates a number of elements any 'guidance' is usually qualitative in nature utilising a combination of PIM and lists. It is rare to find evidence of quantitative models and the use of probabilities/distributions."

"For me Risk Management is optimising long term decisions, actions and choices whilst considering unknowns, risks, ambiguities etc. There are some words that go along with Risk Management that are part of the approach if not what it is: Balanced, Strategic, Open Minded (one of yours), Considered, Use numbers and calculations but acknowledge importance of feelings and judgements, Humbleness when considering own ability to foresee & influence the future, Pragmatic, Creative."

"Really interesting question. Are you managing risk when you remember the Green Cross Code when crossing the road? Yes, but do people associate that everyday decision with risk management? Probably not. Has risk management tried too hard to become a discipline/science in its own right that just means people only think about the process and worry about the potential for it becoming a cottage industry? My organisation struggles with, on the one hand wanting to manage risk, while on the other worrying about the bureaucracy of 'risk management', and also not knowing what type of 'risk management' would actually work. Also - 'in publications' is a bit broad, do you mean specialist publications, or do you mean in all publications, including daily newspapers etc?"

"Q3 The way you phrase the question and options rather betrays your dislike of lists of risks but I don't think that they are unhelpful. You need to document what you are up to somewhere and review progress and state etc. Blinkered by them and totally without them are equally bad. Q4 Either because they know no more than that or because they recognize Q3 above, so ideally result needs caveat or clarification. Q5 and Q7 options don't really allow for extensive experience in this field! Q6 The options don't really allow for the oft reality of 'badly described, randomly described and confusion encouraged'. Option selected was selected for this reality." (This respondent answered Q3 and Q4 in the usual way, suggesting that extensive experience does not help with knowing what others think.)

"I believe that risk registers (RRs), as currently used, lack rigour. But they are simple and appealing, especially to supposedly 'practical' people. They work in Excel. When we come to the literature the coverage is much more mixed. As far as I can see although RRs have found a 'best practice' role this did not come directly from the old COSO work in the 1990s. RRs are beloved of people who want to sell software and advise on processes and auditors who use them to do 'risk-based' audits. My personal view is that RRs can play a useful organisational role - the list! They are *much* weaker when reliance is placed on the probability-impact method of risk assessment c.f. Hubbard's The failure of risk management and many other sources. Currently RRs tend to equate risks to events. Worse, different 'risk owners' assess the probabilities of these events at different points on the (underlying probability) curve, especially in the absence of guidance. IMHO the only hope of saving risk registers is to have an assessment process which is (a) complete, (b) internally consistent and (c) able to be recalibrated without asking assessors for another lot of work."

"People not involved in the risk and internal audit profession would favour generally understood definitions which means something like 'bad things could happen that would hurt you'. Regulators have pushed rules that increasingly cause people to associate it with risk registers. What internal auditors mean when they say 'risk-based' plan may be the least precise of all."

"Our problem is that the word 'risk' itself means many different things to different people. How can we possible 'manage' something we misunderstand?"

"Risk management means quite different things to professionals in different venues, which makes it difficult to answer these cross-venue questions coherently."

"I would have thought that the value in managing risk is in our ability to understand measure, treat and monitor the causes and effects of attaining business outcomes, at variance to expectation."

"In answering your survey I have assumed you are referring to the systematic management of risk. I would argue that a lot of people, as individuals, are managing risks in a less than systematic fashion. The approach involving making-lists and managing-those-listed-risks is probably better-considered than raw pattern-recognition-and-reaction. Both are sub-optimal in their own ways."

"In question 4 I interpret 'Other people' as ordinary people, who have no familiarity in the subject area."

"My context deals primarily with project management and risks on projects."

"Half the time, I seem to find that 'risk management' means 'management of a portfolio of insurance contracts'. But then, I live in North America."

"Q6 - it depends on the publications you read so difficult to answer in general. Could be any one of 3 options given."

"Risk management is important. It comes more naturally to people who are more proactive. Its application/impact changes with context (people, situation, level of knowledge, etc). It is also misunderstood/misapplied/undervalued due to lack of a better mental model/awareness. It depends on a good/healthy management (maturity/paradigm). One perfect area to make a significant change is for HR to focus on managers' failure modes and effects and how to mitigate/provide contingency to those i.e. how to grow people."

"The term risk management has come to stay but beyond the terminology, the profession is looking at the core puzzle of humanity: how to live a happy life in a world of uncertainties. Organizations are faced with the same challenge. Mathematical models, psychological/cultural considerations and scenario planning are all good ways of forging a way through the unknown. The debate is focused on whether to go strictly mathematical or strictly risk listing or working in uncertainty. Working in uncertainty seems the best option to me, however, there is a nuance. Beyond the right mind-set and method to tackle uncertainties, much focus should be placed on strengthening the expertise of the risk manager and creating an environment fostering genuine human dignity in organizations. The risk manager is to the organization what the general practitioner is to the human body. The risk manager's training should be extended in years to incorporate in-depth knowledge about key business units that characterize every organization (Finance, HR, Operations, Production, Legal, etc). Then, roundtable discussions about ways to effect incremental changes in organizations will be more effective. The second key requirement rests on genuine human dignity. Creating an environment where every single person in the organization feels they matter and that without them the organization is not complete. Beyond monetary reward, a smile, a genuine concern about an employee's working conditions and family well-being can foster better contributions of employees at roundtable discussions regarding the business uncertainties. Finally, a company that breaks the traditional top down management process can impact considerably on risk attitudes. An environment where the employee can feel a questionnaire about his line manager will create an atmosphere of flexibility where one knows that his ideas will not cause him to lose his job but will be assessed critically. The open-mindedness culture of uncertainty management will be given life."

Respondents

Respondents were recruited by direct email requests to about 800 people who have shown an interest in my work over the years.

The survey was completed by 96 people.

1. Your first language:

Most respondents had English as a first language:

2. Do you consider yourself to be a professional in any of these? (click all that apply)

Limitations of the survey

The survey suffered from a number of typical limitations. In particular, respondents were self-selected and from a group that is not entirely representative of people with an interest in management. They were people with an interest in risk who had shown a specific interest in my work. That makes them, to some extent, a special group. However, the analysis of different respondents in the survey shows that even the group most oriented towards risk-listing methods (the auditors) overwhelmingly preferred to define 'risk management' as management of risk without implying a particular method. Previous studies using this same pool of respondents have given similar results.

Wider implications

The definition of 'risk management' is not just a matter of pedantry. If 'risk management' is defined as risk-listing then it leaves no room for other methods. Risk-listing is like a cuckoo chick, ejecting the legitimate occupants from the nest to favour itself alone. Using 'risk management' in this way is an unfair and unethical sale trick, typical of consulting sales pitches. Offer someone 'risk management' and it is hard for them to refuse it without seeming to reject responsible management. Offer them 'risk-listing' and they can reject it and ask for a better way to manage risk.

The fair way to manage this situation is to define 'risk management' in an open way and let proponents of alternative methods for risk management offer their ideas under names that suggest a method rather than an aspiration. For example, they can offer 'risk-listing', 'decision trees', 'probabilistic forecasting models', 'scenario planning', and so on, but not 'risk management' per se.

All regulators, standard writers, and other influential authors should review their current practice and, if they find that risk-listing has insinuated itself into their definition of 'risk management', they should put an open definition in place instead. This can be done by simply defining 'risk management' as 'managing risk'. They might also like to point out, in a note perhaps, that risk management is usually one outcome of managing well, and that many different approaches may be applicable.

References

Leitch, M. (2012a). The risk-listing school. Available online at: http://www.workinginuncertainty.co.uk/risk_listing.shtml.

Leitch, M. (2012b). Relevant authoritative guidance. Available online at: http://www.workinginuncertainty.co.uk/authoritative.shtml.