Working In Uncertainty

A first step towards successful risk management standards

by Matthew Leitch, first published 6 March 2007.

Broadening the support for risk management standards

There is one simple step that would dramatically broaden support for risk management standards and similar authorative guides such as these:

• ‘Enterprise Risk Management – Integrated Framework’ published by COSO;

• ‘A risk management standard’ published jointly by IRM, AIRMIC, and ALARM;

• ‘RAMP: Risk Analysis and Management for Projects’ published by the Institute of Actuaries and Institution of Civil Engineers;

• AS/NZS 4360:2004 Risk Management;

• ‘PD ISO/IEC Guide 73:2002 Risk management. Vocabulary. Guidelines for use in standards";

• ‘The Orange Book: Management of Risk’ published by the UK government; and

• numerous other guides produced to promote the ideas contained in these.

That simple step is for them to acknowledge the existence and value of more sophisticated risk management practices than they currently describe, making them acceptable alternative methods of compliance. In this article I will explain how easily this could be done, usually with only small revisions.

Why this would help

Currently, the main standards and similar authoritive guides are inconsistent with the thinking behind more mathematically oriented approaches to risk. Consequently they exclude support from financial risk managers, operational risk managers in larger banks, actuaries, decision scientists, scientists looking at food and other chemical safety issues, statisticians, and mathematicians.

This means that the world's most knowledgeable risk specialists think in ways that are not consistent with the standards.

It also means that an organisation interested in moving towards more sophisticated approaches can find it would become non-compliant with the standards as a result, discouraging progress and innovation.

If the existing standards were amended to just acknowledge the existence of more sophisticated approaches so that they were also considered compliant then the standards would gain more support and cease discouraging improvement.

Where the standards need changes

Changes are needed in the following areas:

• Risk identification: Currently standards are usually written as if risks exist and just have to be spotted and named. However, in much risk modelling it is the whole system that is modelled, with the ‘risks’ being derived from and defined by the model. Often, each uncertain variable in the model is a risk.

• Risk ratings: Most standards currently describe a process of considering probability (also referred to as likelihood) and impact (also known as loss). The implication is that these are considered as separate assessments, as in the widely used probability and impact matrices. The usual mathematical approach is to consider the probability distribution of impact, perhaps derived from other curves. In this approach it is not possible to consider probability separately from impact level.

• Risk appetite: Currently the standards are usually written with the idea that an organisation has an amount of risk that it will tolerate and decisions on whether or not to act on a risk are made on the basis of whether or not the risk is beyond the risk appetite. In constrast, the established tradition in decision analysis is to take actions if their net effect is beneficial, subject to resource constraints and uncertainty about those benefits.

What changes are needed and why

To illustrate the simplicity of the changes needed to bring about this minor revolution here's how key phrases in some well known standards could be amended.

Risk identification

Most standards refer to ‘identifying’ risks without mentioning others ways to create a sensible set of risks. This does not encompass all the good practices currently in use around the world. These include, for example:

• Workshops where people simply suggest ‘risks’ and these are added to a list, usually with no organisation beyond mapping to objectives or some general categories.

• More systematic processes where some kind of non-mathematical model of the system/plan/business/etc is created and then risks are derived by a rule from the model. For example, a project might have a number of steps and failure at each step could be considered a risk.

• Mathematical modelling where a system is represented as a system of variables, some or all of which are random to some extent. Some or all of these random variables are then classed as risks, or their distributions are used to calculate risk statistics. For example, a financial risk manager might have a model of the investments in a portfolio that gives a distribution for returns over different time periods. The portfolio return takes the place of a ‘risk’ and statistics about it provide the risk assessment. A more elaborate model might include events that drive the investment values, each of which is random and considered an additional ‘risk.’

Only the first of these examples easily fits the description ‘risk identification.’ The others create a view of risks as a by-product of a wider analysis.

The more inclusive wording I suggest is ‘risk identification or derivation.’ For example, in ‘A Risk Management Standard’, published by AIRMIC, IRM, and ALARM jointly, the text often mentions risk identification. For example, it includes this statement:

‘Risk identification sets out to identify an organisation's exposure to uncertainty.’

This would be better as:

‘Risk identification/derivation sets out to identify, or derive a view of, the organisation's exposure to uncertainty.’

(Arguably the authors' real intention is more derivation than identification as this sentence from the standard implies:

‘Risk identification should be approached in a methodical way to ensure that all significant activities within the organisation have been identified and all the risks flowing from these activities defined.’)

Risk ratings

Most standards refer to considering the ‘probability and impact’ of risks. This wording naturally fits techniques where a rating for probability is given, and a separate rating for impact. However, it does not naturally fit techniques where a probability distribution of impact is used, or some set of distributions that are or could be combined to create a probability distribution of impact or approximation to it.

This is a pity since almost nobody argues that separate probability and impact ratings are superior to the more sophisticated mathematical approaches. The argument in favour of Probability Impact grids is nearly always that they are familiar and convenient, whereas doing something more rigorous would be hard.

The alternative wording I suggest for this is to say ‘probability distribution of impact, or some simplification of it.’

For example, COSO's ERM framework includes, in the executive summary, the statement:

‘Risks are analyzed, considering likelihood and impact, as a basis for determining how they should be managed.’

This would be better as:

‘Risks are analyzed, considering the probability distribution of impact or some simplification of it, as a basis for determining how they should be managed.’

Risk appetite

Most of the standards refer to deciding some kind of ‘risk appetite’, usually in the form of a level of risk beyond which things are not acceptable and something must be done. Quite often this is applied risk by risk.

A more sophisticated alternative that is used by many is to consider the net benefit of alternative courses of action and choose the best, having made some kind of allowance for risk. In this kind of analysis there is no set level of risk that is tolerable, but there is usually some way that risky courses of action are penalised for the risk they carry.

The wording change I suggest is to replace ‘risk appetite’ with ‘risk appetite or other means of weighing risk in decisions.’ For example, the UK government has produced several guides to risk management, one of which is called ‘The Orange Book: Management of Risk". It includes the statement:

‘An important issue in considering response to risk is the identification of the “risk appetite” of the organisation.’

This would be better as:

‘An important issue in considering response to risk is the identification of the “risk appetite” of the organisation or another means of weighing risk in decisions.’

What should be done to each standard

Some standards need less amendment than others. Here is a table of some of the most frequently cited standards showing what would be worth doing to each to implement the above suggested changes. Where I have written ‘Substitute phrases throughout’ this means use the suggested replacement phrases and should be taken to imply some editing to correct the English. Often there is a need for some other small word changes so that a standard/guide does not put forward one crude method as if it is the only possible good practice.

Other helpful changes

The phrase substitutions and related tweaks described above would be a great step forward because at last the best methods would be allowed into consideration.

However, there are many other changes that would be helpful. In time it would be good to see more positive statements about more sophisticated methods and even some discouragement of the crudest ideas, in particular the probability × impact matrices.

It would also be a great step forward to include more material about the design of controls, such as how to go about managing and doing the design work. Currently this takes up a lot of time for people but gets little or no coverage in standards and guides.

Final words

Imagine that the board of a television company looked at viewing figures and then issued the following instruction to its employees: ‘We have noticed that many people watch TV shows that feature celebrities and telephone voting. Therefore, henceforth all TV shows we produce and show will feature celebrities and telephone voting.’

What a frightening thought! But this is analogous to the situation we have with standards for risk management. Because one rather crude approach is popular the standards are written so that only that approach is consistent with them.

The first step out of this is for the standards to acknowledge the existence of more sophisticated alternatives and this can be done very simply using the substitute phrases described in this article. In time this could open the way for more thorough reforms.

Further reading

When the British Standards Institute began to test interest in a new standard they held events to gather views. I was one of the presenters at those events and outlined some areas that existing standards tended to cover poorly in"Problem areas for current risk management standards.’

The familiar model of risk identification, rating, and risk appetite comparisons has at its heart a single mistake I call the Single Risk Fallacy. This is the belief that the items on a risk register are single things that exist already and are a property of the world. The logic and implications are discussed in ‘What's on your risk registers?’

For a flavour of just some of the alternative ways of thinking about and managing risk try ‘Risk modelling alternatives for risk registers.’ This article contains helpful ideas on how to decide which methods are right for you.

Words © 2007 Matthew Leitch. First published 6 March 2007.