Working In Uncertainty
The psychology of devising internal controls: Building controls design skills effectively
by Matthew Leitch; first appeared on www.irmi.com in January 2007.
A director of a new client recently said to me ‘What I've seen a lot of is where we do plenty of risk analysis up front but then don't do much about it. Can you shed any light on that?’
There could be several explanations but a likely one is that people are not devising efficient, effective controls in response to risks they see. Either they are not thinking of anything at all or what they think of is inefficient, hard to set up, or just too vague for anyone to want to act.
Devising good controls is something many people find difficult. It's much easier to tinker with the risk analysis than to come up with good controls in response to it. Obvious examples are in IT security, where effective action usually relies on being able to buy a service or tool that will provide the control needed. However, even apparently simple requirements about reviewing accounting information or doing reconciliations can go unfulfilled due to lack of ideas good enough to be worth implementing.
The polite way out is to agree that a risk is ‘within our risk appetite’ and do nothing. The right way forward is to get better at thinking of efficient controls.
This article is about devising internal controls. What kind of activity is it? How can we help people do it better and more easily?
What kind of activity is it?
The psychology of thinking can help us understand this activity in two ways. The first is by helping us identify what kind of activity it is.
Is it, perhaps, a decision making activity? Certainly there are decisions to be made. However, to be a pure decision making activity all the actions available need to have been identified already, in sufficient detail that no invention is required. In most situations where a system of controls has to be devised the options are not known in advance, other than in some trivial and abstract sense. For example, you could decide to ‘treat’ a risk but this is not a defined action, just a general direction, and it leaves a lot of work still to be done.
Is it more of a problem solving activity? Now we're getting closer, but still this does not quite describe the thinking required. Activities studied by psychologists under the heading of ‘problem solving’ often involve a large but well defined ‘problem space’. What this means is that all the rules, constraints, goals, and elementary operations are known in advance, but there remains the problem of putting the elementary operations together, and in the right sequence. Examples of activities with this kind of problem space are chess playing and the Tower of Hanoi problem.
Devising internal controls does not match this description because there are no hard and fast rules and much is unknown at the start.
Furthermore, problem solving often means a lot of thinking to arrive at a simple conclusion, such as E = mc2. What is needed when we devise internal controls is a much longer description of one or more controls, including details.
Devising internal controls is a design activity. It involves complex but often unclear requirements, constraints, and potential elementary actions and decisions, and results in a large and detailed output – the design.
Controls design is complex too
A crucial feature of controls design is that it is complex. Rarely can one control be designed for each risk. Instead, a system of controls maps in a many-to-many way to a breakdown of risks. The controls that are designed need to fit in with existing systems, the constraints of human error, timing requirements, economics, cultural issues, and so on and on. Think of it as similar to architecture or the design of IT systems.
How do humans design?
The second way that psychology can help us here is in understanding how humans are able to do this kind of design work. It used to be thought that the key to successful design was a magic ability called creativity. This was something you fed with information, then left to ‘incubate’, perhaps while you slept or smoked opium, and which finally pushed up a magic solution from the unconscious.
Not so. Top designers are clear thinking intellectuals, not substance abusing poets. Creativity in design activities has turned out to be largely a myth.
The next idea was that design was powered by a design method. Various theories depicted design as a series of stages, such as ‘analysis’ or ‘constraint identification’. People constructed ever more elaborate recommendations for such design methods. In the world of computing, this became known as the ‘methodologies’ movement and methodologies still persist today, but controversially.
In 1970, J Christopher Jones's book ‘Design Methods’ described dozens of design methods culled from different disciplines. He hoped his design methods movement would spark a surge of interest in design methods and better design as a result. It didn't.
We now know that design methods, while important, are only a tiny fraction of the total knowledge required for skilled design.
The new thinking arose from the early work using computers to try to simulate human thought. This helped clarify what it would really take to solve problems, make decisions, plan, and design. These activities turned out to be much more complicated than expected and also helped people see that intelligent behaviour relied heavily on knowledge.
In ‘The Sciences of the Artificial’ (first published 1969), Nobel Prize winning economist and seminal psychologist Herbert A Simon crystallised much of the new thinking. Computerised ‘Expert Systems’ were developed, many using an idea called a ‘production system.’ A production system is a collection of rules, each one having a ‘condition’ part that evaluates to true or false in the current situation, and an ‘action’ part that can be executed if the condition is true. Production systems also need a process to identify all rules whose condition is true and decide which to execute, or in what order.
In many experiments, production systems seemed to match actual human behaviour quite well. In design tasks, the ‘action’ part is like the solution to a problem, while the ‘condition’ part says when that solution is useful. Solutions can be combined.
Separately, in the late 1970s, architect and academic Christopher Alexander began writing about ‘design patterns’, which were a way of writing down design knowledge that strongly resembled, but improved on, condition-action rules. He called a system of design patterns a ‘pattern language.’ Patterns were picked up by software engineers Kent Beck and Warren Cunningham in the late 1980s and applied to object oriented software development, where re-use of existing solutions was an important theme.
Pattern languages are not a complete description of design skill, but they help to document what we know in a way that promotes good design. Furthermore, the importance of domain specific knowledge is made very clear. In retrospect this should always have been obvious. A great designer of buildings is not necessarily a great designer of cakes, or software, or gardens, even though he/she might become one more quickly than other people because of understanding how to build the knowledge necessary for great design.
In summary, although design methods remain important it is now fairly clear that humans design using a large body of knowledge that is specific to the field where they have the design skill. It is also a good bet that a large part of that knowledge is organised into solutions linked to the situations where they are likely to be applicable.
Better design of internal controls
The first practical step towards getting better internal controls designed is to recognise that design is needed, makes a big difference to the results achieved, and that design is far from trivial.
Have a look at COSO's internal control framework and you will see that although it intends to cover all activities to do with devising, implementing, and operating a control system it does not mention design. What about your own organisation's literature on its risk management and internal control systems? Does it mention design? Quite likely not. Remember also that design needs a method – perhaps more so even than risk analysis.
So, if your existing image of how to manage risk or maintain controls looks something like this first diagram (the small, simple one), then why not show some more of the design activities involved. The second picture shows just one way of doing so.
If this seems like a lot of new boxes, consider the importance of each activity and the time you already spend doing it. ‘Solution (re)search’ is the time spent looking at potentially relevant controls – a lot of it window shopping with software vendors. Remember also that risk coverage is just one of the influences on controls design. Fitting in with what existing systems can do is just as important in practice. And what if there isn't time to implement a fully automated control feature immediately? What if cultural constraints rule out certain types of authorisation procedure? What if we can't afford a certain control now, but might in the future? These are all vital constraints the designer needs to consider.
The second diagram also splits design into high level assembly (i.e. of design ideas) and low level design assembly and detailing. This split can be very helpful because a rough idea of what is to be built and where the effort will be needed helps in early implementation planning and resource decisions.
None of these is trivial, and all justify a box of their own, at least.
The second practical step is to recognise that an empty table (e.g. a spreadsheet template or form) on its own is unlikely to be much help to people in coming up with good controls. They need content too; lots of it. That means lots of ideas for control mechanisms, with guidance on when they are applicable.
Mental preparation is important. If you can stuff people's heads with good control ideas, then good ideas will start coming out. (‘Good stuff in; good stuff out.’)
There are some quite good examples of controls databases around. However, the most common weaknesses are:
Supporting people in designing internal controls can lead to controls being implemented where otherwise nothing worthwhile would have been done, and to more effective controls that are easier to sustain and carry out consistently.
Put design on the map in your risk management and internal controls processes and make an investment in building better design knowledge.
Words © 2007 Matthew Leitch. First published January 2007.