Working In Uncertainty

How to interview someone about risks and controls

Should you read this?

Anyone in a management role can benefit from the skills explained here. The people who spend most time interviewing others to learn about risks and internal controls are external and internal auditors. However, ‘management’ according to the official guidance on internal control, is responsible for designing and operating effective systems of internal controls. One way to find out if your subordinates have been doing that is to interview them about the risks they see and how they are managing them.

This page is about how to find out about risks and controls by interviewing people. It is based on my audit and coaching experience and reflects leading thinking on how to analyse and assess risks and internal controls, as well as Neil Rackham's inspiring work on effectiveness models for major sales interviews. The result is a style of questioning that creates a natural and productive flow of information in the interview.

When you have finished reading this page I hope you will feel that what you have read is common sense, logical, and easy to remember and do. Surely everyone with any experience does it this way? Unfortunately, no. In fact even very experienced professional auditors rarely follow an effective pattern and gaps in generally accepted auditing theory also cause problems.

There's a lot more to interviewing for risks and controls than interviewing skill; knowledge of organisations, risks, and controls makes a huge difference to how quickly you can assimilate what an interviewee says and the quality of specific questions. However, the right pattern of questioning will transform your productivity whatever your level of knowledge.

Having said that, it's not easy to develop skills like this just by self-study, so if you really want to master them please consider engaging me for some individual technical tutoring or teletutoring. This is a highly time-efficient approach to developing technical skills.

The guidance on this page is written as if for auditors, but it is easy to ignore the few points that do not apply to line managers if you are one.

This effectiveness model is based on questions

The idea of an ‘effectiveness model’ and the focus on types of questions comes from the brilliant work of Neil Rackham, a psychologist who has made his name from his research, writing, and consulting on skills for making major sales.

Like Rackham's famous SPIN model, the questions that work well in risk/control interviews encourage the interviewee to go through a process of thinking. General interviewing advice you may have seen, such as distinguishing between open and closed questions, or following a pattern of progressive narrowing down, has little value. Rackham found that the use of open and closed questions has no impact on sales success and I would be surprised if it has any relevance to internal control interviewing.

So, if you've been on a course about interviewing or read a book or article and think you know what I'm going to say, just keep on reading because you're probably wrong. In this page I'll explain what really makes risk/control meetings work using question types specifically designed for this kind of interview.

Get it right and the interview will flow naturally and productively as one thought leads to another.

For each type of question there are a number of recommended wordings. Although it is very effective to use these tried and tested phrases, anything that achieves the same basic message will do. One of my favourite questions is ‘What are the changes and challenges you face this year?’ but not everyone feels comfortable saying this. If you think this sounds cheesy use other words or practice in front of a mirror until you can do it with a straight face and a look of genuine interest in the answer. Trust me. This question works like magic because it asks people to talk about the things they are thinking about most at work.

The effectiveness model

Conversations about risks and controls can be encouraged to flow along productive lines. One thought follows another logically, with a little encouragement. If you ask your questions correctly an interviewee who knows the answers will sometimes answer your next question before you can ask it.

How is this achieved? Ask the right type of question at the right time. The general flow of questions is like this:

Scope out the area with . . .

Questions to ensure that everything relevant has been identified

. . . then find out about what the risks might be with . . .

Questions to help people think of inherent risk facts (including how things work)

. . . and move on to what people are doing about those risks using . . .

Questions to help people think of control facts

. . . followed by evidence on whether those actions are working using . . .

Questions to help people think of control results (i.e. whether they work)

. . . then grinding down as far as necessary to get comfortable with . . .

Questions to get down to specifics and to validate.

As you go you will often need to ask . . .

Questions to check understanding

. . . and occasionally use . . .

Questions or statements to stay relevant.

Normally the questions to ensure complete coverage would be asked first for a topic, followed by inherent risk questions, control questions, control result questions, validation questions, and then a final completeness question. Questions to get specific/validate, stay relevant, or check understanding are often needed throughout.

It may be that the interview as a whole moves in this way, or that sections of it do, or that, with some backtracking and jumping around, the sections of the interview that relate to controls follow this pattern.

Preliminaries

Begin by preparing mentally. With experience and study your mind becomes better able to take in what interviewees say. However, before a meeting it is vital to read everything directly relevant you already know about the area where you are trying to find or confirm facts. (If you are examining computer security it is very useful to find out exactly how to view parameters, run reports, etc as system administrators often do not know.)

If your knowledge is weak the initial questions about scope and inherent risks will tend to lead to a lot of factual background and explanations. It is easy to lose the whole interview to this kind of information and that may be the best choice but, if it can be avoided by preparation, it should be.

Obviously you need to think of some kind of agenda, or plan for the meeting. It also helps if you can base this on a systematic way of recording the facts gathered that helps ensure you get everything you need. For example, you might construct a table to fill in, or a checklist, or draw a diagram which needs to be annotated.

Giving the agenda to the interviewee in advance is helpful with some people in some organisations. In other organisations the cultural norm is to do no preparation and an agenda provided in advance will be ignored.

Begin the interview with a credible self introduction and explain the purpose of the meeting. Begin signalling to the interviewee, with just a few sentences, what it is you are trying to find out and why. The interviewee can then help. For example, if you are an external auditor talking to someone in IT you might say:

‘Hi, I'm Chris from KPMG TouchewaterhouseCoopers and I'm part of the audit team.’ Assorted pleasantries, establish that they would like to know why you're there, then continue with ‘We have to give an opinion on the reliability of your company's financial statements, and of course most of the numbers come from the computers you look after. So we want to ensure that the computers have been managed in a safe and reliable way and that there's nothing there that should make us worry about the numbers coming out. I'd like to find out about the events of the year and how you ensure that problems are minimised. Is that ok?’

Keep the language simple and don't use audit jargon.

Interviewees like to feel that you've not forgotten things they told you or your team in the past. So show you've done your homework. e.g. ‘I've been looking through our notes and correspondence and I see that ...’ Then later, start 10% to 50% of your lines of questioning with some kind of reminder that you know the business. e.g. ‘You said last year that ..... Has that changed at all?’, ‘What about the implementation of upgraded Oracle you were planning?’

Following on from your introduction, it can help to focus the interviewee and maintain cooperation if you occasionally explain the reasons why you consider certain points to be significant risks. e.g. ‘After the problems you've been having with people not submitting correct codes, I'm wondering if there have been any other problems due to the company reorganisation. Can we consider that next?’ (Note how much better it is if you can give a reason why something is a risk. When you probe difficult issues the interviewee would prefer to avoid, giving good reasons makes it seem more reasonable and less like persecution.)

Another way to maintain cooperation is to show empathy, for example by taking an interest in the interviewee's job, taking the trouble to learn about it in preparation for the meeting, and generally listening actively. However, do not be caught out empathising for too long if the conversation has strayed off the agenda.

During the body of the meeting, fact finding questions are the key. The open vs closed question distinction is not specific enough for our purposes. In a typical interview for a review such as an audit, questions should come from all these categories:

Questions to ensure that everything relevant has been identified

Most auditors have had the experience of discovering something quite important that should really have been discovered in a previous review. Occasionally we miss things ourselves and find them out late in the review, which is stressful. On an unknown number of occasions we miss something big completely and it is others who will find our mistakes.

When an interviewee is chatting happily about his/her job it is easy to go with the flow and not realise that you have missed something huge and important. Avoid that mistake using questions that scope out the area thoroughly.

Recommended wordings include:

• Could you give me a brief overview of the organisation structure for [the area of interest]? / What's your role/job? / What goes on in your team/department /division?
• Could you give us a brief overview of the products/services/platforms/applications?
• What are all these other files/computers/people/etc for? / Is there anyone else who does that?
• What's changed since we last met/this year? / What are the plans for the future?
• Is there anything we've missed? / Is there anything I should have asked you about but didn't?

The first question above, asking about their organisation structure, is probably the most useful opener in most situations (except where you should already know!) Asking if there is anything else you should have asked about makes you look a fool if your previous questions have been weak, but is quite easy to ask if you have been going well.

Questions to help people think of inherent risk facts (including how things work)

Some people are happy to be asked directly what ‘risks’ (i.e. potential issues) they perceive, but I suggest you focus on facts that make their job a challenge.

Recommended wordings include:

• Tell me about the changes and challenges you see this year/at the moment?
• How do the systems work? / Then what? / How does ....? / Can you talk me through the process please?
• What's changed? / What changes are happening?
• How often does that happen? / How many times? / What financial value? / How long has this been happening?
• What makes your job hard? / How complex is that? Show me please. / Manual or automated?

Asking about change is probably the most important and useful line of questioning in most situations. If it is not clear what a person is supposed to be achieving it is helpful to ask for an explanation at this point. Some styles of risk analysis make objectives the main source of risk, but this tends to lead to rather sterile and unhelpful ‘risks’. Besides, most people have jobs where they chase vague and continually moving objectives. It is more productive for you to ask about projects and changes in circumstances.

Once the drivers of potential issues have started to surface people naturally start to think about what they are doing about them.

Questions to help people think of control facts

Asking directly what controls someone is using sometimes works but more often people are not sure what a control is or have a very narrow concept of controls and so do not give themselves credit for good things they are doing. It is best to start questions of this type with open, non-leading questions but if the interviewee is struggling move into making more suggestions to help them remember relevant facts. The better your skill at anticipating what should be in place the easier it is to do this.

Recommended wordings include:

• How do you know you can rely on that information/system? / How do you know that's worked? / How do you know you've entered everything? / How do you know you haven't entered something twice? / etc
• Do you have/do [a control you expect]?

Questions to help people think of control results (i.e. whether they work)

Audit theory books tend to suggest that the only way to see if controls are working is to decide if they are well designed and test if they are operating. Fortunately, this is not true.

There is usually plenty of direct evidence of success in the form of error statistics, backlogs, delays, and the time people spend fire-fighting. Always probe for it.

Recommended wordings include:

• What issues are you working on? / What issues are there? / What outstanding system bugs are there?
• Do you track those backlogs/errors/complaints? / Can I see the figures please? / What's the situation now?

Questions to get down to specifics and to validate

Probing for specifics is often necessary because people tend to talk in generalisations and abstractions. Statements like ‘We've taken a root and branch approach to our internal control framework process.’ and ‘We've had some minor issues with run time compatibility and user reactions.’ are so vague they are barely worth noting down. Since you want to know the truth they cannot be allowed to pass and asking for specifics is a powerful way to get to the truth.

Verification means seeking additional evidence. It may not be possible to do it in the interview so decide quickly what verification you want and go ahead if there is time, or make a date to follow up later.

Recommended wordings include:

• When you say [vague term used by interviewee], what do you mean? / Can you give me some examples? / Would you mind talking me through an example?
• Can you show me? / What does that look like? / Why don't we do this at your desk then you can easily show me?
• Can we look at some examples at random? / How could I see that? / How could I confirm or corroborate that?
• How often does this happen? / When was the last time? / How many times has that happened so far this year?
• Then what? / What do you do if X?

Questions to check understanding

Clarity is important. The most intricate parts of processes are often the controls, not the underlying work.

Getting people to start making some sense requires persistence in many cases. It is easy to feel that you ought to have understood what they just said and to shy away from asking for clarification. Occasionally people do react angrily to patient questioning that checks understanding often. But this is rare and in almost all cases I have found that my failure to understand is more because of a poor explanation than my inadequacy.

Most people are not very good at explaining things and the main reason is failure to consider the other person's lack of knowledge. When someone is not considering your limitations the best approach is to take firm control of the conversation and direct them to give you the information you need to make sense of what they are saying. The alternative is usually to let them repeat themselves at ever greater length, often with excruciatingly complicated examples explained badly in minute detail.

Recommended wordings include:

• I don't understand. / Can you just say that last sentence again please? / I'm not sure I got that. Can you please say that again - just the same words.
• So, let me see if I've understood. [summarise] / In summary, then, X, correct?
• You're saying that X. So, presumably Y (inference)? / Does that mean that X? / So X, but earlier I thought you said Y (contradictory)?

Questions or statements to stay relevant

Sometimes people have to be politely redirected.

Recommended wordings include:

• Thank you, that's fine/enough. I wonder if we could move on to X. / We've only got another 20 minutes, so I wonder if we could move on to X.
• What I am trying to establish is X.

Preparing the way for solutions

If you want to go on and fix control problems you need to explore solutions and begin gathering support for them. Two further types of question become important as well:

Questions to get the interviewee to think about causes and effects

If a risk appears not to be well controlled and there appears to be a need to do better it is important to get people thinking about how important it is to reduce the weakness. Asking them about implications is a powerful way to do this. You may find that in fact it is not important, but more often you will find it is.

Recommended wordings include:

• What are the implications of that problem/risk/issue? / What other implications might there be? / Can we try to list/quantify the effects to see just how serious that might be? / What would that lead to?
• Why does that happen? / What are the main causes of this problem? / What is creating/driving this risk?

Questions to prepare the way for proposing a solution

Without actually proposing a specific solution it is useful to test points for intervention.

Recommended wordings include:

• If we could improve X/stop X happening/reduce X/increase X/etc (a particular point in the cause-effect network built through the previous type of question), would that provide at least a partial solution? / Suppose there was a way to X. Wouldn't that help?
• Can you think of any other effects of improving X?

These two types of question are useful for bringing interviewees along with you. Once you've found out some facts and identified a problem, the ‘cause and effect’ questions can be used to assess the importance of the problem, and get the interviewee to understand it too. The more they understand the impact of a problem the more they want to solve it, if that is justified.

When the interviewee is sufficiently concerned about a problem to want to solve it, he/she will begin to consider the options seriously. The temptation is to produce a solution at this point (if you have one) but even that is often too early. Hence, questions are needed to confirm that the interviewee agrees an intervention at a particular point (or set of points) could be effective. This requires an ‘If we could improve...’ type of question.

Once there is agreement in principle that an intervention at particular points could be effective, finally you can present your solutions and demonstrate as far as possible that they would work.

Finishing off

Winding up the conversation with thanks and cheery conversation is useful, along with asking if it's ok to call if there are any follow up questions. It may be useful to ask if a further meeting would be possible if needed.

Another important skill in fact finding is note taking. It helps to write a lot, and very fast (easily 6 pages in an hour). Note all the details you will need in order to write up the point or test later without having to go back again. Write up your file (if you have one) as soon as possible, before the details fade from memory. If you have set up a table or other format that ensures you have a place for every piece of information you need, that is also very helpful. For example, if you ensure you can summarise in a few words the function of every process and the content of every dataflow on a diagram, that is a good check on the completeness of your understanding.

Conclusion

The basic pattern of questions is: what should we talk about?, what's going on that means risk?, what are you doing about it?, is it working?, and can I check that?

Simple? Common sense? Of course, but if you do it you will be ahead of most other people who do this kind of interviewing. I am available to provide individual technical tutoring to help you master the skills quickly.

If you have any ideas, questions, or complaints feel free to let me know at matthew@workinginuncertainty.co.uk. I usually respond within a couple of days.

Made in England