Working In Uncertainty

Two simple tools for expressing objectives clearly

First published 10 September 2003 (updated 17 December 2003).

What are these for?

Consider an objective like ‘Achieve turnover of £3.25m this quarter on our UK operations.’ Could it be any clearer? Yes! In fact with just this information the objective is nearly useless for making decisions and communicates little.

What's missing is any indication of how important this is. First, how important is it compared with other objectives? Second, how valuable would other levels of achievement be, such as turnover of £3.1m, or £4.1m? It may be that it is hard to make a plan that achieves all the objectives of which the turnover target is one, so how should we choose what to do and what to leave out? Once the quarter starts it may be that events make £3.25m either easy to exceed or impossible to reach. What should we do then? How should we value outcomes other than exactly meeting all the objectives? Finally, how can we judge the need for actions to manage risk if we have no idea how to value outcomes other than the objective?

In ‘Dynamic Management’ I discussed in detail the difference between objectives and objective functions. Both are useful, especially the objective function.

To capture a person's views about the value of different levels of achievement on a set of measures involves asking a lot of questions and yet it needs to be quick and easy if people are to do it frequently and willingly. That's what the software linked to this page does.

Tool 1: capturing values and comparing them with other values

How do you use it?

Take a sneak preview now, then come back to this page for some explanation. Click here.

The smaller table at the top with headings ‘Option A’ and ‘Option B’ is the first thing you use. In this demonstration some objectives and achievement levels have been set up for an imaginary research project. Achievement levels don't have to be numbers. Sometimes it makes more sense to describe outcomes in a qualitative way.

You begin by choosing your favourite between Option A and Option B, clicking the buttons to show which you prefer and how much. As soon as you make your choice a new pair of options is presented and below the table you will see that your preferences are converted instantly into numbers, which are also displayed as coloured bars.

Do lots of these choices before moving on to the next stage. By the time you have expressed your preferences 10 - 30 times you will see a pattern emerging that you recognise as your preferences quantified. However, it may be that the pattern isn't exactly the way you see it.

In particular, there may be some outcomes that you would like to give more extreme values to. For example, if you work for a sports team perhaps ‘relegation at the end of this season’ is something very, very bad for you. Scroll down and click the button next to that achievement until you get the value you want.

If the bar gets so long you can't see all of it click the button a few times. (The Zoom buttons are down near the bottom so scroll down.) This means you can position extreme values then zoom back in to adjust the fine details.

The buttons just reset a rating to zero. This is useful if you need to change a rating from being positive to negative, or negative to positive. Using the arrow buttons alone will not do this. Click on the zero button and then carry on using the arrow buttons.

The buttons select which achievement level is the baseline for the objective. The baseline level achievement level always have a rating of zero and the ratings for other achievement levels are, therefore, by comparison with the baseline.

There's no database to store your results, so either print them off or click to send them by e-mail before leaving the page. (Note that the email address used is mine unless you change it by editing the HTML code, as described in the next section.)

What applications are there for this?

By yourself

You could use this by yourself to clarify your thinking about how you value alternative outcomes. If you send the results to yourself as an email you can copy and paste them into a spreadsheet. The e-mail is formatted as a list of ‘Rxx’ field names and values. These are the levels of achievement in the order they appear on the rating screen so you will have to match them up with the objective names and descriptions of levels of achievement.

As you think of a plan you can set up a column next to the ratings and select an achievement level for each objective. Add up the ratings for each selected achievement level and you have your score for that plan/option. For numerical achievement levels you will have to interpolate for values in between the levels you have rated.

A more complex but better approach is to simulate the plan using probability distributions to show the likelihood of different events and results. Again, each outcome can be valued using your ratings.

The fact that your ratings are not according to some absolute scale is not a problem because it is comparisons between scores using the same rating system that you will use to make decisions.

With others

When organisations make lists of their objectives there's often talk about priorities. Usually this goes no further than statements like ‘Turnover of £3.25m is a key objective this quarter’ and ‘We need to put long term growth above short term results.’ Without the sort of ratings described above these statements have little meaning, even though they are familiar and commonly made.

Yet people are aware of the vagueness involved. When they hear the Chairman of the Board saying that key objectives include investment for the long term and helping staff find an acceptable work-life balance most suspect that in practice these will prove less ‘key’ than keeping turnover rising on a straight line and screwing as much un-paid overtime out of them as possible.

If you are the Chairman you may feel this vagueness is convenient, but I hope you feel frustrated that people don't accept your honest statements.

Using a tool like the one linked to this page is an alternative. Express your values precisely, then have them written into the HTML script of the page (see below for instructions on how) as the benchmark. Then get members of your team to make their own ratings and use the comparison mode to see how their ratings compare to yours. Look at the differences and discuss why they appear. Undertake to make decisions based on the values you have stated, or updates of them that you will be equally open about.

(There is a limitation with this tool you should know about. In comparison mode the tool shows you what the benchmark preference between Option A and Option B would be by colouring the appropriate button yellow. I had hoped that clicking the yellow button repeatedly would gradually bring a person's ratings in line with the benchmark. However, although there is a slight effect in this direction it is not effective. You have to nudge the ratings using arrow buttons to get closer to the benchmark.)

Some simple technicalities

This tool is not for sale. It is free. You can copy it and change it as much as you like.

This tool has been created entirely using HTML (including its dynamic features) and Javascript. This means that to copy and change it you don't need any special software. Just a simple text editor.

The downside of this simplicity is that you have to edit the page in order to type in descriptions for Objectives and achievement levels, and to type in benchmark ratings if you want to use that functionality. You can also opt out of using comparison mode, change the e-mail address for sending results, and change the number of rows of information shown in the Option A vs Option B box.

Note: some very old browsers cannot support the dynamic features of HTML so the tool will not work. Small screens could also lead to poor layout.

How to get the source file

You can see the source code of a web page by clicking on the menu item View, then Source.

Having done that you can choose View, Source and you'll see the code. You can save it for editing and use later.

How to edit the e-mail address to which results are sent

Very near the top of the file is a line of HTML like this:

<form name="Ratings" action="mailto:matthew@workinginuncertainty.co.uk" method="post" enctype="text/plain">

You can see my e-mail address in there after "mailto:. Just replace my e-mail with the one you want your results to go to.

How to disable comparison mode

A little further down is a block of variables. One of them is this:

var comp_avail=1;  /* comparison mode available flag */

To disable comparison mode simply replace the 1 with a 0 (zero). This will remove any reference to comparison mode in the instructions and the comparison mode button will not appear.

How to edit objective and achievement level descriptions

Near the top of the source file you will see this:

atts = ["Research cost", "Research time","Research progress"];
vals = [];
vals[0] = ["£100,000","£150,000"];
vals[1] = ["2 years","2.5 years","3 years"];
vals[2] = ["None","Small refinement","Medium improvement","Large improvement","Major breakthrough"];

As you can see there is a line beginning "vals[...] = ..." for each Objective.

Replace the text with your own objective names/descriptions. Keep them fairly short if you can. You don't have to have the same number of objectives. If you want more simply insert them into the list, separating with commas. Do not insert a line break even if the line gets very long. Just let the words wrap as they will and put up with the layout.

For each objective in your list you must have a line like the ones starting "vals[0] = ["..." containing the descriptions of levels of achievement. Again, edit the text. You can have as many levels of achievement as you like provided you have at least two for each objective. If you have added to the list of objectives you will need to copy a line starting "vals[2]..." and paste a copy that you can edit. Make sure the number inside the square brackets continues the sequence.

How to put in benchmark ratings for comparison

A few lines further down comes:

cratings = [24,0,-45,-17,4,56,14,-30,-13,28];

The numbers inside the parentheses are some dummy ratings of no particular significance. Replace them with the ratings you want to use. They need to be in the order they will appear on the screen, matching your objectives and achievement levels. Count the number of numbers to make sure.

How to change the number of rows in the Option A vs Option B table

A few lines later comes:

var trialrows=3;

This controls the number of rows in the Option A vs Option B box. You might prefer it as 2 so delete 3 and enter 2 in its place.

Finally, here's another link to the tool.

Tool 2: capturing values for many objectives

What problem does this solve?

Tool 1 is an example of conjoint analysis in action. It fits an additive model to your responses to the questions. There are many similar tools and most are statistically more sophisticated than Tool 1. However, all such models suffer from the same problem: separating out the values of the various objectives gets harder and harder as the number of objectives involved increases. Just using more sophisticated mathematical procedures will not solve this problem.

Tool 2 demonstrates a different way to solve this problem, which is to put more control in the hands of the user. First, the user is able to nudge the values away from those chosen automatically. Second, the user can freeze the values for achievement levels on an objective once happy with them. Third, the user starts with just three objectives involved and can then add more, one at a time. This makes sense once one or two objectives have been settled and fixed. Gradually, the user works through the total set of objectives.

The tool is here.

Adapting this tool

As with Tool 1, you need to grab the source code from your browser, save it, and edit it. The objectives (or attributes) are in an array, as are the achievement values for each objective. The text in these is currently just some illustrative hypothetical material.

atts = ["1. % of employers claiming grant", ... ,"14. Employee Satisfaction Survey"];
vals = [];
vals[0] = ["34.6%","35.1%","35.5%","35.8%","36.0%"];
...
vals[13] = ["26","32","38","39","40"];

Made in England