
Solutions that cannot work
The method tells you what to do. This tells you what you are looking at.
The numbered white papers are a method: five questions in a fixed order, and depth on each. They are sober by design, because a diagnosis that is amusing is usually a diagnosis that is wrong. This is a different object, and it is allowed to be funnier, because the thing it does is recognition, and nobody has ever recognized a category. People recognize a scene.
So the entries are scenes. Most of them are real, several of them are ours, and in at least one we are the ones who look ridiculous. That is deliberate: a guide to other people’s mistakes is a sales brochure, and a guide that includes your own is an instrument.
Why it is funny, and why that is not a joke
One thing runs underneath all seventeen and is worth saying once rather than seventeen times. None of these scenes is a story about ignorance. A discipline for exactly this has existed since 1962. It has a professional body, ten standards, sixty years of literature and a five-question sequence that takes a competent person about a week. It is not secret, it is not expensive, and it is not difficult. In every scene that follows, somebody could have run it, and instead booked a room.
Not from people being foolish. From the distance between what was available and what was chosen: a week of asking, against a two-day offsite that was arranged before anybody asked. Read every entry with that gap in mind and the whole thing becomes funnier and considerably more uncomfortable.
The seventeen forms come from the twenty set out in Kostbaar misverstand (Arets & Heijnen, 2008), which is where they were first named. The scenes come from the practice notes in that book, rewritten for an English reader and reordered by where you meet them rather than by their numbering there. Every entry names the form and points to the number of this series that takes it apart.
Two of the original twenty are deliberately absent, those about the balance between formal and informal learning, because that argument belongs to a different body of work and this series has kept it out on purpose.
Nobody has ever recognized a category.
People recognize a scene.
The two agencies, and the one that lost
A commercial organization is hard to reach by telephone. Customers say so and staff say so, and the complaints arrive every month in slightly larger numbers. The manager decides something has to be done about it, and he is right.
He speaks to a few of his people. He watches for a while himself. By the end of that he is convinced that training is unavoidable, and he asks around his network for agencies who can run a course in telephone handling, and, he adds, in a motivating and inspiring way.
He meets two. The first gives him a good feeling. The trainer thinks he can start almost immediately and would like a short conversation with a few people, so that the program can be tailored. The second gives him a bad feeling. That trainer wants to run an analysis first, to find out whether factors other than a shortfall in competence might be decisive in why the required results are not being reached.
The manager finds this mildly irritating. He is genuinely convinced that training is the answer.
The training reflex, meeting supply creates demand in the same afternoon. The market offers courses, so requests arrive shaped like courses, and an agency that offers to find out what is actually wrong is competing against one that offers to start on Tuesday. The second agency lost the work for offering to do the work properly, and it will keep losing it until somebody separates the request from the diagnosis.
Kostbaar misverstand, forms 6 and 19. The training reflex is our own term and that book is where it first appears (Arets & Heijnen, 2008); the habit of treating training as a one-size-fits-every-performance-problem remedy is Willmore’s (Willmore, 2004). Taken apart in numbers 12 and 10.
What was on the table, and what was bought
Two observations are worth adding to that scene, and the first is the one that would sound ridiculous in any other trade.
The second agency was offering, at its own expense, to establish what the problem was. It lost on that. Imagine the same conversation in any other trade: two builders quote for a leak, the first proposes to start Tuesday and asks which color, the second would like to find out where the water is coming from, and loses for asking. Nobody would tell that story about a roof.
The part worth sitting with is the word “tailored.” It sounds like diligence and it is the opposite of it: the content is being adjusted to the audience while the question of whether content is the problem at all remains unasked. Tailoring is what you do after the diagnosis. Before it, it is decoration.
The interim manager who arrested himself
This one is worth quoting almost in full, because it is the clearest description of the performance paradox anybody is likely to write, and it was written by the person inside it.
It begins as an irritation. Working as an interim manager, he finds himself annoyed by professionals who behave differently from how he would like. It builds over weeks. And then the thought arrives, fully formed and entirely reasonable: it is time to ask a training agency to train this group in results-orientation. Then he is rid of the irritation and the organization performs better.
Read that sentence again and notice how much work it does. An irritation about behavior has become a diagnosis. The diagnosis has produced a remedy. The remedy will remove the irritation, which is true, and improve the organization’s performance, which does not follow from anything.
What makes it remarkable is the second half. Later, the same man writes that he knows perfectly well it does not matter whether somebody drinks coffee for a long time, or talks to colleagues for too long, or keeps to arrangements. What matters is that the environment is arranged so that those fragmentary observations of behavior genuinely stop mattering, because the desired performance is measured, and a measurement is a more objective basis for a conversation with professionals than an impression is. And then: he increasingly realizes that he too is a prisoner of the performance paradox, because he keeps wanting to improve the professional instead of the performance.
The performance paradox, with the diploma chase and the training reflex visible underneath it. A better result is demanded, an intervention is chosen that cannot reach the cause, and the result is expected anyway.
Kostbaar misverstand, forms 5 and 6. Taken apart in number 12.
What he actually bought, and for how long it worked
“Results-orientation” is not a subject. Nobody has ever come back from two days of it and produced a result, and everybody involved knows this, including the agency that will sell it. What is being purchased is relief from an irritation, and it will work: for about two weeks, which is roughly how long it takes for the same professionals to resume the same routines in the same environment that was never touched.
What the alternative would have cost is worth putting beside it. The five questions, on a team of that size, is a week: pull the measure, cut it three ways, find the person who already achieves it, walk the floor for a morning, ask four questions of four people. One week of one person’s attention against two days of everybody’s, and at the end of the week you know whether results-orientation was ever the issue.
Note also what he did not do, which is what the guide is for. He recognized the form while he was standing in it. That is available to anybody, it costs nothing, and it is the entire practical value of knowing that these forms have names.
Four more you will meet before lunch
The other forms in this act need less telling. Each one is recognizable inside ten minutes, and each is cheaper to name then than to discover in the evaluation.
| The form | What you will actually hear | What it is a symptom of | Where |
|---|---|---|---|
| Act first, think later form 7 | “We know what the problem is. Can you have something ready for the offsite?” | A date exists before a diagnosis does, and everything after it is arranged backwards from the date. | 01, 12 |
| No number, no case form 8 | “They need to be more customer-focused.” | There is no measure, so nothing can be shown to have improved, and, more comfortably for everyone, nothing can be shown to have failed. | 05, 06, 12 |
| The illogic of training form 11 · Brinkerhoff & Apking, 2001 | “Of course training will help. What else would?” | Training reaches a result only if two conditions hold: the cause really is a shortfall in knowledge and skill, and what is learned can be applied in the work. Both are assumed and neither is checked. | 03, 09 |
| The career-coaching request form 1 | “Mobility in the expert team is below par, so too little expert knowledge reaches the consultants, so mistakes are made. We would like career coaching.” | A chain of reasoning that is entirely coherent and never once touches the mistakes. The remedy is aimed at the first link because that is the link with a supplier. | 05, 10 |
The last row rewards a second reading. The reasoning is impeccable (too few experts move across, so too little expert knowledge reaches the consultants, so mistakes are made) and it arrives with a remedy attached to the first link in the chain, which is the only link with a supplier. Nobody has looked at a single mistake. There are mistakes; they are recorded; somebody could go and read them this afternoon.
The agency that offered to find out what was wrong
lost the work for
offering to find out what was wrong.
The water engineers, and the conference that got better every year
This is ours. It is the one we tell against ourselves, and it is the reason this guide exists in the form it does.
Early in 2000, a telephone call. The manager of a team of engineers responsible for water and safety would like to talk about a two-day working conference, to help the team communicate more effectively with one another. There is a thorough conversation about what is needed. We come back with a program, a pleasant location and an agreeable price.
The program includes teambuilding and it runs beautifully. Exercises alternate with quieter moments of self-disclosure, and the team feels itself growing closer. The evaluations are excellent. The client is satisfied, and the next conference is informally agreed before we leave.
And it must be said: a year later the two-day conference goes even better. The expectations of the client and of the participants are exceeded.
There is also a feeling that something is not right.
The client is content with the training. The team also reports that it is not performing well enough: the required goals are not being met, and comparable teams do better. And although the manager senses that the annual conferences contribute little or nothing to the results he needs, he certainly wants to continue with them. They improve the atmosphere, and that is worth something too. Under the circumstances it seems unwise to him to throw the baby out with the bathwater.
Training as stagecraft. The event was designed to be excellent as an event and it was excellent as an event. Every measure taken was real. Not one of them was the measure the team was being judged on, and the better the conference got, the more firmly it established itself as the thing that would happen again next year.
Kostbaar misverstand, form 9. Rossett and Sheldon (2001) is the standing treatment of what a training event delivers as against what it is meant to produce. Taken apart in number 11.
What we had, and never once looked at
The team is responsible for water and safety. Their measure exists, it is reported, and it was available to us on the first morning. We spent two days on communication and never once looked at it, and then, because everybody enjoyed it, did the whole thing again a year later and did it better. There is a name for a two-day intervention that improves a measure nobody is judged on. It is a conference.
The special location, and the Tuesday after it
The second scene in this act is a sentence you have almost certainly heard in a planning meeting, and it is the opening of a curve with a very reliable shape.
“We really must do something special, so that everybody goes home completely inspired. And I think that is a combination of a separate location, something out of the ordinary …”
Which happens, and it works. The enthusiasm at the start of an improvement or change program is genuine, and it is not manufactured. Then the participants go back and try to apply what they learned, and they discover, again, that they knew perfectly well what needed to change and how, and that the blockages they had already identified are still exactly where they were.
The curve runs from inspiration to frustration, and the second half is steeper than the first. What is left afterwards is worse than nothing, because the next proposal arrives in a room that has been inspired before. The budget went almost entirely on the first half (the location, the catering, the arc of the day) and by the time the learning meets the work there is nothing left to spend on supporting it, which is the argument Wick, Pollock, Jefferson and Flanagan (2006) built their six disciplines around.
The survey they filled in, before they were inspired
The participants arrive already knowing what needs to change. They said so in the pre-program survey, which was read, summarized and put on a slide. Then they were taken somewhere pleasant and inspired into a state of wanting to do the thing they had already told everybody they wanted to do, and sent back to the conditions that were stopping them, which nobody had touched, because touching them was not in scope.
| The form | What you will actually hear | What it is a symptom of | Where |
|---|---|---|---|
| The inspiration–frustration curve form 10 · Wick et al., 2006 | “The energy in the room was extraordinary.”, and then, six weeks later, nothing. | The enthusiasm was real and it was aimed at people who already knew what to change. What stopped them was never touched. | 03, 09 |
| Improving the person, not the performance form 4 | “By the end they will be much stronger at this.” | The objective has quietly become the person rather than what the person produces. Both may be true, and only one of them was promised. | 05, 08 |
| No connection to the business activity form 1 | “It is a generic program, but we will tailor the examples.” | The content is about the topic rather than about the work, so nothing in it maps onto anything anybody does on Monday. | 05, 10 |
Thousands of people, and never once a no
There is one observation in this literature that does more work than its length suggests, and it takes the form of a question you can ask anybody.
Ask a professional whether they could perform better than they currently do. Harold Stolovitch (whose Training Ain’t Performance (Stolovitch & Keeps, 2004) is the standing statement of the distinction this series rests on) reports that across thousands of people he has never once had “no” as the answer (Stolovitch & Keeps, 2004). Every manager and every professional gives the same reply.
Follow that where it goes. If essentially everybody believes they could perform better, something stands between what they are capable of and what they produce, and since it is present for almost everyone, it is unlikely to be a property of any of them. It is in the work: the physical means, the structure, the supporting processes, the information about how the work is going.
The blockages are still there. It is also the single best question to open an analysis with, because it is impossible to answer defensively and it puts the person doing the work on the same side of the table as you. The follow-up does the real work: what stops you?
Kostbaar misverstand, form 2. Taken apart in numbers 03 and 09.
It is one question. It costs nothing, it takes eleven seconds, and it has apparently never once produced a no. In most organizations nobody has asked it, and the reason nobody has asked it is that it belongs to no function: it is too operational for HR, too soft for operations and too obvious for a consultant to charge for.
The other two forms here are quieter. The environment steers elsewhere: handling time is discussed weekly and the result you actually want is not, so the work moves towards what is measured (numbers 09 and 10). And too little reaches the floor: transfer was assumed rather than designed (numbers 03 and 14).
Three shapes that arrive when somebody finally asks
The last act arrives at the end, usually in a review, and by then the cost of each shape is fixed. All three are questions about what was collected rather than about what was done.
| The form | What you will actually hear | What it is a symptom of | Where |
|---|---|---|---|
| Data-free evaluation form 16 · Phillips & Phillips, 2007 | “The feedback was excellent: 4.3 out of 5.” | Something real was measured at the level of the room. Nothing was measured at the level where the result was promised, and the two are being allowed to stand in for each other. | 11, 14 |
| Evaluating afterwards form 17 | “Now let’s work out how to show that this worked.” | The design was decided after the intervention, so the evidence that would have made a claim defensible was never collected. Nothing repairs this later, at any price. | 11, 16 |
| Individual results read as organizational form 15 | “Everyone who went through it says it changed how they work.” | Individual change is genuine, and it is a different level from the one the organization holds its results at. | 08, 11 |
The satisfaction score is collected from every participant, on the day, without fail, by a process nobody has ever questioned. The business measure the program was funded to move is collected too: monthly, by a different function, into a report the training team has never seen. Both numbers exist. They have never been on the same page, and putting them there is an afternoon’s work that has been available for as long as both reports have existed.
And a closing observation that belongs here rather than anywhere else. In almost every one of these scenes the participants are not fooled. They sense, often from the first morning, that the work being done is not the work that would help, and they say so in the corridor and not on the form. A room full of people who are being polite is the most reliable early warning in this whole profession, and it is the one nobody records.
A room full of people being polite
is the most reliable early warning in
this profession.
Seventeen shapes, sixteen addresses
Which is what makes this a layer over the series rather than an appendix to it. Every form has an address. You can arrive at the method by knowing which of the five questions you are on, or by recognizing the shape in front of you, and both routes end in the same numbered papers.
Four claims it does not make
A guide made of stories has a particular way of going wrong, and it is worth naming before somebody uses it as an argument.
- These are observations, not findings. Not one of the seventeen forms is a research result, none carries a frequency, and nothing here says how often any of them occurs. They are patterns two practitioners have seen repeatedly over twenty years, first written down in 2008. Number 02 sets out the evidence rule the series follows and why this field has few studies.
- A scene is not a diagnosis. Recognizing the shape tells you what to look at, and it tells you nothing about this case. Every entry points into a number precisely because the work still has to be done, and the guide is at its most dangerous in the hands of somebody who has just learned four names.
- The people in these scenes are competent. The manager convinced of training, the interim manager irritated by his team, the agency that started on Tuesday: all of them are doing something reasonable under the pressures they are actually under. A guide that reads as a catalogue of other people’s stupidity has misunderstood its own subject.
- One of them is ours. The water engineers is our conference, our excellent evaluations and our informally agreed repeat booking. It is in here because a guide to other people’s mistakes is a sales brochure.
Seventeen shapes, four acts, and every one of them with an address in the numbered papers. The method tells you what to do once you know you have a problem. This tells you that you are looking at one, which is the step nobody writes down, and the step everything else depends on.
Six lines, on the sentence in front of you
Ten minutes, with this guide open. Line one is the only one that needs anybody else in the room, and line six is the one that makes the recognition survive you.
Two problems, one method, sixteen papers
| Business problem | Training request | |
|---|---|---|
| Owned by | The manager or director who owns the result. | L&D and the HR business partner. |
| What it prevents | Spending on the wrong intervention: a system, a reorganization, a hire. | Spending on a program that cannot reach the cause. |
| What it returns | The fix that moves the number, chosen on evidence. | The budget and the working hours that were not spent. |
| Method used | The same three analyses and five questions, which is why one capability serves both, and why the diagnosis no longer has to be bought in case by case. | |
Two models carry the method. Gilbert’s six conditions ask what kind of cause; Rummler and Brache’s three levels ask where it sits. Both are sound and this series uses both, and it leads with Gilbert, because his unit is the accomplishment, which is what an organization already counts, and because his six conditions are the categories an organization can actually change.
Human performance technology travels the way a craft does: from hand to hand. Vivian Heijnen and Jos Arets learned the analysis from Dennis Mankin, in San Diego, more than twenty years ago, and have kept in contact with him ever since. Both remain grateful to him: for the rigor of the method as he taught it, and for the leadership he has given performance consulting over a long career.
What followed was twenty years of use. Vivian took the method into organizations in the Netherlands and well beyond them, and what accumulates over that many analyses is largely tacit: the cut nobody thought to make, the question that opens a defensive room, the finding that turns out to be an artifact of the data. Making that knowledge explicit, case by case, is what this series is for. Jos wrote it down as it went, in two books and more than forty articles and blog posts, most of them in Dutch: Liever (g)een training with Boudewijn Overduin, Kostbaar misverstand with Vivian Heijnen, which this series takes a good deal of its thinking from.
The seventeen entries in this guide come from that second book, where twenty forms of the costly misunderstanding were first named and where the practice notes retold here first appeared. They are reworked rather than reprinted, reordered by where a practitioner meets them, and pointed back into the number that takes each one apart.
Where this comes from
This guide makes no empirical claim of its own. Every scene and every form is an observation from practice, first set out in Kostbaar misverstand and labeled as an observation here. Where that book named a source for a form, that work is listed below in the edition we checked, and it is cited for what it is: an attribution of where a form comes from, not evidence that anything works. The numbers each entry points to carry the argument, the evidence and the limits, and number 02 sets out the evidence rule the series follows, which is that empirical claims rest on meta-analyses and systematic reviews and never on a single study.
Arets, J., & Heijnen, V. (2008). Kostbaar misverstand: Van training naar business improvement [A costly misunderstanding: From training to business improvement]. Academic Service.
Brinkerhoff, R. O., & Apking, A. M. (2001). High impact learning: Strategies for leveraging performance and business results from training investments. Perseus Publishing.
Gilbert, T. F. (1978). Human competence: Engineering worthy performance. McGraw-Hill.
Phillips, J. J., & Phillips, P. P. (2007). Show me the money: How to determine ROI in people, projects, and programs. Berrett-Koehler.
Rossett, A., & Sheldon, K. (2001). Beyond the podium: Delivering training and performance to a digital world. Jossey-Bass/Pfeiffer.
Rummler, G. A., & Brache, A. P. (2013). Improving performance: How to manage the white space on the organization chart (3rd ed.). Jossey-Bass. (Original work published 1990)
Stolovitch, H. D., & Keeps, E. J. (2004). Training ain’t performance. ASTD Press.
Wick, C. W., Pollock, R. V. H., Jefferson, A. M., & Flanagan, R. D. (2006). The six disciplines of breakthrough learning. Pfeiffer.
Willmore, J. (2004). Performance basics. ASTD Press.
Form numbers throughout refer to the twenty forms of the costly misunderstanding as numbered in Kostbaar misverstand. The other two books of the series are listed in full in number 02.
Said here rather than hidden in a bracket. The training reflex needed no outside source: it is our own term, first set out in Kostbaar misverstand. And Wick and colleagues are cited here as 2006 rather than 2008, because the four-author edition is the first and the only one carrying all four names.