Structured Decision Making Beats Being Clever About It

Most executives believe their decisions are limited by the quality of their analysis. Better data, a sharper model, one more week of work. The largest study of real business decisions found the opposite, and the size of the gap is the reason structured decision making is worth taking seriously rather than treating as bureaucracy.

Process mattered six times more than analysis.

Process Beats Analysis by Six to One

Dan Lovallo and Olivier Sibony studied 1,048 major business decisions made over five years, covering investments, acquisitions and capital spending, and compared how each was analysed with how each was decided.

Process explained about 39 percent of the variance in outcomes. Analysis explained about 8 percent. Moving a decision from the bottom quartile to the top quartile on process was worth 6.9 percentage points of return on investment.

The authors add a caveat worth carrying, since it stops the finding being read as an argument for winging it: almost nothing in their sample combined a very strong process with very poor analysis. Good process is what makes analysis count, by deciding which analysis gets done and whether anyone is allowed to argue with it.

What they mean by process is unglamorous. Explicitly exploring alternatives to the proposed course. Testing whether the numbers were assembled to answer the question or to support the answer. Deciding in advance what would count as evidence against. Naming who decides and when.

None of that requires more information. All of it requires agreeing the rules before you see the result.

The Variation You Cannot Feel

Here is the finding that usually changes people’s minds.

In a study described in Noise, insurance executives were asked how much two qualified underwriters in their own company would differ when pricing the same risk. They guessed around 10 percent. The measured median difference was 55 percent.

Same firm, same training, same file, wildly different numbers. Nobody in that company could feel it, because each underwriter experienced their own judgement as considered and reasonable, which it was.

That is the argument for structure in one statistic. The problem being solved is variation. Unaided expert judgement moves around far more than the experts believe, and it stays invisible from the inside because nobody is running the comparison.

What Structured Decision Making Looks Like in Practice

The clearest evidence comes from hiring, where the same decision is made repeatedly and outcomes can be tracked.

A 2023 review of the selection research, correcting errors in the meta-analyses everyone had been citing since 1998, put the structured interview at .42 and the unstructured interview at .19 for predicting job performance. Structure roughly doubles what an interview is worth, and on the revised figures the structured interview is the single best predictor of any method reviewed, ahead of cognitive ability testing.

Structured here means something specific and slightly boring: the same questions, in the same order, to every candidate, scored against criteria agreed before anyone was interviewed. That is the entire intervention. It costs nothing and almost nobody does it, because an unstructured conversation feels more insightful and produces more confidence.

Confidence is the thing structure takes away from you. That is why it works and why it is unpopular.

The Forecast Is Wrong in a Predictable Direction

Bent Flyvbjerg’s study of 258 transport projects worth about $90 billion across twenty countries found that 86 percent cost more than estimated. Average overrun was 27.6 percent. Rail averaged 44.7 percent.

The detail that matters most for anyone forecasting anything: accuracy had not improved in seventy years. Not a slow improvement, not a plateau after early errors. Seven decades of the same mistake in the same direction.

The fix is reference class forecasting, which means predicting your project by looking at what happened to similar projects rather than by reasoning about this one. It feels like a downgrade. You are replacing detailed knowledge of your own situation with a base rate from other people’s. It is more accurate anyway, because the specifics you know about are mostly reasons the estimate is too optimistic.

The same correction applies to a business plan, a product launch and a build. Ask how long this took the last five times, for anyone, then start there.

Three Structures Worth Copying

The checklist, for anything you do repeatedly under pressure. The WHO surgical safety checklist, nineteen items, was tested across eight hospitals in eight countries. Death rates fell from 1.5 percent to 0.8 percent and inpatient complications from 11 percent to 7 percent, across roughly 7,700 patients. The surgeons were not better after the checklist. The process stopped competent people skipping steps they knew.

The premortem, for anything large and irreversible. Before committing, assume it is eighteen months later and the decision has failed badly, then write down why. Imagining an outcome as already settled improves the ability to identify its causes by about 30 percent, and it gives the quiet sceptic in the room a legitimate way to speak.

Written criteria before the options. Decide what a good answer must achieve, in writing, before evaluating candidates for it. Whether the decision is a hire, a supplier, a market or a plan, criteria written afterwards are a description of what you already chose. This is the cheapest structure on the list and the one most often skipped, because it feels like delay.

Can You Train the Judgement Instead?

Partly, and less than you would hope.

The best evidence comes from forecasting tournaments, where thousands of people answered geopolitical questions over four years and were scored against what actually happened. A probability training module lasting under an hour improved accuracy by 6 to 11 percent, consistently, across four separate cohorts of several hundred forecasters each.

An hour of training buying a durable ten percent improvement is an excellent return. It is also far smaller than the gap between a structured process and an unstructured one, which tells you where to spend the effort when you only have appetite for one change.

Worth being straight about one popular recommendation. Decision journals, where you write down what you expect before you find out what happened, are sensible, and there is no controlled study showing they improve outcomes. Nobody has measured it properly. That does not make the practice wrong, and anyone citing evidence for it is citing something that has yet to be produced.

The asymmetry between those two paragraphs is the point. One of them has four years of scored forecasts behind it. The other has a good argument and a lot of enthusiasm, which is how most management advice arrives.

What This Does Not Fix

Structure improves the average decision. It does not turn a bad strategy into a good one, and a process run by people who have already agreed the answer is theatre with minutes attached.

It also has a cost, and the cost is speed. Not every decision deserves it. The rough filter: how reversible is this, and how often will you make it again? A reversible decision made weekly should be fast and roughly right. An irreversible decision made once a decade deserves a week of structure, and typically gets an afternoon.

There is one modern failure mode worth naming. A confident machine-generated answer feels like analysis and removes the friction that structure depends on, which is why the accuracy question around AI tools matters more for decisions than for drafting. Fluency is not evidence. If the process would have required a second opinion from a person, it still requires one.

Process and Theatre Look Identical From Outside

The failure mode of everything above is a company that adopts the forms and keeps the habits. A premortem where nobody says anything uncomfortable. Criteria written after the shortlist. A checklist signed at the end of the week for the whole week.

Two tests separate the real thing from the performance. First, has the process ever changed an outcome? If no proposal has been killed, no candidate rejected on criteria, no forecast revised upward after looking at the reference class, then it is producing documentation rather than decisions. Second, does the most junior person in the room speak before the most senior? Order of speaking decides how much of the room’s information reaches the table, and it is the cheapest structural change available.

Both tests are uncomfortable to run on your own organisation, which is the usual sign that they are worth running.

Where to Start

Pick the decision you make most often and write down the three criteria it should be judged against, before the next one arrives. That is structured decision making at its smallest, and it is where the measured gains actually come from: the same question asked the same way every time, so the variation you cannot feel stops being free.

The decisions worth protecting this way are the ones with a long tail, where the consequence arrives years after the choice. Retirement structure is the obvious example, and the solo 401k and SEP IRA comparison is a decision most people make once, quickly, on the advice of whoever happened to be in the room. It is worth the afternoon.