J. Paul is a London based designer and researcher with expertise in Speculative Design, Service Design, Design Research, and Strategy.
The most common question we get from finance teams, procurement leads, and heads of strategy is the one the discipline has the hardest time answering: how do you know it worked?
The honest answer — that you cannot evaluate a future that has not happened yet — is unsatisfying and also wrong. You can evaluate speculative design rigorously. You just have to evaluate the right thing. Most attempts to measure it fail because they measure the future, when the value is in the present.
This essay sets out the framework we use to make speculative design evaluable, the four metrics we track on every engagement, and the trap to avoid when reporting it upward.
The category error: measuring the future instead of the present
Imagine commissioning a piece of strategic research and then, eighteen months later, evaluating whether it was good research by checking whether its forecasts came true. You would never do this. You would evaluate the research by whether it changed how the organisation decided things during the planning cycle it informed.
Speculative design is the same. Its value is not produced by the accuracy of the futures it depicts. It is produced by the decisions those futures make available to the present. The category error is to measure speculative work by whether the future arrived, when the right measurement is whether the present moved.
Once this shift is made, the work becomes measurable in the same way strategic research, scenario planning, customer research, and innovation portfolio work are measured: by the changes in present-day decisions that can be traced back to it.
Four metrics we track on every engagement
We track four things on speculative design engagements. They are not a balanced scorecard; they are a hierarchy. The further down the list you can demonstrably measure, the more value the engagement produced.
1. Question quality
The most underrated metric. Before the engagement, what questions was the leadership team asking about the future of the category? After the engagement, what questions are they asking?
Better questions are the first product of speculative work. They are also the most leveraged: a leadership team that asks better questions for the next five years is worth, conservatively, several quarters of mediocre answers. We capture this with a simple before/after question audit at engagement kickoff and engagement close. It is the metric finance teams find least intuitive and most defensible after six months.
2. Decision shifts
The middle of the hierarchy. What decisions, made during or shortly after the engagement, were influenced by the work? Not "inspired by" — influenced by, traceable in artefacts: revised briefs, changed budget lines, modified hiring, altered partnership structures, new policy positions.
We track this by maintaining a decision log during the engagement and a follow-up audit at 6 and 12 months. Even a small number of named, traceable decisions (three to five) is enough to demonstrate that the work changed organisational behaviour. Anything more is a strong result.
3. Strategic optionality
The most strategically valuable metric, and the most demanding to measure. Did the work expand the set of strategic options the organisation considers viable?
This matters because innovation portfolios suffer not from a shortage of good options but from a shortage of considered options. Organisations rule out futures early, often unconsciously, by deciding what is "not for us." Speculative design expands the considered set by making previously-unthinkable futures thinkable. We measure this by tracking the optionality space at the beginning and end of the engagement — what was on the table, what was ruled out, what was newly on the table.
The dollar value of optionality is the value of being able to make a different decision later. It is real. It is also the metric most likely to be invisible to a quarterly review.
4. Avoided cost
The most concrete and the least common. Did the work prevent a decision that would have been costly in retrospect?
Avoided-cost stories are powerful when they exist, but they are over-claimed in the speculative design world. Most engagements do not produce a clean avoided-cost story. When they do, the story tends to involve a strategic commitment that was paused or redirected following the work — a category bet not made, a partnership not signed, a market entry sequenced differently — and the avoided cost is the counterfactual spending that would have followed the original direction.
We are careful with this metric. Over-claiming avoided cost is the fastest way to make a speculative design practice non-credible with finance. We report it only when the counterfactual is well-documented and the link to the engagement is unambiguous.
What we do not measure
A short list of things we have learned not to track.
- •Number of artefacts produced. Bigger decks are worse decks. Measuring output by volume rewards the wrong work.
- •Stakeholder satisfaction immediately after the engagement. Speculative work that everyone loved on the day usually did not provoke enough to change anything. Measure satisfaction at 6 months, not at 6 days.
- •Whether the speculative scenarios "came true." The category error. The value was in the present.
- •Press coverage of the artefacts. Pleasant; not the product.
A note for procurement
The ROI conversation in procurement often gets stuck because speculative design is being priced against the wrong comparator. It is not a cheaper consulting engagement. It is not a more expensive design project. The right comparator is strategic research that shifts decisions, and on that comparison the discipline prices favourably.
A typical SOCD engagement runs from a four-figure workshop to a six-figure programme. The shift in question quality alone, traceable through a question audit, is generally enough to defend the line in a quarterly review. Decision shifts and avoided-cost stories, when they exist, are bonus.
A note for designers
If you are doing this work and are not tracking these four metrics, your case for the discipline is weaker than it needs to be. The artefacts will eventually go out of date; the decisions will not. Make the decisions legible — in writing, in the log, in the follow-up — and you will have a case that survives the leadership change that always comes.
Where to take this next
If you want to commission a speculative design engagement and want to ensure it is measurable, start with the framing workshop — name the decisions the work is meant to inform before any artefacts are made. If you want training in this evaluation framework as part of a full practitioner workflow, our Speculative Design Masterclass covers it.
Key takeaways
- •Speculative design is measurable, but not by whether the future arrived. The value is in the present.
- •Four metrics, in order of increasing concreteness: question quality, decision shifts, strategic optionality, avoided cost.
- •Track decisions in a log during the engagement and audit at 6 and 12 months.
- •Do not over-claim avoided cost. Over-claiming is the fastest way to lose finance credibility.
- •The right comparator for pricing is strategic research that shifts decisions, not design or consulting.