If a pilot cannot be killed, it cannot be trusted.

A familiar failure mode is a pilot that begins with permission to try something but no permission to stop.

That imbalance changes the work. Once people have sponsored an idea, selected a tool and announced a pilot, continuation becomes the path of least resistance. A promising demonstration becomes a programme. A programme accumulates cost and expectation. The original question disappears.

Kingsbury’s position is deliberately severe:

If a pilot has no credible way to produce “stop” as its answer, it is not an experiment. It is a roll-out seeking retrospective approval.

A killable pilot is not designed to fail. It is designed so that success means something.

A pilot exists to make a decision

“Explore AI” is not a decision. Neither is “prove the technology works”. Modern models can perform an enormous range of tasks in favourable conditions. The relevant uncertainty is whether a specific intervention is good enough, valuable enough and operable enough in a particular service.

A useful pilot resolves a bounded choice, such as:

• whether to proceed to a controlled production release;
• whether to change the workflow or technical approach and test again;
• whether to stop because the benefit does not justify the risk, cost or organisational change.

Write that decision before building. It determines what evidence the pilot must collect and prevents the output becoming a collection of anecdotes.

The GOV.UK Service Manual advises teams in alpha to focus on their riskiest assumptions, build only enough to test them and expect to discard code or ideas. That discipline is especially important with AI, where a convincing interface can conceal unresolved service risk.

The pilot contract

Before the first build, record eight things. This is the pilot contract: a compact agreement about what the experiment is allowed to prove.

1. Decision

What decision will be made at the end? Name the available outcomes now: proceed, change and retest, or stop.

2. Claim

What do we believe will improve, for whom and why? Make the claim specific enough to be wrong. “The assistant will help” cannot be tested. “Reviewers will identify the same high-risk cases with less handling time and no increase in material misses” can.

3. Baseline and comparator

What happens without the proposed system? Compare the pilot with the current service or the simplest viable alternative, not with an empty space. A model that beats no process may still lose to a clearer form, a rule or a search tool.

4. Scope and sample

Which users, cases and conditions are included? Which are excluded? Use material that represents routine work and important edge cases. A hand-picked demonstration set is useful for debugging, not for making an investment decision.

5. Success threshold

What result is strong enough to continue? Set the threshold before seeing the final numbers. Measure the outcome, not just model behaviour, and combine quality with time, cost and user experience where relevant.

6. Guardrails

Which result forces a pause regardless of average performance? This may be a serious privacy failure, an unacceptable pattern of missed cases, harmful disparity, loss of traceability or an operating cost above an agreed ceiling.

Guardrails prevent a good headline measure from cancelling a consequence the organisation cannot accept.

7. Owner and date

Who makes the decision, and when? The owner must have authority over the service and its risk, not just the prototype. A pilot with no decision date tends to become background infrastructure by habit.

8. Exit

What is removed, retained and learned if the pilot stops? Decide how access will be closed, data handled, suppliers notified, users informed and reusable evidence preserved. Stopping should be an operated state, not abandonment.

Evidence that can say no

An evaluation earns trust when it gives the proposal a fair chance and still allows contradiction.

That requires more than an average accuracy number. Examine the distribution of failures. Separate harmless awkwardness from mistakes that change a decision. Measure the human work around the system: checking, correcting, escalating and recovering. Include the cost and latency of the complete path.

NIST’s voluntary AI RMF Playbook calls for acceptable performance limits and thresholds for bypassing, deactivating or decommissioning a system. For a pilot, evaluation is not a ceremonial score at the end. It is the evidence that connects behaviour to an operational decision.

HM Treasury’s guidance on evaluating AI interventions adds another discipline: design the evaluation with the intervention, define the baseline and describe the comparison with business as usual before implementation. A result cannot show improvement if the counterfactual is invented afterwards.

Useful evidence often includes:

• outcome measures against the existing service;
• task-specific quality reviewed against agreed criteria;
• failures grouped by severity and cause;
• time and cost including human oversight;
• feedback from the people doing and receiving the work;
• observed security, privacy, fairness and reliability concerns;
• assumptions that remain unresolved.

The final item matters. A pilot does not have to answer everything, but it must not hide what it failed to learn.

Three honest endings

Proceed

The threshold is met, guardrails hold and the organisation can operate the service. Proceed does not mean uncontrolled expansion. It means a defined next release with monitoring, ownership and rollback.

Change and retest

The underlying opportunity survives, but the intervention or evidence does not. Narrow the scope, improve the workflow, change the technical approach or collect a more representative sample. State which uncertainty the next test will resolve.

Stop

The benefit is too small, the risk too high, the economics wrong or the service not ready to absorb the change. Record the result and release the team. The decision has created value by preventing a weaker investment.

The GOV.UK Service Standard requires teams to define success and use performance evidence to understand whether a service solves its intended problem. Proceed, change and stop should be equally real when the pilot begins.

Warning signs

Be suspicious when:

• the pilot is described by the tool being used rather than the decision it serves;
• success means only that a model can produce a plausible response;
• the demonstration cases were selected by the build team;
• the baseline will be worked out later;
• quality is measured but the checking labour is ignored;
• no person has authority to stop;
• the pilot quietly enters live work before its review;
• every result leads to a larger phase.

These are not administrative defects. They change the truth the pilot is capable of producing.

The standard

Before Kingsbury calls something a pilot, we should be able to show its decision, claim, comparator, sample, threshold, guardrails, owner, date and exit. At the end, we should publish the result at the level confidentiality allows, including a stop decision.

This makes the work harder to sell and easier to trust.

The point of a pilot is not to protect the idea. It is to protect the organisation from believing the idea without enough evidence.

Sources and further reading

• The GOV.UK Service Manual on how the alpha phase works, including testing risky assumptions, building only enough to learn and deciding whether to proceed.
• The GOV.UK Service Standard on defining success and publishing performance data.
• HM Treasury’s guidance on evaluating AI interventions for defining objectives, a baseline and a credible comparison early.
• NIST’s voluntary AI Risk Management Framework and AI RMF Playbook for risk tolerances, evaluation and controlled deactivation across the system lifecycle. NIST is revising AI RMF 1.0 at the time of publication.