The pilot is over. The decision is not.

The technology partner has a rollout proposal open on screen. The project sponsor says employees are saving time, but Finance cannot see any savings. Operations is still trying to determine how much checking, correction, and support underpin the reported improvement.

By the end of the meeting, you are expected to decide whether the business should invest further.

Scale it? Change the use case? Keep it with one team? Run another test? Or stop?

This is where visible AI activity can outstrip business judgement. The pilot has happened. The evidence may still be unclear. The real question is not whether the technology is impressive or whether people enjoy using it. It is whether this use is helping the business, how strong the evidence is, and what commitment that evidence warrants next.

You don’t need perfect proof before starting to learn. A simple, reversible test might help. However, a pilot should lead to a decision, not just a temporary pause between enthusiasm and accountability.

Start With the Decision, Not the Technology

McKinsey’s April 2026 AI Transformation Manifesto offers valuable insights into this topic. It outlines 12 enterprise themes based on six core capabilities, emphasising that merely having accessible technology is insufficient for a sustained advantage. True advantage hinges on how organisations leverage technology to address business challenges, with success supported by effective business leadership, operational changes, adoption, trust, and ongoing learning.

That direction is sound. A licence gives you access to a tool, but it does not give you operating capability.

An operating capability is the repeatable ability to use the technology in real-world work, with the right people, workflow, data, support and judgement. A demonstration may show that a tool can produce an answer. It does not show that your business can use that answer reliably, repeatedly or economically.

McKinsey’s article is an enterprise transformation thesis, not causal proof of a universal SME method, return or staffing model. The practical lesson for a smaller business is not to replicate an enterprise programme with fewer people. It is to focus effort on a problem that matters and make the smallest responsible commitment that can yield a useful answer.

The SME context already makes that discipline important. In the OECD’s 2025 report Generative AI and the SME Workforce, a weighted telephone survey of 5,232 businesses with up to 249 employees found that 31% were using generative AI, either by the respondent or a colleague. That does not mean 31% had a company-wide AI programme. It means someone in the business reported use.

Non-adopters reported concerns about copyright, legal or regulatory issues, data entered into models, and employee skills. The survey was conducted from 14 October to 6 December 2024 in Austria, Canada, Germany, Ireland, Japan, Korea and the United Kingdom. It did not include Singapore or Australia. These were self-reported concerns, not legal findings or evidence of financial return.

For an SME, the sensible response is neither paralysis nor an enterprise-sized programme. Explore where learning is affordable and reversible. Focus implementation effort where the opportunity matters. Buy specialist capability where it makes sense. Keep business judgement accountable within the company.

Do Not Let One Word Do Four Jobs

Back in the meeting, the word “success” is probably doing too much work.

Someone says the pilot was successful because people used it. Someone else says it was transformational because the workflow changed. A third person says it saved money because a task became faster. Those statements may describe four different outcomes.

AI activity asks: are people using the technology? Licences, prompts, pilots and experiments show that something is happening. They may create useful learning. They do not show that the business is better.

Business improvement asks: has a relevant outcome changed? The result might concern customer response, quality, error rates, risk, speed, capacity, learning or financial performance.

To judge that, you need a credible starting comparison, often called a baseline. That might be the same team’s performance before AI, similar work completed without AI, or another group doing comparable work. The comparison must be close enough to answer a simple question: better than what?

A before-and-after number is not automatically credible. Work volume, staffing, customer mix, seasonality or another process change may have shifted at the same time. You do not need a laboratory experiment for every business trial, but the comparison should be strong enough for the decision you are about to make.

Transformation asks a different question: has AI changed how the business operates, decides, serves or learns in a durable and material way? Material means significant enough to matter to this business, not a universal percentage. A useful assistant for one person may streamline a recurring task without transforming the company. There is nothing wrong with that. Transformation is not a prize every deployment has to win.

Success is the judgement of whether the change was worthwhile after considering its intended outcomes, relevant costs, consequences, controls, and uncertainty. A business can make a real, durable change and still discover that the quality, customer impact, risk or economics are unacceptable. Transformation describes the nature of the change. Success judges the result.

These are separate lenses, not compulsory stages. A durable, material change can be a transformation and still fail. A local deployment can succeed without transforming the business. Keeping the judgements separate prevents you from dismissing useful improvements because they are not grand enough, or celebrating transformation before anyone has shown that it worked.

What the Research Can and Cannot Tell You

At this point, someone in the room will usually ask: what should we reasonably expect AI to do?

The honest answer is that there is no single number for “AI productivity”. The result changes with the work, the users, the tool, the workflow and the way performance is measured.

Can AI improve a real job? Yes, in some settings.

In the peer-reviewed study Generative AI at Work, Erik Brynjolfsson, Danielle Li and Lindsey Raymond examined 5,172 human customer-support employees at one organisation. The AI assistant did not take over the customer conversation. It monitored the chat and offered real-time response suggestions; the employee remained responsible and could ignore or edit them.

Access was introduced to different employees at different times rather than to everybody on one day. This staggered introduction allowed the researchers to compare performance as access changed.

The main outcome was issues resolved per hour: the number of customer conversations an employee successfully resolved in an hour. That measure combined handling time, the ability to manage multiple chats, and issue resolution. Access to the assistant increased the number of issues resolved per hour by about 15%. The largest gains went to newer and lower-skilled employees. Among the highest-skilled group, some speed measures improved slightly, while resolution rates and surveyed customer satisfaction declined slightly.

That is useful evidence for a specific job in a specific setting. It does not guarantee that your company will gain 15%, reduce headcount or achieve a financial return. It also shows why an average can obscure a practical management question: who benefits from the tool, and who may be hindered?

Can AI also make experienced people slower? In another setting, yes.

In its study Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR examined 16 experienced software contributors who completed 246 real issues in mature open-source repositories. In plain English, these were established software projects with publicly available code familiar to the participants. Each issue was randomly assigned to allow or disallow AI tools. Participants recorded total implementation time, including work before and after the review, rather than counting only how quickly an initial piece of code was implemented.

With access to tools available from February to June 2025, contributors took 19% longer in that setting. The 95% confidence interval ran from 2% to 39% longer. A confidence interval is a statistical range that shows the uncertainty around an estimate: the central estimate was a 19% slowdown, but the exact size was not known precisely. Participants expected to work faster and later believed they had done so.

METR’s February 2026 update explained that a follow-on experiment could not provide a reliable current estimate. Some developers declined to participate when AI-disallowed work was required, weakening the comparison, and the way working time was recorded became less reliable when work occurred in parallel. METR judged that early-2026 tools probably helped more than the earlier tools, but the size of the effect remained unresolved.

The lesson is not that AI slows developers. It is that confidence can run ahead of measurement, a result from experienced developers in familiar codebases does not automatically apply elsewhere, and a historical result should not be stretched across newer tools and different working conditions.

Does widespread use mean firm-level improvement is already visible? Not necessarily.

In the 16 March 2026 revision of the NBER working paper Firm Data on AI, 69% of almost 6,000 surveyed senior executives in the United States, United Kingdom, Germany and Australia said their firms used at least one listed AI technology. The matched survey questions were fielded primarily between November 2025 and January 2026. More than 90% reported no employment impact over the previous three years, and 89% reported no productivity impact. In the paper, productivity was defined as sales per employee, a broad firm-level measure rather than the speed or quality of a single task. The same respondents expected productivity to rise by 1.4% over the next three years.

The March 2026 document was a working paper: a public research paper that could still be revised and was not presented as a final, peer-reviewed journal article. The findings were based on executive self-report, meaning the responses came from executives rather than an audit of company records. The 1.4% figure was an expectation about the future, not a realised result, and the survey design did not, by itself, show that AI caused the reported outcomes.

Use, expectation and realised improvement are different things. Research can help you ask better questions and avoid easy assumptions. It cannot make the decision for your company.

Put the Evidence to Work in the Quotation Team

Now bring the question back to the hypothetical business.

Suppose the pilot sits in the quotation team. Before the trial, the business records how long comparable quotes take from request to approved output, how often they require correction, how quickly customers receive them and what happens to conversion. That becomes the starting comparison.

The team then uses an AI assistant. Total staff time per quote, including checking and correcting the output, falls from 90 minutes to 60 minutes while the approved quote still meets the same quality standard.

That is an operational improvement. The team has released 30 minutes of capacity on each comparable quote. Released capacity means the same people now have time to take on other work. It is real even if payroll does not change.

A nonfinancial improvement may also be real. The team might respond to customers sooner, reduce a backlog, improve consistency or create more resilience during busy periods. Those outcomes matter. Describe and measure them as service, quality or capability improvements rather than automatically converting them into revenue.

A demonstrated financial benefit requires another step. Overtime might fall. External quotation support might no longer be needed. The team might produce more quotes through an existing constraint, and those additional quotes might create measurable contribution. Capacity might be redeployed to work that produces an evidenced financial result.

Until one of those mechanisms occurs and is observed, the extra capacity is real; a financial benefit has not yet been demonstrated.

The checking burden matters as well. The relevant measure is the whole job, not the speed of the first draft. Time spent verifying facts, correcting errors, gaining approval, answering user questions and supporting the system belongs in the operating and cost picture. A tool that drafts quickly but requires extensive review or rework may yield little net improvement.

Licences are only part of the investment. Relevant cost may also include integration, configuration, testing, training, support, monitoring, management time, rework, transition and eventual switching or exit. A time saving should not be counted once as spare capacity and again as financial contribution unless the business can show that both outcomes occurred without overlap.

This is not an argument against return on investment. It is what makes a return claim believable.

Work the Decision, Not the Technology

Most owners do not need another AI maturity model. You need a practical way to move from an idea to a defensible decision.

Six working functions can help. They are not six compulsory stages, documents or workshops. They can overlap, repeat or end early.

• Select. Choose the business problem, constraint, decision, workflow or learning question that deserves attention. State the intended outcome and compare AI with process simplification, existing software, outsourcing and other credible alternatives. For the quotation team, the governing question is which worthwhile problem the business is trying to solve, not where it can use AI.
• Bound. Set the limits before the test begins. Define who may use the tool, what data may be entered, what actions it may take, what still needs human approval, how long the test will run, who owns the decision and how the business will stop or fall back if the use does not work. The greater the possible consequence, the tighter and more explicit the boundary should be.
• Prove. Test the tool and the redesigned workflow against the starting comparison. Measure the intended outcome, quality, errors, rework, checking effort, support burden, cost and relevant risk. “The tool produced an answer” is not enough if people spend the time saved correcting it, or if the result fails under normal working conditions.
• Adopt. Put the useful change into repeated work. Clarify roles, train users, provide support, maintain controls and monitor what happens when the original pilot champion is unavailable. Occasional use by an enthusiastic individual is activity; adoption means the workflow can operate consistently as part of normal work. Adoption can begin during proof and may rationally remain local.
• Establish Value Evidence. Keep the status of the claim honest. Separate what the business intended, what it forecast, what it observed, what it can credibly attribute to the change, what it has actually realised and what continues under normal conditions. Record the type of value, the time period, the full relevant cost, competing explanations and what remains uncertain.
• Decide. Choose the commitment justified by the evidence instead of allowing a pilot or licence to continue by default. A decision can be made at any point; the right answer is not always expansion.

Proof may reveal that the wrong problem was selected. Adoption may expose a support burden that changes the economics. A decision to contain or stop may be correct before a wider rollout is considered. The functions are useful because they keep the work connected to a business decision, not because every initiative must march through them in order.

You Can Buy Capability; You Still Own the Decision

An SME does not need to build every AI capability internally. A vendor, implementation partner, or specialist adviser may be the most rational way to obtain technology, integration, expertise, support, or assurance.

Three decisions still need to remain clear.

The business decision concerns the outcome being pursued, the acceptable data or operating exposure, and who within the company can decide whether the initiative continues, expands, or ends.

Capability delivery is about who supplies the technology, implementation, integration, specialist knowledge, support and assurance. The answer may be an internal team, a supplier, a partner or a mixture.

The investment decision is about what commitment the business should make now, given the evidence, cost, uncertainty and alternatives.

Contracts and law will allocate responsibilities differently. The management point is simpler: buying technology or expertise is compatible with retaining business judgement. A supplier can recommend expansion. Your business still has to decide whether expansion is justified.

When the Pilot Ends, Scale Is Only One Answer

The familiar AI story is pilot, prove and scale. Real decisions are more varied. Think of these as six honest answers to the meeting.

• Scale when the result is valuable, repeatable, supportable and economical at a wider scope. For the quotation team, that may mean extending the assistant to another team or more quote types only after the wider operating case is checked.
• Pivot when the current direction is wrong, but the underlying opportunity remains worth pursuing. The business might stop using AI to draft the entire quote and instead use it to extract information or to prepare a preliminary checklist for human review.
• Contain when the deployment is useful at its present scope but wider rollout is not justified. It may remain with one team, one quote category or one group of trained users, with support and review kept in place.
• Defer when a named dependency is not ready. The business might wait for pricing data to be cleaned, an integration to be completed or a responsible owner to be appointed. A defer decision needs a trigger or review date; it is not permission for an indefinite pilot.
• Retest when a specific uncertainty remains, or a material condition has changed. The business might run another bounded test to learn whether quality holds on complex quotations or whether a changed tool reduces the checking burden. Name the uncertainty and the stop condition in advance.
• Stop when the case is weak, the exposure is unacceptable, or the economics fail. Stopping means deliberately retiring the use, properly handling access and data, returning to a fallback process, and preserving what the business learned.

The distinctions matter. Pivot changes direction. Contain keeps a useful use deliberately local. Defer waits for a named dependency. Retest gathers evidence about a specific uncertainty. Stop closes the use. None of those decisions is the same as drifting on because nobody wants to call the pilot finished.

Contain is particularly underrated. A tool may work well for one team and still be a poor candidate for company-wide rollout. Wider scope brings different users, data, support needs, failure consequences and economics. A successful local use is evidence for that use. It is not a permission slip for the whole business.

Return to the Meeting

The rollout proposal is still open. The technology partner is still waiting. The project sponsor still believes the pilot worked.

Before the meeting ends, ask the team to put one page in front of you. It should show:

• The business outcome the initiative was intended to change and the comparison used to judge improvement.
• What changed, for whom, and whether anyone was helped, hindered, or exposed to a new burden.
• The checking, correction, support and full relevant cost required to make the use work in normal operations.
• The value actually realised, the remaining uncertainty, and the next recommended decision.

Do not accept “people are using it” as proof of improvement. Do not accept “it saves time” until the entire job, including checking and correction, has been counted. Do not accept “scale” as the default merely because the pilot has ended.

The answer may be Scale, Pivot, Contain, Defer, Retest or Stop. The decision need not be dramatic. It must be explicit. Name the owner, the evidence required, the next review point and the condition that would change the decision.

If the team cannot produce that page, the initiative may still merit another bounded test. It does not yet warrant a larger commitment.

Your job is not to prove that AI works. It is to decide whether this use has improved something that matters, under conditions the business can support, and what the evidence warrants next.

That is the ROMA Reality: visible AI activity earns attention; evidenced business improvement earns the next commitment.