Most AI proofs of concept succeed. That is the problem. They are built in conditions arranged to make success likely, on clean sample data, without permissions, integrations or the awkward cases, and the result transfers to production about as well as a driving test taken in an empty car park.
MIT's Project NANDA studied enterprise generative AI adoption across more than 300 deployments, 52 case studies and 153 leadership surveys, and found that 95 percent of enterprise generative AI pilots produced no measurable financial return. The cause identified was integration and organizational learning rather than model capability. McKinsey's State of AI survey reports adjacent numbers: 88 percent of organizations use AI somewhere, about a third have begun scaling it, and 7 percent describe it as fully scaled.
Those numbers describe a very large population of proofs of concept that worked and went nowhere. Avoiding that outcome is mostly a question of what you decide before starting.
Write the success criterion as a number, before you start
The single most common defect in an AI proof of concept is that success was never defined, so the result is a demonstration that everyone interprets according to what they already believed.
The criterion has to be a number with a threshold attached. Correct extraction on at least 90 percent of a sample of 200 real documents. Correct classification often enough that a human reviewing only the low-confidence cases reviews fewer than a quarter of them. A response produced within the time a user will actually wait.
Write it down, agree it, and agree in advance what happens if the result lands below it. A proof of concept that cannot fail is not a test.
Use the ugly data
The sample matters more than the model. Take real records, including the ones that are incomplete, badly formatted, unusually long, in the wrong language, or produced by a process that changed three years ago.
Teams tend to test on the clean subset because it is easier to assemble, and the clean subset is exactly the part that was never going to be hard. The value of a proof of concept lives entirely in what it tells you about the difficult cases, and their frequency in your real data is the number that determines whether the whole idea is viable.
Sample size matters less than sample honesty. Two hundred genuinely representative records tell you far more than five thousand tidy ones.
Do the data access, even though it is boring
The part that decides whether a proof of concept transfers is whether it reached the real data through the real path.
An extraction that works on a folder of exported files has not established that you can get those files continuously, with permissions applied, when the source system is under load. That gap is where the 95 percent figure comes from. The model was never the constraint.
If reaching the real data is genuinely too expensive for a first exercise, that fact is itself the most valuable finding, and it belongs at the top of the report rather than in a footnote about limitations.
Build the evaluation harness now
A model feature without a way to measure output quality can be built once and then never safely changed. Prompt changes, model upgrades and new data sources all become gambles, because nobody can say whether the output got better or worse.
The harness does not need to be sophisticated. A fixed set of representative inputs, the expected outputs, and a repeatable way to score them is enough, and it takes a small fraction of the effort of the thing it protects.
Build it during the proof of concept rather than after. It is what turns a one-off result into something you can hold onto, and it is the difference between a proof of concept whose findings survive and one whose findings expire when the model version changes.
Decide what happens when the output is wrong
Correct most of the time is the normal condition for these systems, so a proof of concept should establish what happens the rest of the time.
The specific questions: does a person see the output before anything irreversible happens, can they tell easily that it is wrong, and what does the system do with low confidence. A design where wrong outputs are cheap to catch can tolerate a much lower accuracy threshold than one where they are not, which can change the entire viability assessment.
This is also where the accuracy target actually comes from. An abstract accuracy number means nothing without knowing what a mistake costs.
Measure the running cost against real volume
Inference is a running cost that scales with use, which puts it in a different budget category from the build.
A proof of concept naturally runs at trivial volume, so extrapolate rather than reporting what it cost. Take the cost per operation, multiply by realistic monthly volume, and check the result against what the process costs today. That comparison occasionally ends the project, which is a good outcome to reach after two weeks rather than after a year.
Keep the code disposable, keep the findings
Proof of concept code is written without the error handling, permissions, monitoring and structure that production needs. Promoting it carries those omissions forward, and the shortcuts stop being visible once the system is running.
What you keep is the evaluation harness, the representative dataset, the documented data access path, and the measured numbers. Those are the expensive parts and they transfer directly into a real build.
What it costs
An AI proof of concept done properly, including real data access and an evaluation harness, sits in the $5,000 to $15,000 range with us, toward the higher end when the data is genuinely difficult to reach. A version that skips the data access is cheaper and, on the evidence above, usually worthless.
Scoping which question is worth answering starts at $3,000 and is credited toward whatever follows. If the proof of concept succeeds against a real criterion, a production build starts at $18,000, and the harness and data path built during the exercise carry straight over.
Frequently asked questions
How long should an AI proof of concept take?
Two to four weeks in most cases, with the majority of that spent on data access rather than on the model. Anything much longer usually means the question was too broad, and anything much shorter usually means the real data was not involved.
What accuracy is good enough?
It depends entirely on what a mistake costs and how easily it is caught. A workflow where a person reviews flagged cases can run usefully at accuracy that would be unacceptable in a system acting alone. Define the threshold from the consequence, not from a benchmark.
Should we test several models?
Test more than one, since it is cheap once the harness exists, and expect the differences to be smaller than the difference made by the data you feed them. Model choice is rarely the deciding factor, which is the consistent finding across the research above.
Can a proof of concept become production?
The findings should. The code should not. Proof of concept code lacks the handling that production needs, and promoting it means carrying those gaps into a system people depend on.
Why do so many AI pilots fail?
MIT's 2025 research put the figure at 95 percent producing no measurable return, and attributed it to integration and organizational learning rather than to models. Most pilots avoid the difficult data work, succeed on that basis, and then cannot be scaled because the difficult work was never done.
Where to start
If you have an AI idea and want it tested against a real criterion rather than demonstrated, the first step is picking the question and defining what would count as a pass. Our audit does that, starting at $3,000 and credited toward the work.