Custom AI model development: do you really need one?

Roughly once a week someone emails me asking for custom AI model development. The message usually reads: we want our own AI model, trained on our data, so it understands our business. It is a completely reasonable sentence and it is almost never a description of what the company actually needs.

What they need, in the overwhelming majority of cases, is a custom AI web app for their business: a piece of software built around a model that already exists, wired into the systems they already run, doing one job their staff currently do by hand. That distinction is worth five and sometimes six figures, so it is worth twenty minutes of your time before anyone writes a proposal.

Three different things called “a custom AI model”

When two people say “custom model” they are usually describing different projects. Sort yourself into one of these before you talk to a vendor.

Route 1 is a software project with a known shape and a fixed price. Route 2 is a software project with a research tail attached. Route 3 is a hiring decision disguised as a purchase order. If a vendor moves you from 1 to 3 without arguing hard for it, ask why.

“Trained on our data” usually means something else

This is the single most common misunderstanding I run into, and untangling it on the first call is often the most valuable thing that happens all week.

Fine-tuning changes how a model behaves. Its tone, its output format, the shape of the task. You teach it by showing it hundreds or thousands of worked examples of input and correct output.

Retrieval changes what a model knows right now. When someone asks a question, the system searches your documents, your database, your ticket history, pulls the relevant handful of passages and hands them to the model along with the question. The model was never trained on your data and does not need to be.

Nearly every “it should know our products, our policies, our clients” requirement is retrieval. And retrieval has three properties that training does not:

• It updates instantly. Change the price list this morning and the AI is correct this afternoon. A fine-tuned model is a photograph of the day it was trained.
• It cites. The system can show which document an answer came from, which is the difference between something staff trust and something they double-check.
• It respects permissions. You can filter what gets retrieved per user. Training bakes everything into the weights for everyone, permanently.

That last one matters more than people expect. Fine-tune on your full document set and you have created a system that can repeat any of it to anyone. In a company with an HR folder and a salary spreadsheet, that is not a technical detail.

The four tests

Run these in order. If you get a clear no to all four, you want route 1, and the rest of the budget conversation gets much easier.

Test 1 — Can a general model already do this?

Not in theory. Take twenty real examples from last month, paste them into any frontier model with clear instructions, and grade the answers yourself. This costs an afternoon and nothing else. If the general model gets fifteen of twenty right, you do not have a model problem — you have an integration problem, and it is solvable software.

Do this before any vendor call. It reframes the whole conversation, and it is the cheapest piece of due diligence in the entire process.

Test 2 — Do you own labelled data nobody else has?

Fine-tuning needs examples of the right answer, not just raw documents. Ten years of quotes is not a dataset. Ten years of quotes paired with which ones were accepted is. Three thousand support emails is not a dataset. Three thousand support emails tagged by category and resolution is.

Most companies discover at this point that the labelling work is the project, and the model was never the hard part. That is a genuinely useful thing to find out in week one rather than week nine.

Test 3 — Do latency or unit cost force a smaller model?

This is the most legitimate reason an SMB ends up with a fine-tuned model. If you are classifying hundreds of thousands of items a month, a small fine-tuned model running on modest hardware can be dramatically cheaper per call than an API to a large one, and fast enough to sit inside a real-time flow.

The arithmetic is simple and you should do it explicitly: your monthly volume, times the API cost per call, against the build plus hosting plus retraining cost of owning a model. Below a certain volume the API wins so decisively that it is not worth a meeting. Above it, this becomes the strongest argument on the table.

Test 4 — Must it run with no external connection?

Air-gapped environments, an on-premise requirement your regulator or your largest customer wrote into a contract, or a plant with unreliable connectivity. If nothing may leave the building, hosted APIs are off the table and a self-hosted open model — fine-tuned or not — becomes the only route. That is a real constraint and it changes the answer.

Note what is not on this list: “we want to own our AI”. You own the application, the data, the prompts, the retrieval index and the source code in route 1 too. The model is a supplier, and a swappable one. That is a feature, not a compromise — the model you would have trained this year would be behind the general models by next.

What owning a model actually costs after launch

Build cost is the part everyone quotes. The part that decides whether a company regrets the decision is what happens in the following two years.

• Evaluation. Once you own the model you own the question “is it still any good?” That needs a held-out test set and someone who runs it. Without it, quality drifts and nobody notices until a customer does.
• Drift. Your business changes. New products, new policies, new language. A fine-tuned model degrades quietly against a world that moved on.
• Retraining. Every meaningful change means gathering new examples, re-labelling, retraining, re-evaluating and redeploying. Budget for it as a recurring cost or it will not happen.
• Hosting. Weights need somewhere to run, usually on hardware you would not otherwise rent, sized for peak rather than average load.
• The opportunity cost. General models improve every few months for free. A model you froze last spring does not.

None of this makes fine-tuning wrong. It makes it a commitment. Take it on when tests 2, 3 or 4 gave you a clear yes, and not because it sounds more serious in a board slide.

What the good version looks like instead

A concrete shape, because “a custom application” is too abstract to price against. A typical build I ship for a 40-person company:

Four layers, one fixed price, typically €4,950 to €49,950 depending on how many systems it touches and where it is hosted. The full arithmetic behind those bands, including the multipliers, is published on the pricing benchmarks page, and the method I use to reach a number is in how to estimate cost and time.

The reasoning layer — the part people imagine is the whole project — is usually 20 to 30% of the work. The retrieval, the integrations and the audit trail are the rest. Which is exactly why a project that starts as “train us a model” and is priced as “train us a model” tends to run long: the model was never the expensive part.

Five questions for anyone proposing to train you a model

1. What did the general model score on my twenty examples? If they have not run that test, they have not established that training is necessary.
2. Which labelled dataset are you training on, and who is producing the labels? If the answer is “your documents”, they mean retrieval and are calling it training.
3. What is the evaluation set, and what score counts as done? A trained model without an agreed pass mark cannot be signed off, only argued about.
4. What does retraining cost, and how often? Get it in the proposal as a recurring number, not a footnote.
5. If a better general model ships mid-project, what happens? A good answer describes swapping it in. A bad one explains why that would be inconvenient.

These are not gotchas. Any vendor who genuinely should be training you a model will have crisp answers ready and will be pleased you asked. More questions in that vein are in questions to ask before hiring an AI developer.

So: do you need one?

If you got a clear yes on test 2, 3 or 4 — genuinely proprietary labelled data, volumes high enough that unit cost dominates, or a hard offline requirement — then yes, and it is worth doing properly with the ongoing costs written down from day one.

If you did not, you want a custom AI web app for your business, built on a model that already exists, and you want it in weeks rather than quarters. Same business outcome, an order of magnitude less money, and no research risk.

Either way, run test 1 this week. Twenty real examples, one afternoon. It is the cheapest decision you will make in this entire process, and it is the one that determines the size of every number that follows.

Related reading

• Custom AI Development in 2026: Real Cost, Timeline & How to Pick a Developer — the comprehensive guide
• Custom AI vs off-the-shelf — when a subscription is genuinely enough
• How to spec a custom AI system — writing the one-page brief
• How custom AI development actually works — six-stage methodology