What is an LLM and how to choose a model for a company

An LLM (large language model) generates text by predicting successive tokens from context. That explains why it can produce coherent answers, but it is not a standalone mechanism for checking truth. The risk of confabulation is described by NIST’s generative AI profile.

You will deliberately not find a “best models” ranking here. Rankings can shift quickly, while selection criteria — quality on your task, cost, latency, data terms, and maintainability — are a more stable reference point.

What an LLM can and cannot do by definition

It fits well for: text transformation (summarising, translating, changing format), classification, extracting data from unstructured text, drafting, and turning a question into a system query — after testing quality on real examples.

It does not inherently guarantee: factual or calculation accuracy, knowledge of your data without supplied context, or identical output every run. Protect those requirements with code, sources, and evaluation.

This has a direct design consequence: the model may interpret language, while rules and calculations should remain in controlled code. Code calculates a discount. The model may recognise which discount a customer asks about in an email, and the result then goes through validation appropriate to the cost of error.

Six criteria for selecting a model

1. Task difficulty, not model prestige

Classifying an email into one of five categories often does not require the market’s strongest model. Reviewing a contract for contradictions may need stronger capabilities, but a test on representative documents decides that. Choosing a model “just in case” raises cost without guaranteeing a better result.

The practical rule is: start with a smaller, cheaper model, measure quality on your own examples, and move up only if it is insufficient. Not the other way around.

2. Cost per unit of work, not price per million tokens

A price list gives token cost. What matters to you is the cost of one processed ticket or one generated description. The two can differ substantially when much documentation enters context; material supplied to the model may make up much of the input cost.

Measure the context retrieved by RAG before changing the model. Shorter, relevant material may cut input cost and noise at the same time, but test that effect on your own question set.

3. Context-window size

This is how much text the model accepts at once. A large window simplifies some scenarios, but does not remove the need to select material: unnecessary fragments raise cost and can make the correct basis for an answer harder to distinguish.

4. Latency

In an interactive chat, people usually expect a short response time; overnight document processing can have a much looser limit. These are different requirements for both model and architecture. Providers may offer separate, cheaper batch modes, which should be verified against the current price list.

5. Where data is processed

A provider-hosted cloud model, a model in a chosen region, or a model on your own infrastructure. This is a compliance, risk, and operational-responsibility decision, not just a technical one. Check data purpose and retention, processing location, subprocessors, contract terms, permissions, and incident procedure.

6. Replaceability

The integration layer should limit how much business logic depends on one model interface. An adapter may reduce the change needed to switch provider, but it does not remove the need to re-evaluate quality, security, limits, and response formats.

Model through an API or open weights?

	Model provided through an API	Open-weight model on your own infrastructure
Time to first result	often shorter because infrastructure is ready	often longer due to deployment and infrastructure maintenance
Cost at small scale	often lower	may be higher because of fixed infrastructure cost
Cost at large, steady scale	grows with use	can be lower
Control over data	depends on contract and configuration	greater, but requires your own safeguards
Your work	integration	integration plus infrastructure maintenance

Without special constraints, an API is a useful starting point for measuring quality, use, and cost. Consider your own infrastructure sooner if compliance, data control, latency, or a measurable steady scale require it. “Open weights” do not always mean unrestricted use — check the licence of the particular model. Decide how production data will be processed before using it, not only after a pilot.

What we do not recommend to companies like our clients is training a model from scratch. The data scale and budget rarely justify it; a cheaper test is a ready-made model connected to your own data and measured for quality.

How to measure whether a model is good for us

Public rankings describe general tasks, not yours. A minimum in-house measurement looks like this:

1. Start with 20–50 varied, real examples from your process, including difficult cases and ones the system should refuse to handle. This is a screening sample, not a universal deployment-ready number.
2. Run two or three models on them and count how often the result is correct.
3. Record the cost and time of each run too.
4. Keep this set and run it after every model or prompt change.

Such a small set does not replace full evaluation, but it is a better purchasing filter than a general ranking alone. When a solution goes into production, extend the set with errors and new cases from actual use.

When an LLM is the wrong choice

• When character-for-character repeatability is required. An invoice, transfer, or liability calculation needs code, not a model.
• When input is already structured. If you have a well-kept database, a database query is faster, cheaper, and more reliable.
• When there is no way to check the result. A task where nobody can assess whether an answer is good is not ready for implementation, because you will not notice when it stops working.

Frequently asked questions

Is a larger model always better?

No. A larger model may perform better on some difficult tasks, but it has no guaranteed advantage on your process and may cost more and respond more slowly. Compare candidates on the same case set and choose the smallest model that meets the agreed quality criterion.

How often should a model be changed?

There is no fixed replacement calendar. Evaluate a new model when the current one no longer meets quality, cost, latency, or provider requirements. An adapter shortens some technical work, but decision time depends on re-evaluation and compliance checks. We review this article quarterly.

Can several models be used at once?

Yes. A cheaper model may handle routine cases and a more expensive one only cases that exceed an agreed threshold. Routing adds complexity, so it pays off only when measured savings and quality justify the extra rules, tests, and monitoring.

---

We choose models for the task and budget, not fashion, and build the integration layer so changing a model does not mean rewriting the system. See LLM integrations or tell us what you want to achieve.