Position
Choosing a model is the last decision, not the first
Most of the quality in a working system comes from the components around the model. That is also the case for staying vendor neutral.
Enterprise AI conversations often open with a model selection: which provider, which tier, which context window. The decision feels concrete, and a meeting can settle it.
It also has the least leverage. Take a system that performs badly, swap the model for a stronger one, and the improvement is usually small. Fix retrieval so the right passage reaches the context, and the improvement is often large. The harness decides the outcome.
What the harness includes
The harness is the retrieval strategy and how you chunked the corpus. It is what enters the context and in what order. It is the tools the system can call, how narrowly you scoped them, and what happens when one fails. It is the evaluation suite that tells you whether last week's change helped. None of that is specific to a model, and all of it survives a model change.
Teams that invest there find model upgrades routine. Teams that do not find every model change a small crisis, because nothing in the system tells them whether the new one is better.
| Component | Survives a model change | Typical share of quality |
|---|---|---|
| Chunking and retrieval | yes | large |
| Context assembly and ordering | yes | large |
| Tool definitions and scoping | yes | moderate |
| Evaluation suite | yes | moderate, and it is how you know |
| Prompt wording | partly | small once the rest is right |
| Model choice | no | small |
Neutrality is a technical position
Staying vendor neutral can look like avoiding a decision. It is the opposite. Different work has different profiles: a high-volume classification task and a long-horizon agentic task differ in cost, latency, and failure mode. Committing the whole estate to one provider means accepting the wrong tool for part of it.
Where inference runs matters as well. Some workloads cannot leave a region, a facility, or a device. That constraint removes most of the model market before anyone discusses capability, which illustrates the general rule: the environment decides the architecture.
What to do instead
Design the interface to the model as a replaceable component from the first commit. Keep the evaluation suite independent of the provider. Measure cost per completed task rather than cost per token, because a cheaper model that needs three attempts is not cheaper.
Then choose a model, late, on evidence. It is a smaller decision than it appears, and making it last is what keeps it small.
Bring us the problem.
Tell us the outcome you are trying to create, what you have already attempted, and where the constraints are.
Contact nuperX