Should you build or buy LLM features?

What are the options for adding LLM features?

"Build or buy" sounds like a binary choice. In practice there are four options, and each one moves more of the work, and more of the control, onto your team.

• Buy a SaaS product. An off-the-shelf copilot, meeting assistant, or support tool. Fastest to start, least control, and you live with the vendor's roadmap.
• Buy a platform and configure it. A tool that lets you connect your own data sources, set permissions, and design workflows without writing much code. More fit to your needs, still bound by what the platform supports.
• Build on model APIs. Your engineers call model APIs directly and build the retrieval, prompts, tools, and interface themselves, often with a vector database or search index for your company data. Full control over the experience and the data flow.
• Build more deeply. Self-hosted models, fine-tuning, or a custom agent framework. Only worth it when you have a clear reason, such as strict data residency, unusual quality needs, or a feature that is the product itself.

How do you decide whether to build or buy an LLM feature?

Score each use case against the same criteria. The answer often differs between, say, an internal knowledge search and a customer-facing agent.

• How differentiating is it? If customers choose you partly because of this feature, owning it matters. If every company needs the same thing, buy it.
• Data and access control. Search over company data must respect who can see what. Check whether a product honors your existing permissions, or whether everyone would see everything.
• Integration depth. Reading documents is easy to buy. Taking actions in your own systems, as agents do, usually needs custom work.
• Quality and evaluation needs. If a wrong answer is costly, you need to test outputs against your own examples. Ask whether a vendor lets you do that, and whether you can see why it answered the way it did.
• Team skills. Building means someone owns prompts, retrieval, evaluation, and monitoring after launch. If nobody on the team can, buying is safer for now.
• Lock-in and terms. Look at data export, how prompts and configuration can be moved, and what the vendor may do with your data.
• Privacy and compliance. Know where data goes, whether it is retained or used for training, and what your customers' contracts and regulators require.

What are the hidden costs of building or buying LLM features?

The license fee or the first build is the visible part. The ongoing work is what surprises teams.

When you build, you own evaluation: a set of test questions and expected answers that you rerun whenever something changes. You own monitoring, so you notice when answers drift or users stop trusting the feature. Model providers update and retire models, and prompts that worked last quarter can behave differently. Integrations with your own systems need upkeep as those systems change.

When you buy, you still own more than you expect. Vendors change prices, plans, and terms, sometimes with short notice. Their model changes can shift quality without warning, and you may have no way to test it. Connectors to your data still need someone to maintain permissions and fix sync failures. And if the vendor is acquired or shuts down, you inherit a migration.

Either way, plan for an owner. An LLM feature without a named owner degrades quietly.

When should you switch from buying to building, or back?

Buying first is often the right way to learn. A purchased tool shows you what users actually ask, where quality falls short, and which data matters. That is useful input to a build later.

Consider building when the product blocks you: it cannot reach the data you need, cannot respect your permissions, cannot be evaluated against your own cases, or its terms no longer work for you. Also consider it when the feature becomes central to how you sell.

Consider moving back to a product when a commodity has caught up with what you built, and your team is spending its time maintaining something that no longer sets you apart. Retiring custom code is a legitimate win.

Make switching cheaper from the start. Keep your evaluation set, prompts, and data pipelines in your own hands, whichever path you take.

A simple checklist for build vs buy

Answer these for each use case. Mostly "yes" in the first group points to buying; any strong "yes" in the second group points to building or a configurable platform.

If you are unsure where your data and team stand, the AI Readiness Scorecard gives you a quick baseline before you commit either way.

• Lean toward buying: Is this a common need across many companies? Is a good-enough answer acceptable? Does it only need to read data, not act on it? Does a product already meet your privacy and compliance requirements?
• Lean toward building: Does it set you apart with customers? Must it enforce detailed, per-user permissions? Does it need to take actions in your systems? Do you need to test quality against your own cases? Do you have, or can you hire, someone to own it after launch?
• Either way: Who owns it? How will you measure quality? What is your exit plan?