“Can we use AI without sending our data to OpenAI?” is one of the most common questions I get from clients in healthcare, legal, finance, and anyone with enterprise customers asking hard security questions.
The answer is yes — open-weight models you can run on your own infrastructure are genuinely capable in 2026. But “private” comes with real costs and trade-offs, and for many businesses a properly configured API is the better choice.
Option 1: Commercial APIs (OpenAI, Anthropic, Google)
Business and API plans from the major providers typically include:
- No training on your data by default
- Data retention controls and, for some plans, zero-retention options
- Security certifications (e.g. SOC 2) and data processing agreements
- Regional data processing options on some platforms and cloud marketplaces
For most business data, this is enough. The bigger real-world risk is usually staff pasting company data into personal consumer accounts — which is a policy and tooling problem, not an API problem. An internal AI assistant on business-grade APIs usually fixes that.
Option 2: Self-Hosted Open-Weight Models
Open-weight models can run on servers you control: your own hardware, a private cloud account, or a dedicated GPU host. Prompts and documents never leave your environment.
Good for:
- Regulated data where contracts or regulators say data can’t go to third-party AI providers
- Air-gapped or on-premise environments
- High, steady volume where fixed GPU costs beat per-token pricing
- Customisation — fine-tuning a model on your domain data
Side-by-Side
| Commercial API | Self-hosted open model | |
|---|---|---|
| Data leaves your environment | Yes (under contract) | No |
| Quality on complex reasoning | Best available | Good, usually a step behind |
| Quality on focused tasks | Excellent | Often close to equal |
| Setup cost | Low | $5,000–$15,000 |
| Monthly cost at low volume | $20–$500 | $500–$3,000 (GPU always on) |
| Monthly cost at high volume | Grows with usage | Mostly flat |
| Maintenance | Provider handles it | You handle updates, scaling, security |
| Time to launch | Days to weeks | Weeks |
The cost row surprises people. At small-business volume, APIs are almost always cheaper — self-hosting means paying for a GPU around the clock whether anyone uses it or not. The details are in the LLM running costs post.
Option 3: Hybrid
The setup I recommend most often for privacy-sensitive businesses:
- Sensitive data (patient records, financial documents, legal files) → private self-hosted model
- Everything else (general questions, marketing, coding help) → top commercial API
- A routing layer decides which model handles each request based on data classification
This gets you privacy where it matters, best-in-class quality where it doesn’t, and a sensible bill. It also makes security and compliance questionnaires — including EU AI Act and GDPR questions — much easier to answer.
How to Decide
Ask three questions:
- Does a contract, regulator, or client explicitly forbid sending this data to a third-party AI provider? If yes, you need private hosting for that data.
- Is your volume high and steady enough that a GPU would be busy most of the day? If yes, self-hosting may be cheaper.
- Do your tasks need top-tier reasoning? If yes, you’ll want a commercial model somewhere in the mix.
If the answer to all three is no, a business-tier API with good data controls is the right starting point. You can always move sensitive workloads in-house later if the architecture is designed for it.
Need AI but can’t compromise on data privacy? Let’s talk. I’ll help you work out whether an API, a private model, or a hybrid setup fits your data, budget, and compliance requirements — and build it.