How to choose the right LLM for enterprise use cases (August 2026 Update)

Choosing an LLM for enterprise use is no longer "just use GPT." The model market in 2026 has fragmented further. OpenAI's GPT-5.6 family (Sol, Terra, Luna) unified its general and coding lines into three effort tiers. Anthropic's Claude 5 lineup (Opus, Sonnet, Haiku, and the newer Fable tier) pushed context windows to roughly 1M tokens across the board. Gemini 3.1 Pro remains Google's multimodal workhorse. Open-weight models like DeepSeek V4 and Meta's Muse Glimmer now match frontier-era performance at a fraction of the cost. Here's how to choose. For the architectural layer that ties models together, see our AI orchestration platform guide.

The major models

Menlo Ventures' 2025 State of Generative AI in the Enterprise report, based on a survey of 495 U.S. enterprise AI decision-makers, found that Anthropic (40%), OpenAI (27%), and Google (21%) together account for 88% of enterprise LLM API usage - yet only 16% of enterprise AI deployments qualify as true autonomous agents. The model matters, but it's rarely the deciding factor. Architecture, routing, and prompting almost always explain the gap.

GPT-5.6 (OpenAI)

Best for: General-purpose enterprise tasks, broad platform integration.

GPT-5.6 reached general availability on July 9, 2026, replacing GPT-5.5 as OpenAI's flagship. It ships as three effort tiers under one architecture - Sol (maximum capability), Terra (balanced), and Luna (fast and cheap) - rather than a single fixed model, and all three now share a 1.05M-token context window. The tooling community remains the largest: most AI tools and frameworks support OpenAI first.

Strengths:

• Broad capability across text, code, analysis, and creative tasks, tiered by effort level instead of a single fixed model

• Largest community of tools, integrations, and developer resources

• A 1.05M-token context window on every tier, including the budget Luna model

• Strong function calling, structured output, and Agents SDK integration

Limitations:

• Data privacy concerns for sensitive industries (data is processed on OpenAI's infrastructure)

• Less transparent about training data and model behavior

• Pricing can escalate quickly at high volumes without intelligent routing across the three tiers

Pricing per million input/output tokens: Sol $5/$30, Terra $2.00/$12.00, Luna $0.20/$1.20 (OpenAI cut Terra's and Luna's prices in late July 2026).

Claude Opus 5 / Sonnet 5 (Anthropic)

Best for: Agentic coding, long-document reasoning, safety-sensitive applications.

Claude Opus 5 is Anthropic's most capable model for complex reasoning and autonomous coding tasks, with Sonnet 5 as the balanced mid-tier and Haiku 4.5 for high-volume, latency-sensitive work. A fourth tier, Fable 5, sits above Opus for the most demanding reasoning workloads. Claude Code - Anthropic's CLI tool - uses this family to autonomously write and debug production code, and Opus 5, Sonnet 5, and Fable 5 now all carry roughly 1M-token context windows.

Strengths:

• Extended context (roughly 1M tokens on Opus 5, Sonnet 5, and Fable 5) with strong recall across long documents and codebases

• Exceptional coding ability, particularly for agentic coding and complex debugging

• Consistent adherence to instructions and constraints

• Strong safety characteristics for regulated industries

• Native tool use and MCP integration for agent workflows

Limitations:

• Smaller community than OpenAI (but growing fast)

• Higher cost for the Opus and Fable tiers compared to competitors' mid-range models

• Limited fine-tuning options compared to OpenAI

Pricing per million input/output tokens: Haiku 4.5 $1/$5, Sonnet 5 $2/$10 (Anthropic canceled a planned September 2026 price increase and held this rate), Opus 5 $5/$25, Fable 5 $10/$50.

Gemini 3.1 Pro (Google)

Best for: Multimodal tasks, Google Cloud integration, long context.

Gemini 3.1 Pro is Google's current Pro-tier flagship, with a context window of roughly 1M tokens in production. It handles text, images, video, and audio natively. Pricing steps up above 200K tokens of input, which matters for teams that routinely send very large documents.

Strengths:

• Roughly 1M-token context window for processing large documents

• Strong multimodal understanding (text, image, video, audio)

• Deep integration with Google Cloud and Vertex AI

• Competitive pricing versus OpenAI and Anthropic on the standard tier

Limitations:

• Quality can still be inconsistent on complex multi-step reasoning

• Google Cloud dependency for some enterprise features

• Third-party tooling smaller than OpenAI

• The higher-context surcharge above 200K tokens can surprise teams that don't budget for it

Pricing: $2/M input, $12/M output for requests up to 200K tokens; $4/$18 per million above that threshold. Free tier available through Google AI Studio.

Muse Spark / Muse Glimmer (Meta) - Llama's successor

Best for: Cost-sensitive, high-volume use cases with data privacy requirements.

Meta retired the Llama flagship line in April 2026, when its new Superintelligence Labs team shipped Muse Spark as the replacement. Muse Spark itself is closed and API-only (available through the Meta Model API and OpenRouter), with roughly a 1M-token context window. The open-weight, self-hostable role Llama used to fill now belongs to Muse Glimmer, a smaller 30-billion-parameter companion model licensed under Apache 2.0 that Meta says can run entirely offline on a single 24GB consumer GPU. Meta has said a future Muse Spark release will be open-weight, but that hasn't shipped as of August 2026.

Strengths:

• Muse Glimmer: full data privacy, runs on your own infrastructure, no per-token API costs

• Muse Glimmer: a genuinely permissive Apache 2.0 license, a lower legal-review burden than Llama's old community license

• Muse Spark: strong native coding support (it powers Muse Code) with a large context window

• No vendor lock-in on the open-weight tier

Limitations:

• Requires ML infrastructure expertise to self-host Muse Glimmer

• Muse Glimmer's 30B parameter count trails frontier commercial models on complex reasoning

• Muse Spark, the more capable tier, is API-only - it doesn't offer Llama's old "run anything yourself" story

• No managed hosting for the open-weight tier means you handle scaling, monitoring, and updates yourself

Cost: Muse Glimmer is free to self-host; compute costs vary, typically $1-5/hour for GPU hosting. Muse Spark is priced per API call through the Meta Model API and OpenRouter.

Mistral Large 3 (Mistral AI)

Best for: European enterprises with data sovereignty requirements.

Mistral is a French AI company offering strong models with European data residency. Mistral Large 3, released December 2025, is a 675-billion-parameter mixture-of-experts model (41B active per token) licensed under Apache 2.0 - fully open-weight, unlike the closed flagships from OpenAI, Anthropic, and Google.

Strengths:

• European data residency for GDPR compliance

• Competitive performance on reasoning and coding tasks

• Open-weight (Apache 2.0) and self-hostable, unlike the US frontier labs' flagship models

• Strong multilingual capabilities, especially European languages

Limitations:

• Smaller community than OpenAI or Anthropic

• Fewer enterprise case studies

• Function calling and tool use less mature

• Self-hosting a 675B-parameter model, even with a smaller active-parameter count, requires real infrastructure investment

Pricing: Competitive with mid-range API tiers from OpenAI and Anthropic; free to self-host under the Apache 2.0 license.

DeepSeek V4 (DeepSeek) - open weight

Best for: Cost-sensitive enterprises wanting near-frontier performance without API dependency.

DeepSeek remains the most capable open-weight challenger from China. Its V4 line replaced V3/R1 in 2026: V4-Flash shipped July 31, 2026 and V4-Pro followed on August 13, 2026, both with a 1M-token context window and released under the permissive MIT license. Notably, the R2 reasoning model DeepSeek had been expected to ship never materialized - the company shipped V4 instead.

Strengths:

• Near-frontier performance on reasoning and coding at a fraction of the cost

• Fully open-weight under the permissive MIT license

• Self-hostable for maximum data privacy, with a 1M-token context window on both V4-Flash and V4-Pro

• Strong performance on math, code, and multi-step reasoning

• Active research community and rapid model iteration

Limitations:

• Chinese origin triggers additional compliance review at many regulated enterprises regardless of technical quality

• Smaller enterprise support and SLA options compared to US providers

• Self-hosting requires significant GPU infrastructure

• Less mature safety tuning compared to Anthropic and OpenAI

• The company's roadmap has been unpredictable - the widely anticipated R2 model never shipped

Pricing: $0 for model weights. Compute costs for self-hosting. API access available at prices significantly below OpenAI and Anthropic.

Comparison table

Choosing for your use case

Customer-facing chatbots

Recommended: Claude Sonnet 5 or GPT-5.6 Terra. Both handle conversational AI well. Claude's instruction-following is slightly better for maintaining brand voice and staying on-topic. For cost-sensitive high-volume chatbots, use a smaller model (Haiku 4.5, GPT-5.6 Luna) with routing to larger models for complex queries.

Document processing

Recommended: Gemini 3.1 Pro for very long documents (100K+ tokens) or Claude Opus 5 for complex reasoning about document content. Both handle long-context well.

Code generation and agentic coding

Recommended: Claude Opus 5. It consistently outperforms other models on coding benchmarks and powers the best agentic coding tools (Claude Code, Cursor). GPT-5.6 Sol is a strong second choice with its unified coding capabilities.

Internal automation

Recommended: Muse Glimmer, DeepSeek V4, or Mistral Large 3 (self-hosted) for cost efficiency at volume. GPT-5.6 or Claude (API) for lower-volume, higher-accuracy needs.

Regulated industries

Recommended: Self-hosted Muse Glimmer, DeepSeek V4, or Mistral Large 3 for maximum data control. If API is acceptable with proper DPA agreements, Claude or GPT-5.6 with enterprise agreements. Note: DeepSeek's Chinese origin may require additional compliance review for some regulated sectors.

The multi-model strategy

Menlo Ventures' 2025 State of Generative AI in the Enterprise survey of 495 U.S. enterprise AI decision-makers found that Anthropic, OpenAI, and Google together account for 88% of enterprise LLM API usage - but that figure measures total market spend, not how many distinct models any one enterprise runs. In practice, few enterprises we work with standardize on a single vendor. Multi-model routing has become the default architecture, not a niche one.

Most enterprises shouldn't pick one model. The standard approach in 2026 is multi-model routing: an abstraction layer that routes queries to the optimal model based on task complexity, cost, and latency requirements.

A typical enterprise multi-model configuration:

• Claude Opus 5 for complex reasoning, agentic coding, and safety-critical applications

• GPT-5.6 (Sol or Terra) for general-purpose tasks with broad tool integration

• Gemini 3.1 Pro for multimodal processing and long-context tasks

• Muse Glimmer / DeepSeek V4 (self-hosted) for high-volume, cost-sensitive workflows

• Claude Haiku 4.5 / GPT-5.6 Luna for simple classification, extraction, and routing decisions

How routing works: A lightweight classifier (often a small model or rule-based system) evaluates each incoming request and routes it to the appropriate model. Simple queries (classification, extraction) go to fast, cheap models. Complex queries (multi-step reasoning, code generation) go to capable, expensive models. This cuts costs 40-60% compared to routing everything through a frontier model.

Open-weight models like DeepSeek V4 and Meta's Muse Glimmer now close most of the gap with frontier commercial models on standard benchmarks. This means the "simple query" tier - which handles 60-70% of enterprise volume - can run on self-hosted infrastructure at near-zero marginal cost. The economics of multi-model routing have fundamentally changed.

What matters beyond the model

Andrej Karpathy - a founding member of OpenAI and former Director of AI at Tesla - captured the shift in how software gets built with a line that's since become a standard reference point in the industry:

"The hottest new programming language is English."

He wasn't talking about enterprise model selection specifically, but the implication holds for it: how you instruct, constrain, and feed context to a model now matters as much as which model you pick. A well-prompted mid-tier model with a tight retrieval pipeline routinely outperforms a poorly-prompted frontier model with no guardrails.

“

How you instruct, constrain, and feed context to a model now matters as much as which model you pick.

Model choice is one variable in a larger system: prompt engineering, the context and retrieval pipeline, evaluation, and guardrails all shape the outcome together. What data you feed the model matters more than which model you choose. Systematic accuracy measurement - not benchmark scores - is how you know if you've chosen right. Output filtering, hallucination detection, and safety checks are non-negotiable in production.

Don't over-optimize model selection. Pick a strong default (GPT-5.6 Terra or Claude Sonnet 5), build a good system around it, and switch models based on measured performance, not benchmarks.

Companies building AI-native products need this multi-model strategy from day one. At RaftLabs, our LLM integration services help enterprises select, deploy, and optimize model combinations across 100+ products. Our model routing strategies cut costs by 40-60% while maintaining accuracy. Talk to our AI engineering team about your LLM strategy.