Everyone is building LLM routers, we deprecated ours.

Imagine building a system that automatically chooses the most economical AI model for each request. It sounds perfect on paper: why pay for GPT-4 when Claude 3 Haiku might be enough? That’s exactly what many companies are doing these days, launching sophisticated LLM routers that promise to drastically reduce inference costs. But there’s a catch: Manifest, a platform that built and launched its router in March 2026, decided to deprecate it completely after just four months. Their conclusion is surprising and counterintuitive: for most use cases, intelligent routing isn’t the solution it appears to be. This change of direction offers an important lesson to anyone considering implementing this technology.

What It’s About
#

Bruno Perez’s article tells the behind-the-scenes story of a difficult decision: why Manifest dismantled a feature that had been presented as central to their LLM gateway. Their router classified each request into four complexity levels (simple, standard, complex, and reasoning) to direct it to the most appropriate model. It seems logical, but after four months of use among thousands of cloud users, the team discovered that the promised benefits weren’t materializing as expected. Instead of reducing overall costs, routing was creating hidden problems that offset the apparent savings.

Why It Matters
#

The relevance of this story extends far beyond Manifest. While the LLM router market explodes with new players promising significant savings, this real-world experience shows that reality is far more complex than it seems.

The first problem is fundamental: the complexity of a task cannot be deduced from the prompt alone. When you ask a model to “evaluate and improve the tests in a repository,” the actual difficulty level depends on factors that only emerge during execution: whether the repo is a simple HTML site or the Linux kernel. The router makes its decision in the dark, based on incomplete information.

The second aspect is economic but counterintuitive: caching is far more effective than routing for reducing costs. Cache reads cost between 90% and 95% less than non-cached inputs. When a router maintains “stickiness” with the same model to leverage system prompt caching and conversation history, it ends up not doing routing. In other words, the router achieves economic efficiency by stopping its primary job.

The third problem is qualitative: routing breaks behavioral consistency. Switching between models during a work session degrades overall quality and distances engineers from mastering their tools. It’s like asking a painter to change brushes midway through a painting because one costs less.

Practical Applications
#

This lesson is particularly relevant if you’re building agentic systems or automated workflows. When you add a layer of uncertainty in routing, everything becomes harder to maintain: evals, system prompts, observability. Every time you don’t know which model handled a request, it becomes more complex to debug anomalous behavior or optimize performance.

For most teams, the best strategy is to consciously choose a battle-tested model and build around it. This means investing time in understanding the trade-offs between available models, just like a craftsman knows exactly which tool to use. If cost is a concern, the focus should be on intelligent caching strategies and prompt optimization, not dynamic routing.

Final Thoughts
#

Manifest’s experience represents a moment of maturity in the LLM industry. Not everything that looks efficient on paper is efficient in practice. LLM routing will probably have legitimate use cases, but it’s not the universal solution that industry marketing suggests. The real lesson is that simplifying often beats sophisticated automation when the hidden cost of automation is higher than the apparent benefit. In an ecosystem where models evolve rapidly and costs change constantly, having control and predictability might be worth far more than a few percentage points of cost savings.

Use Cases
#

Private AI Stack: Integration into proprietary pipelines
Client Solutions: Implementation for client projects

Resources
#

Original Links
#

Everyone is building LLM routers, we deprecated ours - Original link

Article reported and selected by the Human Technology eXcellence team processed through artificial intelligence (in this case with LLM HTX-EU-Claude-Haiku-4.5) on 2026-08-18 08:11 Original source: https://manifest.build/blog/why-we-deprecated-our-llm-router/

Summary #

Introduction #

What It’s About #

Why It Matters #

Practical Applications #

Final Thoughts #

Use Cases #

Resources #

Original Links #

Related Articles #

Summary
#

Introduction
#