For bounded, known-shape tasks, a cheap gate plus deterministic
retrieval can beat a language model on cost, speed, and safety — with
zero generated tokens. The principle is simple: route first,
generate last. Only reach for generation when nothing cheaper can
answer the question.
This is an open-ended series of small, honest experiments —
single-GPU reproductions, modest in scope and candid about their limits —
each one chasing the same question from a new angle. Chapters are added as the
work grows.
→
now launching · open source
LeastGen — The Production System
The principle, proven at scale. 95.1% cache hit rate, 715M tokens saved, zero cloud dependency — live on a single GPU.
The two-chapter series that started this research pipeline has grown into a transparent inference optimizer for autonomous agents,
routing 95% of requests from local cache at 0ms latency. The full project page, including architecture, live metrics, and deployment guide,
is now available. The next chapter of this work is no longer a research note — it's a product.
View the project →