Yes, Nvidia has a fast new AI model. I care more about choosing the right one for every task.

The line that stopped me in Nvidia's latest model announcement wasn't the one about speed. It was a sentence in the Switchyard documentation: point Claude Code or Codex at another model and let the agent keep speaking its native API.

That's a much bigger promise than a faster small model.

If it works beyond a tidy demonstration, you could keep the coding agent, tools and workflow your team already knows while changing the model that handles each turn. Routine work could go to a fast local model. Difficult planning could go to a frontier model. A private task could stay on hardware you control. The application wouldn't need a fresh integration every time the route changed.

I can also see how quickly that idea gets oversold. Translating an HTTP request isn't the same as making an agent model-agnostic. Tool conventions, reasoning formats, system prompts, context handling and plain old model behaviour don't disappear because the JSON arrived at the right endpoint.

So, yes, Nvidia Nemotron 3.5 Lightning looks fast. The more interesting release is the junction sitting in front of it. I think Switchyard tells us more about where agent infrastructure is heading, even though I wouldn't put it into production today.

The model is built for the boring work

Here's all you really need to know about Nemotron 3.5 Lightning: Nvidia built it to handle the repetitive work that keeps an AI agent busy all day.

Most agent turns aren't grand acts of reasoning. They inspect a file, format a result, call a tool, check an error, summarise a response or decide whether the last step worked. Sending every one of those turns to the largest model you can buy is like using a removalist truck to collect lunch. It'll do the job. You just won't enjoy the bill or the wait.

Nvidia says Lightning can produce output up to four times faster than similar models. That's Nvidia's number, and I haven't independently reproduced it. The useful idea isn't the benchmark anyway. It's the division of labour: let a powerful model handle the difficult planning, then give the routine calls to something quicker and cheaper.

The economics only work if the smaller model completes the task. A cheaper token isn't cheap when a malformed tool call starts a retry loop. A fast answer isn't useful when it edits the test until the test agrees with it. Agent costs have to be measured per finished task, including retries, escalations and human intervention. Per-token price and tokens per second can't answer that on their own.

And that's where Switchyard enters the story. The model does the work. Switchyard decides which model gets it.

Related Article11 min read

AI Agent Graphs Explained: Where Loops Fit, When Graphs Help, and What Breaks

A graph does not replace an AI agent loop. It gives loops routes, boundaries and shared state. Here is when that helps, when it makes things worse...

Read full article

What NeMo Switchyard actually does

NeMo Switchyard is a Rust proxy and library for model traffic. Its server accepts three common client formats: OpenAI Chat Completions, OpenAI Responses and Anthropic Messages. It converts the request into an internal shape, chooses a configured target, forwards the call in the backend's format and translates the response back into the shape the client expects.

In plain language, Claude Code can continue sending Anthropic-style Messages while a compatible non-Anthropic backend serves the request. Codex can keep using the OpenAI Responses shape. Switchyard's documented targets include vLLM, Nvidia NIM, Ollama and other OpenAI-compatible endpoints.

The project also exposes launch commands for Claude Code, Codex and OpenClaw. That matters because it lowers the setup barrier. You don't have to rebuild the client before finding out whether a different backend can survive contact with it.

Routing is the second half of the package:

• A classifier can choose between a weaker and stronger model from the request.
• A stage router can use recent tool activity and errors to infer where an agent is in its work.
• An escalation route can try the cheaper model first, then ask a judge whether the task needs a stronger one.
• A random route can split traffic for an A/B test.
• A passthrough route can expose one target without making a routing decision.

Switchyard records Prometheus metrics for requests, errors, latency, tokens and routing overhead. Nvidia's technical article also describes session state and affinity when a policy needs them. Those details separate it from a throwaway reverse proxy. A routing decision you can't inspect isn't much of a control plane.

Still, none of this makes models interchangeable. API portability is the outer shell. Behavioural portability is the difficult centre.

One model may emit tool arguments another parser rejects. One might interpret a system instruction differently. Streaming events can arrive in a different order. A model trained around one harness may understand its hidden conventions while another produces perfectly valid text that the agent can't use. Switchyard can translate a tool schema; it can't give the chosen model capabilities it doesn't have.

That's the strongest argument against my thesis, and it's a good one: Switchyard doesn't free an agent from model dependence. It reduces integration friction while adding a new component, new routing policies and new failure modes. Calling that “model-agnostic” would be marketing. Calling it useful plumbing is fairer.

The test that would change my mind isn't another successful HTTP response. It's the same agent completing the same tool-using task across several models, with the failures, retries, context loss and final quality recorded. I didn't find that cross-client, cross-model evidence in the launch material, and the setup check below doesn't provide it. Until it exists, Switchyard offers protocol portability. Operational portability remains a hypothesis.

Related Article13 min read

Harness matters, sometimes: does GPT-5.6 Sol really run better on Anthropic's Claude Code than Codex?

Devs are saying GPT-5.6 Sol codes better inside Claude Code than in OpenAI's own Codex CLI. I found a real, still-open GitHub issue that explains...

Read full article

What the evidence says so far

Nvidia's most detailed routing result comes from LangChain's internal Deep Agents evaluation. Across 145 multi-turn tasks, LangChain routed between Nemotron 3.5 Lightning and Claude Opus 4.8. Nvidia reports a 74% cost reduction against the frontier-only baseline, with only 7% of calls going to the frontier model.

There was a cost to that saving: Nvidia reports an accuracy trade-off of about 6%.

That trade-off is the story, not an inconvenient footnote. You might gladly take it for low-risk batch work. You might reject it for a production change, a security decision or anything where a wrong action is dearer than the model call. A router doesn't remove that judgement. It turns it into policy.

LangChain's launch-day summary puts the cost-versus-quality bet in one place:

LangChain

@LangChain

Do all of your agents need a frontier model?

We ran NVIDIA's open source router, Switchyard, through our Deep Agents eval suite comprising of 145 multi-step tasks - Tool use, retrieval, filesystem operations, long context.

The result: 93% of turns went to a 30B model. Only 7% needed Claude Opus 4.8.

Routing between the two cut total cost by ~70%, while retaining ~90% of Opus's accuracy on the same calls.

Try out the deepagents integration with NVIDIA's Switchyard today: https://buff.ly/1HtHoJ2

67

11 Aug 2026

Nvidia also reports partner results from Cognition, Ramp, Boomi and others. They point in the same direction: routing can reduce mean cost or runtime on a defined workload. They don't yet give us a broad independent answer. Most of the economics published in the first 24 hours came from Nvidia or organisations participating in its launch.

The early public tests I found were too narrow to tell us how the proxy handles a long session, provider outage, context pressure or changing tool schema. They can't carry the thesis here. Still, concrete local tests are more useful than vague claims. Saiyam Pathak shared one comparison from a DGX Spark, including throughput, memory use and the resulting tool call:

Saiyam Pathak

@SaiyamPathak

Ran Nemotron 3.5 Lightning on my DGX Spark a few hours after release. 72-87 tok/s decode through Ollama, 26GB in memory with 256K context, and on an agent task it reached the same correct tool call as Qwen3.5 35B-A3B in a quarter of the tokens (485 vs 1,953). The execution-layer pitch holds up on a desktop box. Full numbers: https://blog.kubesimplify.com/nemotron-3-5-lightning-on-dgx-spark

7

11 Aug 2026

A setup check, and an honest limit

A reproducible setup check installed nemo-switchyard 0.2.0 in a disposable Python 3.13 environment. Its command line exposed the documented launch paths for Claude Code, Codex and OpenClaw.

The test environment didn't have an OpenRouter or Nvidia inference credential, so it couldn't run a model request end to end. That means I can't tell you, from this check, that a real Claude Code tool sequence survives translation to Nemotron. I can't give you latency, malformed-call or cost numbers either. The setup surface exists. The behavioural claim remains untested here.

I prefer that untidy answer to a fake benchmark.

There's another limit that matters more: Switchyard's own README calls the software pre-alpha and says its API and algorithms may change significantly before version 1.0. It carries an explicit warning: experimental software, not for production use. The Python package is marked Alpha.

That warning should govern every adoption recommendation today.

Does Switchyard reduce vendor lock-in?

Some of it, yes.

If your application speaks to Switchyard rather than directly to one provider, changing the backend endpoint or model ID should become easier. A team could test an open-weight model beside a closed one without rewriting every client integration. It could keep sensitive work on premises when the deployment supports it, or route around a price change or outage. Those are plausible forms of portability, but only if the replacement model still completes the work and the router behaves reliably. This article hasn't demonstrated either condition end to end.

But the lock-in moves as well as shrinks.

Routing rules, evaluation data and operational habits start accumulating around the control layer. The router may become more durable than one model because it remembers which model gets which work. That decision machinery gets harder to replace once teams trust it.

That's the case for Switchyard mattering more strategically than Nemotron 3.5 Lightning, not proof that it already does. Nvidia owns much of the compute underneath AI. With Switchyard, it's bidding for the layer that decides where the work goes. The code is open under Apache 2.0 and the targets aren't limited to Nvidia models, so this isn't a closed trap. It is still a very Nvidia-shaped path through an Nvidia-shaped stack.

There are alternatives. OpenRouter, LiteLLM, Hermes and gateway products already cover parts of provider switching, routing and translation. I didn't find a neutral head-to-head evaluation showing that Switchyard makes better agent-routing decisions than those options. It has to earn its place through agent-aware routing, state, observability and better evaluations. A fresh repository and a strong launch partner list don't prove it has won anything.

Who should use it now?

Individual developers: Try it in a disposable project. Keep the task harmless, inspect every tool call and expect rough edges. Don't point pre-alpha routing at a valuable repository and go to lunch.

Technical teams already operating agents: Run a controlled proof of concept comparing finished-task cost, errors and escalation rates with your current baseline. Start with your failure tolerance, not Nvidia's claimed saving.

Risk-sensitive organisations: Wait. The production warning, moving APIs and thin independent evidence make this a research input rather than an approved control plane. You can evaluate the architecture without placing regulated data or consequential actions behind it.

I'd choose Lightning only after it completed my routine work reliably enough to earn the cheap route. I'd choose Switchyard only if its policies beat a simpler gateway and I could explain them afterwards.

That's a higher bar. It should be.

Key takeaways

• Nemotron 3.5 Lightning is a 30B hybrid MoE model with about 3B parameters active per token and a context window of up to 1 million tokens.
• Nvidia's speed and task-completion figures are vendor benchmarks, not independent proof of cost per successful agent task.
• Switchyard translates OpenAI and Anthropic API formats, routes work across backends and records operational metrics.
• API translation reduces integration friction, but it doesn't make model behaviour portable.
• Switchyard 0.2.0 is pre-alpha, experimental and explicitly not for production use.
• The strategic bet is the routing layer: models can change while routing policy, evaluation data and operational trust accumulate around the control point.

The model is news. The junction is the bet.

Nemotron 3.5 Lightning gives Nvidia a plausible workhorse for the repetitive turns that make agents slow and expensive. If independent testing holds up, that alone will make it useful.

Switchyard asks the more consequential question: what if the agent stays put while the model changes underneath it?

I don't think Nvidia has answered that yet. Version 0.2.0 is too early, the strongest savings still come from the launch circle and behavioural portability is much harder than protocol translation. But the direction makes sense. Agents won't run on one model forever, and somebody's software will decide which model gets the next turn.

We're all still working out how much authority that junction should have. For now, I'd test it, measure it and keep it well away from production. I wouldn't ignore it.

---