Model routing selects a path based on complexity, latency needs, or cost limits. Simple and complex requests can go to different candidate models.
The demo sends requests through a classifier into two lanes. Routing mistakes and retry cost must be measured so savings do not hide quality loss.
When to use
Use it when requests vary enough to justify balancing quality, latency, and cost.