GPT-5 in Production: What a Year of Reasoning Models Taught Us

When GPT-5 launched in August 2025 with unified reasoning across text, code and vision, our first instinct — like everyone else's — was to rip out the router in our stack and point everything at it. That was a mistake we corrected quickly, and a few others we corrected slowly.

Almost a year of production traffic later, here's what actually stuck.

The router isn't dead — it just moved up a layer

You still don't want a reasoning model summarising a two-line email. What changed is that the router is no longer about capability, it's about latency and cost. We route by "time budget" — 300ms responses go to a small open-weights model, 3-second responses go to GPT-5 mini, 15-second deep-work goes to full GPT-5 with extended thinking.

Prompting reasoning models is a different discipline

• Stop over-specifying the chain of thought. The model reasons better when you describe the goal, not the steps.
• Move examples from the prompt into the tool schema wherever possible — reasoning models weight tool signatures heavily.
• Give the model an explicit "escape hatch" tool for "I need to ask a clarifying question". Hallucinated confidence drops meaningfully.

The hidden cost line: thinking tokens

Extended-thinking tokens are billed even when they aren't returned to you. In one early client project we tripled our inference bill in a week because a background batch job silently opted into deep reasoning for tasks that didn't need it. We now cap thinking budgets per route and alert on drift the same way we alert on p99 latency.

What we'd tell our 2025 selves

Reasoning models don't replace software engineering — they raise its ceiling. The teams getting real leverage aren't the ones that swapped a model; they're the ones that redesigned the surrounding system to trust a smarter one.