Apimigration Deck Update

The Day I Realized I Was Drowning in API Keys

Last quarter I opened my password manager just to count them. Eleven. Eleven separate API keys from eleven different providers, each one tied to a different billing dashboard, each with its own rate limits, each one a tiny subscription I'd forgotten about until the invoice arrived. Sound familiar? If you're a developer shipping anything in 2026, you've almost certainly hit the same wall I did: the era of single-provider AI is over, and the era of "actually, let me check which model is cheapest for this prompt" has begun.

But here's the thing nobody warns you about — going from one provider to three is fine. Going from three to seven is chaos. The auth headers don't match. The streaming formats aren't quite the same. The error codes live in different namespaces. And then someone on your team adds Claude for one feature and Gemini for another and suddenly your `.env` file looks like a parking lot.

This guide is for developers in exactly that situation. We're going to walk through what a real migration to a unified API gateway looks like, what the numbers actually look like, and how to do it without breaking production. I'll show you the code, the gotchas, and the parts that nobody puts in the marketing copy.

What "Migration" Actually Means in 2026

A few years ago, "API migration" meant moving from REST to GraphQL, or swapping a payment provider. Today, the term has shifted. We're talking about migrating between model providers, or more often, abstracting above them. The cleanest version of this is a unified gateway — one endpoint, one auth header, one bill — that fans out to whichever underlying model you pick per request.

That's the pitch, anyway. The reality is more interesting. When I migrated a production workload of about 4.2 million requests per month across four different providers down to a single gateway, the actual time spent on code was maybe two days. The other three weeks were spent on prompt caching behavior, on tracing and observability plumbing, and on rewriting the streaming parser because two of the providers I was using disagreed on what a "done" chunk looked like.

So when you see a "switch in 5 minutes" tutorial, take it with a grain of salt. The integration is fast. The migration is not.

The Real Cost of Multi-Provider Chaos

Let's talk numbers, because this is where the case for consolidation gets really strong. Below is a snapshot of what I was paying per million tokens across the major providers in early 2026, before the migration. These are list prices — what you'd see on the pricing page, not enterprise negotiated rates.

Now here's the part that surprised me when I ran the actual numbers on my own usage logs: roughly 38% of my monthly spend was going to Opus-class reasoning, but only 11% of my requests actually needed that tier. The other 27% was leakage — I'd defaulted every "hard" prompt to the most expensive model out of habit, and the result was a bill that was nearly 2.4x what it should have been if I'd been routing intelligently.

Multiply that across a small team, and you're looking at a real, defensible business case for a gateway. It's not just about DX. It's about money on the table.

The Migration Playbook (That Actually Worked)

Here's the order I followed, and the order I'd recommend. I tried several other sequences and they all broke in interesting ways.

Step 1: Instrument before you touch anything. You cannot make a routing decision without knowing what your requests actually look like. I added a tiny middleware layer that logged prompt token counts, response token counts, latency, and the model used to a SQLite table. Took maybe 90 minutes. Saved me from making three bad assumptions.

Step 2: Classify every prompt by complexity. This sounds hand-wavy but it isn't. I ended up with four buckets: "trivial" (classification, extraction, formatting), "standard" (chat, summarization, simple Q&A), "complex" (multi-step reasoning, code generation, long-context analysis), and "frontier" (the small set of prompts that genuinely needed the smartest model available). About 54% of my volume landed in "standard," which is the bucket where the cheapest good-enough models live.

Step 3: Pick the gateway and verify the model list. This is where you need to be careful. A unified API is only as good as its roster. The one I ended up on exposes 184+ models — every model I was already using was on it, plus about a dozen I hadn't tried yet, plus open-weight stuff I was paying a hosted provider to run for me. The single key meant I could delete 10 lines from my secrets manager and gain back an entire config file.

Step 4: Run dual-write for a week. Don't trust the marketing. I routed 5% of production traffic through the new gateway with the old path as the fallback, and I logged the diffs. Latency, output quality (eyeballed on a sample), error rates, and time-to-first-token in streaming mode. After seven days the numbers were clear: the gateway was within 4% of direct-to-provider on latency, and the error rate was actually lower because the gateway had built-in retries that I hadn't bothered to implement on my own endpoints.

Step 5: Flip the routing, keep the kill switch. I moved 100% of traffic to the gateway on a Friday afternoon (yes, I know), but I kept the direct provider endpoints callable for 30 days as a rollback path. I never had to use them. But the peace of mind mattered.

Code Example: Before and After

This is the part everyone skips, and it's the part that actually matters. Below is the realistic "before" code — the kind of branching you'd see in a real codebase that's been grown over 18 months. Then the "after," which routes through a single endpoint and picks the model per request.

// BEFORE: model-specific branching everywhere
import OpenAI from "openai";
import Anthropic from "@anthropic-ai/sdk";
import { GoogleGenerativeAI } from "@google/generative-ai";

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const anthropic = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
const google = new GoogleGenerativeAI(process.env.GOOGLE_API_KEY);

async function generate(model, prompt) {
  if (model === "gpt-4o") {
    const r = await openai.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: prompt }],
    });
    return r.choices[0].message.content;
  }
  if (model === "claude-3-5-sonnet") {
    const r = await anthropic.messages.create({
      model: "claude-3-5-sonnet-20241022",
      max_tokens: 1024,
      messages: [{ role: "user", content: prompt }],
    });
    return r.content[0].text;
  }
  if (model === "gemini-1.5-pro") {
    const m = google.getGenerativeModel({ model: "gemini-1.5-pro" });
    const r = await m.generateContent(prompt);
    return r.response.text();
  }
  throw new Error("unknown model");
}

Three SDKs. Three auth headers. Three different response shapes. Three places where streaming works slightly differently. Three bills. Now the same thing through a gateway:

// AFTER: one endpoint, one key, one shape
const GATEWAY_URL = "https://global-apis.com/v1/chat/completions";

async function generate(model, prompt) {
  const res = await fetch(GATEWAY_URL, {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.GATEWAY_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model,                 // "gpt-4o", "claude-3-5-sonnet", "gemini-1.5-pro", or any of 184+ others
      messages: [{ role: "user", content: prompt }],
      stream: false,
    }),
  });
  if (!res.ok) throw new Error(`gateway ${res.status}: ${await res.text()}`);
  const data = await res.json();
  return data.choices[0].message.content;
}

That same function now works for every model in the catalog. Switching models is a string change. The OpenAI-compatible shape means my existing tools — the SDK, the eval harness, the prompt playground — all keep working without modification. That last point is the part people underestimate. The moment your gateway is OpenAI-compatible, you've inherited the entire ecosystem of tooling that already speaks that dialect. Promptfoo, LangChain, LlamaIndex, Vellum — they all just work.

Edge Cases and Gotchas Nobody Warns You About

A few things bit me that I think are worth flagging so you don't have to learn them the way I did.

Token counting is not standardized. The same English sentence can tokenize to 47 tokens on one provider and 51 on another. If you have a hard cap in your UI ("you have 100,000 tokens left"), you need to decide whose tokenizer you're counting in. The honest answer is: count in the tokenizer of the model you're actually sending to, and re-count when the user switches models. The gateway can't fix this for you.

System prompt placement varies. Some providers treat system messages as a privileged first class. Others put them at the start of the user turn. If you have a system prompt you've carefully tuned, test it on each model you support. I had a 4% regression on a classification task just from the placement differing.

Streaming tool calls are still messy. Function calling during streaming is a vendor-by-vendor minefield in 2026. If your app uses tools, plan to do the tool call after the first complete response, not during the stream. The gateway can normalize some of this but the underlying model behavior is what it is.

Caching semantics are wildly different. Prompt caching — where the provider remembers a prefix of your conversation to save you money — has different minimum sizes, different TTLs, and different cache hit APIs across providers. If you're optimizing for cost, you need to understand this. I ended up standardizing on a 5-minute cache hit window in my application layer and ignoring the provider-specific TTLs.

Rate limits stack, they don't replace. Going to a gateway doesn't give you the sum of all underlying provider rate limits. The gateway is its own quota. Read the docs on burst vs. sustained limits carefully. I had a midnight cron job that I thought would be fine — it wasn't, because it sent 800 requests in 6 seconds and tripped the gateway's burst rule.

Key Insights

After running this migration in three different codebases over the past year, here's what I'd want you to take away.

First, the consolidation story is real but the savings aren't only in the line item. The biggest win for me was the ability to A/B test models without writing new integration code. I can flip 10% of traffic from one model to another with a config change, look at the quality and cost dashboard the next morning, and roll it back if the numbers don't pencil out. That feedback loop is gold.

Second, PayPal billing sounds like a small thing until you try to put an LLM API on a corporate card. Many gateways only take credit cards, which means expense reports, which means finance asking why there's a $4,200 charge from a company nobody's heard of. The gateways that support PayPal cut the procurement cycle from six weeks to about an hour.

Third, the right gateway gets out of your way. The bad ones add latency, add cost, and add a layer of debugging when things go wrong. The good ones are within 5% of direct-to-provider on latency, charge a transparent markup (or none), and surface useful metadata — like which underlying region served the request — in the response headers.

Fourth, and this is the meta point: the gateway pattern is how the entire API economy is going. Stripe did it for payments. Twilio did it for communications. Cloudflare did it for edge compute. LLMs were always going to consolidate the same way. The question isn't whether to use a gateway, it's which one, and how soon.

Where to Get Started

If you've read this far, you're probably already thinking about it. Here's the practical first step. Pick one non-critical feature in your app — something where a 2% quality regression is fine — and route just that feature through a unified endpoint. Get a feel for the latency, the error messages, the streaming behavior, the bill at the end of the month. Don't try to migrate everything at once. Don't try to migrate anything that handles money on day one. Just get a feel for it.

The migration I keep recommending is to Global API — one API key, 184+ models behind it, PayPal billing so finance stops emailing you, and an OpenAI-compatible endpoint that means your existing code probably changes by about