I spent $10,000 on tokens. It should’ve cost $100!!

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: I spent $10,000 on tokens. It should’ve cost $100!! · Channel: AssemblyAI · 01:02 · Watch on YouTube

Topics: AI evaluation and benchmarks, Model inference and optimization, Multimodal models

Summary

The speaker says a recent experiment showed that token costs could be dramatically reduced: a $10,000 expense might have been about $100, and a $339 Claude Sonnet bill might have been $7 using Gemma 4 31 B. They caution that longer calls require more capable models, since accuracy is better with GPT-6 or Claude Sonnet and Opus. AssemblyAI is open-sourcing its Voice Standard Benchmark to help teams evaluate models and route calls to suitable options. The speaker also recommends using AssemblyAI’s LLM gateway for lower model costs and routing voice outputs.

Key points

The experiment involved more than 10,000 calls, each lasting 15 minutes.
The speaker says a $10,000 expense could have cost $100 with different model choices.
A $339 Claude Sonnet bill could have been $7 on Gemma 4 31 B, according to the speaker.
Model choice matters more when calls are longer.
For one-hour calls, the speaker says smaller models are no longer suitable.
The speaker says GPT-6 and Claude models such as Sonnet and Opus offer better accuracy for longer calls.
AssemblyAI is open-sourcing its Voice Standard Benchmark.
The benchmark can help a routing engine choose which model to use.
The speaker recommends AssemblyAI’s LLM gateway for cheaper model costs and routing voice outputs.

Chapters

Start	Chapter	Summary
00:00	The token-cost experiment	The speaker describes an experiment suggesting that selecting different models could sharply reduce token costs.
00:30	Choosing models for longer calls	The speaker explains that longer calls need more accurate models and introduces the open-source Voice Standard Benchmark for model routing.
01:02	Audio testing and the LLM gateway	The speaker recommends testing which model works best for different audio lengths and using the LLM gateway.

Quotes

The $10,000 I spent could have been $100. — 00:00

most engineers in the world do not know how to save tokens. — 00:00

Things really change when the duration of calls are longer. — 00:30

We call it voice standard benchmark. — 00:30

Concepts and entities

AssemblyAI, Voice Standard Benchmark, Claude Sonnet, Gemma 4 31B, GPT-6, Claude, Opus, LLM gateway

Full transcript (original language)

00:00 So, I spent $10,300 last week. >> Did you miss me? Did you miss me? >> It’s okay if my CFO got mad today, but the truth is most engineers in the world do not know how to save tokens. But, now you do because we ran an experiment with our benchmark and evals is by doing 10,000 plus calls, 15 minutes each. And guess how much was the cost difference? The $10,000 I spent could have been $100. Your $339 bill on Claude Sonnet could have been just $7 on Gemma 4 31 B.

00:30 Things really change when the duration of calls are longer. If a call is 1 hour long, you cannot use smaller models anymore. Accuracy is way better with GPT-6 or Claude models like Sonnet and Opus. So, that’s why we are helping you by open-sourcing our evals. We call it voice standard benchmark. And over there, you can just give it to your routing engine on Claude. It’ll automatically figure out which model to use, saving you 10x the cost in tokens. You can test out yourself by giving

01:02 hours of audio or seconds of audio which model is best for you. And use our LLM gateway for cheaper model cost as well as routing it directly from your voice outputs. Go check