AI Agents Are Ditching English to Think Faster

Author: Better Stack · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: AI Agents Are Ditching English to Think Faster · Channel: Better Stack · 02:34 · Watch on YouTube

Topics: AI agents, Model inference and optimization, AI models

Summary

The video explains how a hybrid AI agent can reason in latent space instead of converting every step into English, which can preserve more information and improve speed. The system uses a small adapter called Hybrid Link to move between latent reasoning and text, allowing the agent to make tool calls and expose readable approximations of its thoughts. Researchers say the agent learned when to use latent mode and showed improved efficiency compared with text-based agents. The system remains experimental, but its 8-billion and 14-billion parameter checkpoints are available on Hugging Face under the MIT license.

Key points

Latent reasoning feeds a model’s hidden-state vector back into itself rather than decoding and re-embedding a text token.
This approach skips vocabulary processing and can carry more information per step.
Earlier latent reasoners could not call tools using text or provide thoughts that people could inspect.
Hybrid Link is a small adapter attached to a Qwen-3 base model that translates hidden states into input-embedding-like vectors.
The agent switches into latent mode upon emitting a special latent token and can decide when to return to text mode.
Because Hybrid Link maps near real token embeddings, latent steps can be approximated with readable pseudo text.
The researchers used reinforcement learning to improve problem-solving, without explicitly teaching the model when to use latent mode.
The hybrid agent used up to half as many tokens, ran about twice as fast, and was estimated to cost roughly three times less than regular text-based agents.
The approach is still a research system, and standard serving engines do not yet support this kind of hybrid generation.

Chapters

Start	Chapter	Summary
00:00	Why AI might reason beyond English	The video introduces the inefficiency of converting a model’s internal reasoning into words and presents latent reasoning as an alternative.
00:30	Latent reasoning and its limitations	Latent reasoning feeds hidden states back into the model, but has traditionally made tool use and human inspection difficult.
01:02	How Hybrid Link works	A small adapter lets the agent move between latent thought and text, including writing tool calls and producing readable approximations of latent steps.
01:33	Inspecting thoughts and testing the agent	An example shows the model recalling a sound speed and searching to verify it, while a latent-vector swap test demonstrates that its reasoning matters.
02:03	Training, efficiency, and availability	The agent learned to use latent mode and showed efficiency gains, though the system still requires a custom inference stack and is available as a research release.

Quotes

English is a pretty terrible language for thinking, at least if you’re an AI. — 00:00

It completely skips processing any vocabulary, so it’s faster and it carries more information per step. — 00:30

you can take any latent step, look up the nearest token in the vocabulary, and get a readable pseudo text back. — 01:33

they never told the model when to use latent mode, but over time it learned on its own to do more and more of its thinking there. — 02:03

ran about two times faster, and by their estimates, it was roughly three times cheaper to run. — 02:34

Concepts and entities

latent reasoning, Hybrid Agent, Hybrid Link, Qwen3, Stanford, Together AI, Math 500, Hugging Face

Full transcript (original language)

00:00 English is a pretty terrible language for thinking, at least if you’re an AI. Every time a model reasons through a problem, it has to squeeze a huge internal vector with thousands of dimensions down into a single word, and then rebuild everything from that one word on the next step. So, what if AI agents could skip English entirely and think in their own internal language instead? So, a team from Stanford and Together AI just built an agent that does exactly that. And the coolest thing is that you can still read its thoughts. Here’s how it works. So, researchers have been experimenting with something called latent reasoning. And the way it

00:30 works is that instead of decoding a token, re-embedding it, and feeding it back in, the model takes its last layer hidden state and feeds that vector straight back in as the next input. It completely skips processing any vocabulary, so it’s faster and it carries more information per step. The problem is that latent reasoners couldn’t call tools because a tool call has to be text, and nobody could read what they were thinking, which is a real issue when you need to audit or debug an agent. But now, researchers have developed something called hybrid agent, which solves both of these problems. So, what they did was attach a tiny adapter

01:02 called Hybrid Link to a Qwen-3 base model, which has a two-layer projection with a residual connection and about 10 million parameters on a 4-billion parameter model, and its job is to translate the model’s output hidden state back into something that looks like an input embedding. When the model emits a special latent token, it switches into latent mode and starts looping its hidden states through that Hybrid Link with no decoding at all. And it also decides on its own when to stop. Once the adapter’s output lands closest to the embedding of the closing latent token, the model drops back into text

01:33 mode, writes a Python call, reads the result, and keeps going. And because Hybrid Link is trained to land near real token embeddings, you can take any latent step, look up the nearest token in the vocabulary, and get a readable pseudo text back. In the example from the repo, the model asked how long sound takes to travel 10 km, and from its latent span readout, we can see that the model was recalling 343 m/s and then deciding to search to verify it, and in the end it answers the question in about 29 seconds. They also showed that the model genuinely relies on those latent

02:03 thoughts. When they swapped the 8 billion models latent vectors with ones from a different problem, its math 500 accuracy dropped from 84.6 to 44.8% and to train it, they first taught the model how to read its own latent thoughts, and then they used reinforcement learning to make it better at solving problems. And the interesting thing is that they never told the model when to use latent mode, but over time it learned on its own to do more and more of its thinking there. And the results were pretty impressive. Compared to regular text-based agents, hybrid agent used up to half as many tokens,

02:34 ran about two times faster, and by their estimates, it was roughly three times cheaper to run. But keep in mind that this is still in a research system, and standard serving engines don’t support this kind of hybrid generation yet. So, the team had to build their own inference stack. But the 8 billion and 14 billion checkpoints are already on Hugging Face under the MIT license, so you can run the latent thought inspector yourself. And if you want to stay up-to-date with the latest in AI research and engineering, be sure to subscribe to the Better Stack channel.