Cell Towers Could Soon Broadcast AI on Your Phone!

Author: Better Stack · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Cell Towers Could Soon Broadcast AI on Your Phone! · Channel: Better Stack · 02:34 · Watch on YouTube

Topics: AI infrastructure and hardware, Local and open-source models

Summary

Air LLM is a research approach from MIT and Duke University that uses a cell tower to broadcast LLM weights, letting a phone perform model calculations without storing those weights in memory. It uses OFDM subcarriers and an RF mixer to perform the core dot-product calculation in the radio signal. In simulations using Llama 3.1 8B, next-word prediction accuracy was about 4% worse, while energy per token was about 158 times lower than streaming regular FP16 weights. The approach has not yet been tested between a real phone and tower, and weak or messy signals can degrade performance.

Key points

Air LLM is a paper by researchers from MIT and Duke University.
A 4-bit Llama 3.1 8B model still requires about 4 GB of memory.
The method broadcasts model weights over the air so a device does not need to store or load them.
A central radio, such as a 5G base station, sends weights using OFDM.
Each OFDM subcarrier carries one model weight.
A phone’s RF mixer combines the broadcast signal with the device’s input to produce the dot product used in an LLM layer.
The tower can broadcast to devices in range without knowing who is listening, and prompts stay on the phone.
In simulations, next-word prediction accuracy was about 4% worse and energy per token was about 158 times lower than streaming regular FP16 weights.
The work is still a simulation, and weak or messy signals can cause the model to perform poorly.

Chapters

Start	Chapter	Summary
00:00	The idea behind Air LLM	Air LLM proposes broadcasting model weights from a cell tower so a phone can run an LLM without storing the weights.
00:30	Sending weights over radio	The system uses OFDM subcarriers to transmit model weights instead of loading them from device memory.
01:01	Computing with an RF mixer	A phone’s RF mixer combines the input and radio signal to perform the dot product at the heart of an LLM layer.
01:32	Simulation results and limitations	Tests showed a small accuracy drop and lower energy use, but the approach remains a simulation and can struggle with poor signals.
02:03	Potential future applications	If it works in the real world, the method could let devices such as cheap phones and smartwatches run large models without holding them in memory.

Quotes

What if your phone could run an 8-billion parameter LLM without ever downloading the model? — 00:00

your device never has to store or load them at all. — 00:30

one broadcast serves every device in range while your prompt never leaves your phone. — 01:32

This is still a simulation rather than a real phone talking to a real tower — 02:03

Concepts and entities

Air LLM, MIT, Duke University, Llama 3.1, OFDM, RF mixer, Better Stack

Full transcript (original language)

00:00 Hear me out. What if your phone could run an 8-billion parameter LLM without ever downloading the model? Researchers from MIT and Duke University just published a paper called Air LLM, where a cell tower broadcasts the AI model weights over the air, and your device does the math directly on the radio signal before anything is ever stored in memory. So, here’s how it works. So, the real problem with running LLMs on small devices is memory. Even a 4-bit version of Llama 3.1 8-billion parameter model still takes about 4 GB. And during token

00:30 generation, every weight you fetch from memory only feeds about two floating-point operations. So, moving the weights end up costing way more energy than the math calculations they’re used for. What Air LLM does is send the weights over the air, so your device never has to store or load them at all. Instead, a central radio, like a 5G base station, broadcasts the weights using something called OFDM, which stands for orthogonal frequency-division multiplexing, and it’s the same technique Wi-Fi and 5G use, where one signal is split into thousands of tiny

01:01 frequency channels called subcarriers, and each one carries its own piece of data. In this case, each subcarrier carries one weight of the model. Your device then feeds its own input into an RF mixer, which is a component every phone already has for handling radio signals, and when the mixer combines two signals, the middle frequency of the result works out to be exactly the dot product of the weights and your input, which is the core math behind every LLM layer. So, the calculation happens inside the radio wave itself, and your device just reads off the answer. And since the tower doesn’t need to know

01:32 who’s listening, one broadcast serves every device in range while your prompt never leaves your phone. The team tested it on simulated city models of Paris and Munich using measurements from a real RF mixer, and on Llama 3.1 8-billion parameter model, the model’s accuracy at predicting the next word only got about 4% worse. In practice, that means that model’s answers are almost as good as the original, while energy per token dropped about 158 times compared to streaming regular FP16 weights. There are also some big caveats though. This

02:03 is still a simulation rather than a real phone talking to a real tower and devices with weak or messy signals start to fall apart like one in Munich that got stuck repeating its beer gardens and its beer gardens. But doing the math inside the radio waves themselves is one of the wildest edge AI ideas I’ve heard this year. And if it works in the real world, cell towers could one day stream AI models the same way they stream TV and radio today and even a cheap phone or a smart watch could run a big model without needing the memory to hold it. Wouldn’t that be something, right? And

02:34 if you want to stay up to date with the latest in AI research and engineering, be sure to subscribe to the Better Stack channel.