Understand the Gemma 4 model family
Read the Portuguese translation →
Author: Google for Developers · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: Understand the Gemma 4 model family · Channel: Google for Developers · 02:11 · Watch on YouTube
Topics: AI models, Local and open-source models, Multimodal models
Summary
Section titled “Summary”Gemma 4 is a family of multimodal open large language models available in five sizes across four architectures. The smaller E2B and E4B models are dense, efficient, designed for on-device use, and use per-layer embeddings and dedicated audio and image encoders. The 12B model is aimed at high-end laptops and processes audio by projecting it directly to the language model without a dedicated encoder. The 26B A4B model uses a mixture-of-experts approach, activating 4 billion parameters at a time, while the dense 31B model is described as the family’s most capable.
Key points
Section titled “Key points”- Gemma 4 is a multimodal family of open large language models with five sizes across four architectures.
- The smaller E2B and E4B models are dense, efficient, and suited to on-device use.
- The “E” in E2B and E4B refers to “effective” and to per-layer embedding (PLE).
- The smaller models use dedicated encoders to analyze audio and images.
- The 12B model is intended for high-end laptops and projects audio directly to the language model without a dedicated audio encoder.
- The 26B A4B model is a mixture-of-experts model that activates 4 billion of its 26 billion parameters at a time.
- The 26B A4B model has a larger vision encoder for more difficult vision tasks.
- The dense 31B model is described as the most capable model in the Gemma 4 family and also uses a larger vision encoder.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | Gemma 4 and the smaller models | The video introduces Gemma 4’s model family and explains the smaller models’ efficiency, per-layer embeddings, and dedicated modality encoders. |
| 00:31 | The 12B model | The 12B model is positioned for high-end laptops and processes audio by projecting it directly to the language model. |
| 01:05 | The 26B A4B model | The 26B A4B model uses a mixture of experts with 4 billion parameters active at a time and has a larger vision encoder. |
| 01:36 | The 31B model and resources | The dense 31B model is described as the family’s most capable, and the video closes by pointing viewers to more resources. |
Quotes
Section titled “Quotes”Gemma 4 is a family of multimodal open large language models — 00:00
These are essentially lookup tables of embeddings that the model can use to process its queries. — 00:31
we removed it entirely in the 12B model — 01:05
it only uses 4 billion of them at a time. — 01:36
It’s the most capable one of the Gemma 4 family — 01:36
Concepts and entities
Section titled “Concepts and entities”Gemma 4, E2B, E4B, 12B, 26B A4B, 31B, Per-Layer Embeddings (PLE), Mixture of Experts, encoder-free method
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 This is Gemma 4 explained in 2 minutes. Let’s get started. Gemma 4 is a family of multimodal open large language models that come in five different sizes across four different architectures. When we start with the smaller models, you can see that those are dense, they’re efficient, and and they’re perfect for on-device usage. There’s this E in E2B and E4B that references effective. And that stands for per-layer embedding or PLE that they
00:31 use in their architecture. These are essentially lookup tables of embeddings that the model can use to process its queries. They also process audio and images, and they do that with very dedicated encoders that are capable of analyzing these different modalities. The 12B model is one step larger and is perfect for high-end laptops. But it does something special. It might not use the PLE of the smaller models, but it uses
01:05 an encoder-free method. Instead of using these dedicated encoders that you see for the smaller models, it removes them entirely. So, instead of this audio encoder of the E2B model, for instance, we removed it entirely in the 12B model, and instead directly project the audio to the LLM. Now, the 26B A4B model is a mixture of experts model, where it references the A, which stands for active. And this means that, even though
01:36 it has 26 billion parameters, it only uses 4 billion of them at a time. And because it’s a larger model, we give it a larger vision encoder, so it’s really perfect for these more difficult vision tasks. And last but certainly not least, there’s this 31B model, and it’s a dense model. It’s the most capable one of the Gemma 4 family and likewise, it uses a larger vision encoder. But it has no special tricks other than being the best model in the family.
02:11 If you want to know more about Gemma 4, here a couple of resources that you can use. See you in the next video.