New Model: Reflection Beam is a new Open Weight Model

Read the Portuguese translation →

Author: Hugging Face · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: New Model: Reflection Beam is a new Open Weight Model · Channel: Hugging Face · 1:20:41 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, AI models, Local and open-source models

Summary

Hugging Face and Reflection introduce Beam, an open-weight model presented as token-efficient and suited to agentic tasks, especially coding. Reflection’s team discusses its pre-training, reinforcement-learning process, data curation, harnesses, and efforts to make the model generalize across tools and environments. The livestream also surveys integrations for inference and fine-tuning, and shows practical examples built with Beam. It closes with an invitation to enter Beam Build, a community challenge offering cash and Hugging Face Hub credits.

Key points

• Beam is presented as a token-efficient workhorse for agentic tasks, with a focus on coding.
• The release includes open weights, technical resources, traces, evaluation materials, and ecosystem integrations.
• Reflection describes using scaling ladders to test pre-training methods and model configurations before training at larger scales.
• Pre-training data was curated for quality, quantity, and diversity, including educational material, code, and STEM content.
• The team designed reinforcement-learning environments to encourage cognitive skills such as self-verification, backtracking, and trying new strategies.
• Beam was trained with the Daedalus harness and is also used with Mirror; the team reports that it generalizes to other harnesses such as Pi and OpenCode.
• The team discusses the engineering and scientific challenges of a long, large-scale RL run, including numerical divergence, fault tolerance, and observability.
• Inference and customization options include tools and integrations such as Transformers, vLLM, SGLang, TRL, Miles, and other partner projects.
• Beam Build invites community submissions of spaces, datasets, or models, with $10,000 in cash prizes and $10,000 in Hugging Face Hub credits.

Chapters

Start	Chapter	Summary
00:32	Introducing Beam and the livestream plan	The hosts introduce Beam and outline a session covering the model, integrations, research, and practical demos.
04:10	Beam’s focus and open release	Reflection describes Beam as a token-efficient, agentic workhorse and discusses the value of sharing research and resources openly.
10:59	Where to find and run Beam	The speakers point viewers to Beam’s Hugging Face page, developer resources, evaluation results, and supported inference options.
16:08	Model scale and pre-training choices	Alex explains how scaling ladders help the team test recipes and choose model configurations systematically.
20:20	Training for agentic capabilities and harnesses	The team describes coordinating pre-training and RL, developing the Daedalus and Mirror harnesses, and seeking generalization across tools.
29:42	Pre-training data and skill environments	Alex and Jess discuss curating pre-training material and building high-quality RL environments to teach useful cognitive skills.
33:17	Designing reinforcement-learning environments	Jess explains how the team targets behaviors such as sustained reasoning and self-verification while carefully checking reward quality.
39:34	Underexplored skills and data quality	The speakers consider future training directions, the role of environments in shaping model behavior, and the importance of reliable data.
43:39	The challenges of large-scale RL	Behrouz describes the infrastructure, stability, and debugging challenges involved in extended reinforcement-learning runs.
48:53	What comes next for Beam	The team says work on the Beam family is continuing and highlights the need for increasingly challenging environments and broader domains.
52:04	Why the team built Beam openly	The researchers share their goal of giving builders access to capable models they can adapt and control.
57:50	Open-source collaboration and integrations	Florent describes working with Hugging Face and other partners to integrate Beam into the open-source ecosystem.
1:01:31	Inspecting model behavior and traces	Ben walks through the Hugging Face blog and shows how traces and task-level analysis can help users understand Beam’s behavior.
1:08:47	Beam Build and fine-tuning options	The hosts explain how to submit projects to Beam Build and review partner-supported approaches to fine-tuning.
1:14:28	Demo projects built with Beam	Ben presents examples involving a native macOS app, a multi-agent research project, and a fine-tuned decision model.
1:19:07	Closing and ideas for submissions	The speakers wrap up with a suggestion for a demo idea and encourage the community to build with Beam.

Quotes

It’s just a very strong agentic model in my opinion. — 05:12

We expect people to fine-tune being. We expect people to build with it. — 07:52

we hold ourselves to a really high bar in terms of quality. — 34:50

if the data quality isn’t there, you’re going to have trouble training your model. — 43:09

All of these things, really whatever you can imagine, try it out — 1:10:22

Concepts and entities

Beam, Reflection, Hugging Face, Beam Build, Mirror, Daedalus, Transformers, TRL, vLLM, SGLang, llama.cpp, Alex, Behrouz, Jess, Florent, Ben, Sanyam Bhutani

Full transcript (original language)

00:32 Hi everyone. It’s Ben Bitesize here from Hugging Face and I’m with Sanyam Bhutani from Reflection. >> It’s it’s great to be here with you Ben. We worked a lot closely together. I’m really honored to share the stage with you for this announcement. >> Definitely, it’s really exciting. So today we’ve announced or Reflection have announced a new model on the hub. It’s called Bean. It’s really cool and and we’ve been kicking the tires on it for a while now, really putting it through its paces and it’s been a really exciting experience. It’s probably been one of the most open releases we’ve been a part

01:03 of. If you go into the blog posts from Reflection and from us, you’ll see that there’s really a lot to tuck into there. There’s integrations all the way across the ecosystem, whether you just want to run it or you want to train it, you want to customize it, quantize it, whatever. Like you can take these weights and and they’re they’re ready to go. This live stream we’re going to first of all just walk through what the model is about, why you would use it, how you would use it. Then we’re going to show you some of the practical integrations and talk about

01:33 those, things that actually have been built by the community, right? The folks at SG Lang, the folks at VLM, our site. Everyone’s come together to make this model run on your hardware, on your cloud. So we’ll show you all of that. Then we’re going to get the research team that that trained this model. We’re going to get people from the data side that built the data sets that that made this model what it is and we’re going to kind of go into the into the details, into the weeds and find out why they did things and maybe ask them some really hard questions that they cannot answer. Let’s see. And then at the end we’ll come back and

02:04 and I’ll show you some of the demos that I made and and that other colleagues made which just show you how to get running with Beam right away. Like some really easy demos that you can try without any hardware and some with hardware training and not training, these kinds of things. >> [snorts] >> Cool. So um >> Is this where we announce the prize? >> Right. Yeah, of course. So the most important thing I almost forgot uh is Beam Build. So Beam Build is a is a new hackathon project on the hub uh by Reflection. Do you want to uh

02:35 tell people why they would do it? >> Um we since we’re doing this in open, we’d love for people to try a model. Uh we’d love to give you a chance to build with it. And we’re sponsoring $10,000 cash prize for you all to build on Hugging Face. Uh build anything you want using Beam in the open. And I think you all are also pitching in prizes. >> Yeah, so it’s 10,000 cash, 10,000 hub credits uh going I think to top five. Um it’s super easy to do. You can actually

03:06 uh submit a submit a space. So any kind of demo that you could provide code the demo, you um you could build your own custom demo. You could train a model, fine-tune a model, quantize a model, any kind of model uh obliteration, merge, anything that you can that you can come up with and share that as a model. Or you can make a data set. Maybe that’s a data set with Beam. Maybe you make a synthetic data set. Maybe you make a reinforcement learning environment uh and you use that to maybe RL Beam. Um we’re really open to anything. And once you’ve done this, all you have to do is

03:37 go to the Beam or so Reflection Beam model page that the main model page, so the BF16 main home page. You’ll see the link in the description on this video. And you go to uh a comment on there and you just drop the link on there. You don’t need to explain what it’s about in that comment. Everything needs to be in that repo. So, if it’s a model, the model card needs to say what it’s about and then why you did it and and why it’s cool. Uh same for the data set, same for the space. Everything needs to be self-contained, uh self-contained project. And you can win uh yeah, 10,000 cash and

04:10 10,000 in hub credits. So, you can really do some some pretty big projects on that. Like it’s a really cool bootstrap fund uh for something you’re really excited about. Okay, the next thing we’re going to go into is just maybe Salim could tell us what the model’s about. Like like um we’ve seen uh a lot in comms about how it’s token efficient, how it’s focused on these agentic use cases. It’s really good at code. I’ve used it and and and I’ve experienced that as well. But could you kind of tell us the the big picture behind Beam and and why you think people

04:41 would use this uh instead of another model or what its focus is? >> Yeah, first of all, I’m a very small part of Reflection. We have a few of our research leads as well that I’ll introduce in a second. Uh I’d probably love to hear from them as well. For me, uh my POV is it’s a very token efficient model. Um so, for those of you who know Cad Jams have been saying it’s pretty fast. That’s partly because it’s token efficient. And it’s just a really good workhorse. Um

05:12 So, for me, I can comfortably delegate some of my small day-to-day tasks to the model. Um and leave it running in the background. It’s just a very strong agentic model uh in my opinion. But I I’d love to bring my research leads on stage and probably ask them what’s their favorite thing about Beam. Uh maybe you can start with Alex. >> We can achieve that. Yeah, let’s go. >> Great work. Well, hello everyone. Hello. Uh Alex here. I worked on pre-training data

05:45 for Beam, making sure that the base on which it is built is powerful so that it enables all of the incredible things that we’re going to talk about in a second. This Um I think uh and many people are going to talk about the token efficiency today, but this is probably the most important thing for agentic coding. All of the things that people want to build and create this Beam and custom on top of this. And making sure that the model knows about all of the possible development in

06:17 the world and reason about it has been a massive team effort. I think that if you have time, we are going to dig a little bit into that today. That’s probably my favorite bit about it, focusing on the knowledge, focusing on explaining and teaching the model how exactly we develop things in the year 2026. And the team is going to tell a lot. Yeah, it’s been a pleasure working on it. Yes. >> Yeah, that’s definitely something that I’m super excited about. So, anybody that’s worked in AI for the last like 3

06:51 years has had this experience where we read papers, we investigate what’s going on in the in open models, and we look at open models. And we’re trying to learn from them and understand these models. And then behind this kind of wall, there’s this other like frontier thing that we don’t know that much about. And we kind of maybe we hear rumors, and and we kind of we get kind of like very abstracted versions in releases and these kinds of things. Um but we don’t get as much as we do from the open. We’ve got less to learn from. So, one of the really exciting things for us as as kind of

07:22 students and just just people in the community about a lab like reflection is that we get stuff that we can like tuck into. We get people that we can ask these questions to. We get tech reports that are very detailed. One day we might get papers as well. And we also get all of the code, the harnesses, everything open source. >> Yeah. That’s also very important for a lab that advances open intelligence to write down more about not only how the model is built, but how but make it possible for everybody to

07:52 build on top of it. We expect people to fine-tune being. We expect people to build with it. So so many of the details of how it works are necessary in order for this to be most efficient for builders throughout. >> Cool. >> Yeah. >> Yeah, some of the most exciting things I’ve seen are like the traces, in fact. So we’ve got traces on the hub that that we created with Beam that you can go and check out. You can understand these trajectories and how the model actually works. But I’ve noticed that you’ve also shared trajectories from datasets. You’ve shared all your evals. You’ve

08:23 even shared the implementation of the evals and other models on the same evals. So there’s really a lot to kind of dig into there. So if you’re using this model in production, and you’re kind of going to deploy it to quite serious workloads, you you want to be able to go back to understand like what is that foundation I’m building on top of. So you’ve kind of got this like transparent base to work with. And I feel like that’s something that that exists here, which is really cool. >> Yeah, I’d I’d also love to introduce Behrouz and hear from him what he thinks is like the best thing about the model.

08:56 >> Yeah, hi everyone. I’m Behrouz. I worked on the high compute RL component of Beam. Um you know, to to really kind of push the agentic abilities and the coding capabilities that we see um you know, in the model. Um and yeah, it is it is you know, like it it packs a lot of punch.

09:26 And you know, for me you know, obviously you see a lot of the you know, the benchmarks and the like the numbers going up and all those things, but what really kind of clicked it for me was the demos and like some of the capabilities that the model is is able to do. Uh you know, the first time that you go in and you um you know, zero shot some you know, app generation or

09:58 uh that uh New York City um you know, live map where like this model has to go get a you know, information um build something multimodal. Um you know, that’s the the moment that at least for me it was was clear that this this is a very intelligent model that generalizes to to to these different domains and these different capabilities.

10:28 Um yeah, and you know, I’m personally super excited for for the community to go build on top of this, push it to to uh you know, different domains and different um you know, different settings. >> Cool. Yeah, thanks for that. So, it’s we’re all we’re all really excited. I guess people that are tuning in probably want to find out like how they can use it and maybe just get like a run-through of that like where it’s integrated. I know it’s integrated in VL and SG Lang and the teams there have optimized it

10:59 already. Maybe we can just get like a walk-through of that. Maybe you can show us some of the evals just to kind of save us some time so that we don’t necessarily have to read the blogs if we’re a bit lazy, but I would definitely recommend that you check out those blogs. Maybe we can dive into that. >> Yeah, maybe we can quickly screen share the one I figured out with Star by me. There we go. So, um for folks tuning in, you can find everything on our Hugging Face page. Um if you want, you can go to our blog as well. That should have all of the relevant links.

11:31 Um but I would say Hugging Face page and the developer website is the best place to get started. If you want to try our model, you can go to our first-party API. There’s very nice documentation there. Um if you want the model card, you can find it on Hugging Face as well or on our website. You can look some of the benchmarks, and we’ll hear the story how we got there in a second. But, you’ll see really good performance on agentic coding capabilities. And part of the story of reasoning that I’ve been saying is you’ll see the model

12:02 is very token efficient. And I want to also tease out the story with our leads because if you look at the reasoning of the model, it’s it has a very fun character, in my opinion. So, we can also talk about that in a minute. >> Cool. Yeah, let’s wait for the leads to arrive. Another resource that you should definitely check out is the blog on Hugging Face. This as ever is like our kind of uh homepage, I guess, or sort of like a light tech report. One thing I want to say is that this is probably the most

12:32 like the most invested blog post that that I’ve written whilst I’ve been at Hugging Face. Um and I probably got a little bit carried away on this, but um it’s really like there’s a lot to it. So, um I would probably this weekend put some time aside to it and go and check it out. So, the first thing to say is that like many model releases, we get this that this pre-release we we work with the the model authors to to share the model. So, we get the privilege that that we can try it out. Uh and and I like to kind of uh capitalize on that privilege and and sort of share that with the community so

13:03 that you can you can dig into those things. So, um if you come to this blog post, first of all, what you’ll see is that okay, we already mentioned some of the stuff we mentioned here like Build-A-Bear. The next thing that we do is we we walk through like like what it is from a simple level. So, first of all, it’s integrated into Transformers, which means that the model architecture code is is open, uh and it’s also uh in a canonical form. So, so Transformers is the central central source central source of truth for for Transformer architectures, but also LLMs and and VL

13:35 LLMs and and AI models. So, that means that in this way we can compare the architectures across different models and we can see what it looks like and and how it’s different to others and and how it’s reusing from literature. So, I would definitely come here if you’re interested in ML and kind of tuck into that section. We went through this. We worked on it and and kind of racked our brains about what it was doing and why these things would happen. We shared it back for a reflection. They they validated us. I think they validated what we did. And you can kind of learn from that. So, that’s a really educational section I

14:05 would I would check out. Then of course we have the evals. They’re also on Reflections site and they’re also we even have community evals on the hub. So, you can compare the model with other models from other authors on the hub. So, go and check that out. And then what I would say is like the kind of quick start section is is this inference support. So, if you want to use it in Transformers, this is the snippet to do it. This is where you come. If you want to chat with it, this is how you do it. If you want to build an application on top of it, this is where you do it. And the Transformers

14:36 engineers have have worked on this blog with us. So, everything here is like validated to a high standard. So, we know that this is the best place to go and we’ll maintain this blog with our docs. So, over time you can use it as as things change. If you’re using VLM or SG Lang, we also have examples here. As I said, these are implemented by the respective teams. So, by VLM and SG Lang. There’s also Deep Flash 2 for this model in both inference libraries, which you can use to optimize inference. I would definitely recommend doing that.

15:07 Something that’s I think pretty cool about Beam as well is that it’s shared in BF16 of course, FP8 and NVFP4. We don’t always get um FP8 uh from model authors. So, so this is really useful, right? If you’re using newer Blackwell chips, you can obviously take advantage of the MVFP4 checkpoints, but if maybe you’re using Hopper, you can fall back to the FPA. That that would be your your best bet. Um I get you would use the the BF16 in a kind of

15:37 fine-tuning training training setting. Uh and another thing that’s that’s cool on top of that is that Reflection have also shared the evals on the on the quantize quantized checkpoints, which means you really know what you’re working with. I would I would check that out as well if you’re going to use this model. Um and then I’m going to go into the next set section a little bit later once we’ve had the the research team on and once we’ve asked some questions, but um here in the rest of the blog post I go much deeper and I kind of try some real-world use cases, some real-world

16:08 agentic use cases. >> Also, should we get into Q&A? >> Yeah, let’s do it. >> So, uh before we jump there, I would like to, you know, at least from my perspective, say one thing. That we hear stories about like people that really have an intuition about things. And you hear about people who knew math at their core. I feel that’s the honor I’ve had with my research team. Um for the moments I’ve had to share with them, when they talk, everyone just

16:40 tries to absorb as much as we can. And that’s why I convinced Alex and Verena to join. I’m very grateful they came on. Because they they they have a strong intuition about everything. And we want to also try to open source most of it that we can. Um so, I’ll probably start this question with Alex. Um he’s a pre-training lead. How did we get to 501B size? And what was the journey there? What was the science behind deciding this specific model size?

17:12 >> Yeah. A lot of the team built up the pre-training methodology here but uh over the course of the year but maybe as a high-level intro for folks listening in curious. You when you build up to a large pre-trained model you want to build it up in steps where you test your recipe carefully methodologically scaling it up

17:44 one tier at a time iteratively in bigger and bigger models and every time you introduce a new data set or you test a new change to the architecture to the optimizer you want to test how it works at several different tiers and see that it improves the quality of the model at scale and then you test it on a bigger one and verify that your prediction holds. This is the scaling laws that we all heard about that if you built this

18:16 selection of tiers right then you’ll be able to test every single thing at smaller small and bigger and bigger tiers and predict how it will eventually behave at the larger tiers go and test your prediction will hold. It held for us. So the most important thing really goes into building the scaling ladders so that the predictability holds and the specific sizes whether it’s active parameters the total parameters they kind of fall out of that and these are another things you mainly

18:46 control you control many things like the output ratio of the the number of layers and the aspect ratio of the residual weights to the number of layers um but then from all of the variables that are controlled on the ladder the exact configuration of the model falls out of it. You have to watch out for many things in the ladder design. It’s a difficult thing many people have opinions about the best one so it’s an art really in the community. Uh Uh like

19:17 Is the model going to fit on a particular GPU configuration? Will RL be able to work with it? Does it fit in one node or two nodes? What happens when you deploy that model on a consumer hardware? What exactly is the throughput? Um like you really don’t want to design carefully a ladder and then figure out that if only you reduce it by a couple layers, it would be able to fit in that configuration and then go back and redesign this again. For this.

19:47 So, we spent I think like at least a month thinking carefully about this. That is part of the pre-training science that is often done inside of the labs. Um Beam, as it turns out, is a specific point on the ladder. We can go further than that, but I’m not going to talk about it today yet. This politics. >> And I’m I’m curious, uh maybe Behrooz, were you exchanging notes with Alex on his requirements

20:20 on data? How do you like set the model up for success from beginning to the end for because we were we were really aiming to be strong at agentic and we knew that from the get-go. How How do we decide from the beginning and what’s the science behind that? >> Yeah, so the you know, the process was a very kind of a close collaboration between the two teams, um you know, for for months in advance of the like

20:52 the final model training. Um you know, on both, you know, what what Alex mentioned on the like a you know, model size, whether we can sample effectively from it during RL, do we have enough you know, KB size? And also on the data size is is like a you know, during this process, we you know, use these smaller models in the skating ladder to to, you know, push

21:22 them through RL, identify the strength points, the the places where we need, you know, extra data, like targeted kind of um data additions in their like a pre-training mixture. And um you know, one of those things that you really want to to have and, you know, we we were able to actually get with Beam is before you launch your, let’s say, final RL run of of, you know, final run, you

21:53 want to you know, have a good prediction of where you’re going to end up. So, you know, none of these things should be a surprise. And we spent maybe, you know, the you know, 6 months before we we went through the the final model launch to kind of build that prior and kind of build that science and understanding of, okay, the model is going to be, you know, in in this shape. We we have done all the kind of interventions that we

22:24 needed to do either on the RL side or pre-training side to just make sure those agentic capabilities really pop. >> That’s cool. Thanks for that. So, I mean, leading on from that, it one question I had was around Mira, that the harness, the Rust-based harness that you built. Like, could you maybe tell us the origin story of that harness? Like, how is it involved in post-training? Like, um and maybe even kind of allude to the

22:54 models like cross-capabilities in other harnesses and and and kind of, yeah, the origin story of the model in a harness. >> Yeah, so you know, the the like harness that is uh kind of inside being um so you know, sorry. Mirror is is built on top of another kind of a smaller harness that was used during training. Um

23:25 we call it Daedalus. And um you know, we we use that to to kind of tune our learning, whether we want to focus on um you know, token efficiency or control the latency or if you want to kind of teach the model um let’s say, you know, certain kind of a compaction behavior and then whatnot. Um on top of all these things,

23:56 we also want to make sure that the model generalizes to to different harnesses, different setups. Um I think this is something, you know, very special about open weight models. Um and to do this during the RL one, we do a lot of augmentations, a lot of we we introduce the model to a lot of, let’s say, diversified environments just to make sure that the model is

24:27 um has seen a lot of these different behaviors. Um I don’t know, Ben, if this was >> [snorts] >> this is answering your question. >> Yeah, totally. Um it answers us like where the harness came from. I guess I’m when I look at models, I I’m always thinking about uh my own projects, right? I’m I’m I’m always thinking like, okay, um so how can I fit this into my project? And I think as we start to see more harnesses come out and and kind of

24:57 a there’s a becoming to be there’s beginning to be a one-to-one relationship between models and harnesses or at least model series and and harnesses, right? Um and my like default place to kind of optimize for my use case would be the harness probably. I would probably optimize the harness before the weights. Uh and I think maybe that’s just a a product of like my own skills and like maybe the size of the teams that I work in. I don’t necessarily have access to kind of to fine-tune the model for this harness. So, I guess um what I’m trying to do is like

25:28 like enrich my decision-making process about how I would choose mirror for beam or maybe I would choose a harness that I’m already working with or maybe I would customize a harness or maybe even I would customize beam’s weights for a for another harness. All these kind of options that are available to me because the model’s open essentially. >> Um I’d actually like to raise this question. We although we didn’t focus on other harnesses, we after training the model realized it generalizes as well. I plugged it into pie, I plugged it into open code. We ran some benchmarks and we

26:00 didn’t see any drop in performance. What’s the secret to getting there as well because that wasn’t our main name if I may. >> So, you know, this is model has seen different kind of a diversified tool calling behavior. And you know, in in the kind of the RL run and you know, the the kind of the

26:34 it’s is you know, the RL capabilities is good to kind of like separate out the instruction following capabilities versus the core reasoning and agency capabilities of the you know, of the model. So, with the you know, like a long high compute RL one, you get a lot of those core capabilities in the model and then we, you know, we do introduce a lot of like randomizations, augmentations and uh, you know, diversified environments just

27:04 so that we can, you know, we can generalize to to these different environments. Um, and you know, the hope is as we just build kind of a stronger um, you know, reasoning capabilities and agentic capabilities, the, you know, the gap that the extent like a expect from different harnesses just shrinks. I think that would be the the ideal

27:35 setup for for an open rate model. >> It’s also kind of interesting but even with Mirror, so maybe one thing that’s worth highlighting here, um, we don’t have Jake here on the podcast with us. Jake is uh, the person inside Reflection who initially built Mirror as a, well, as a tool to help us all do our work day-to-day. It is an uh, dedicated harness, blazing fast, built on top of a

28:07 lot of our internal infrastructure to connect with Slack, Linear, all of the tools that we see with the data. Um, but then as we started experimenting with models, we realized, “Okay, so we can really co-develop it together with the model and make sure that this is the way through which people work and experience with Beam. The default harness was not harnesses.” So, this data uh, like developing the Daedalus harness as a default one on top of which Beam is

28:38 trained and then also turning it into Mirror through which people experience it was really the part of like the positive evolution which event and then from there as we discovered that it generalized to other harnesses as well. You can see the experience in open code experience in Pi and whatnot. People started branching out. So, it really developed in stages from a product to a native harness to all other harnesses. I think over the course of

29:11 rapid fire 3 months or so. >> Cool. Okay, thanks. So, that’s really cool to hear about the harness. If you’re interested in third-party harnesses, you can check out our blog post as well. We also tested it with Pi and we shared some of those traces. But the best thing to do in that case is really just to run it within an inference engine and connect it to to whichever harness you want, right? Because the integrations are there. It’s it’s good to go. >> And honestly, I’ll invite everyone just to tell us what you all want to see from us. We have the

29:42 leads here from our team. If you if you send it, this is like a direct line to our open source team if you may. We’ll make it happen for you all. We’re doing this for the community and we’re very happy to share this with you. Um Jess from our team also joined. Uh she leads the data and skills team. So, maybe I’ll queue this question and I’ll also ask for your intro here. But um maybe for Alex and Jess, what’s the journey to collecting the 23 trillion tokens that we trained on? How did you

30:12 go about that? And then how do you define the skill set the model should eventually learn? How do you go about collecting those environments? >> Awesome. I mean Let me start with the pre-training part of it and then Jess, uh you should take the stage and tell us about the cool part of the environment. Yeah. Look at this. So, pre-training data is not the data that you usually see described. We’ll try to describe it a little better this time.

30:43 This cuz when you think about what your model should know before it really goes into the high computer reinforcement learning run, you focus on the overall like quantity, quality, and diversity is how we usually describe it. It is not just collects all of the day pages from the internet that you have access to. It is really a do meticulous work to break down the educational

31:13 quality of every page, every PDF, every code document or out there in the open source. Uh every single written uh document, an essay, a page, or whatnot. And then carefully study how much should the model see from each source. Adding up things like, “Okay, maybe we want to really make sure that the model knows the documentation for this open

31:45 source library and repeat it paired with its actual implementation a couple times so that becomes really centered in the base weight of the model before it goes and learns how to actually complete tasks with the usage of it.” Similarly, for like mathematical data in STEM, a lot of the like base weights of the model need to be uh enriched with the knowledge of what exactly are the foundations of math

32:15 and physics and in in this like these bits that are through which develops reasoning every time the model reads uh a Python notebook on the web or a proof interleafed with English and formulas, that’s something that sets like uh foundation for eventually learning the reasoning process. robotics. So, we invested quite a bit into this curation process. There I say that some of the things that develop there probably

32:46 uh state-of-the-art in terms of both ensuring the precision of the content that we train on, as well as recall that we don’t filter out things that eventually would be educational. We’ll describe more and more of it in the tech report. Yes. Uh but then on top of it, I think what is real magic is learning the skills in the model. So, this is where Jess, you should take tell us a little more. >> Um yeah, thanks, Alex. Um so, yeah, I

33:17 think like you [clears throat] know, obviously there’s like a few obvious buckets that, you know, we would expect like any model worth its salt should be good at like math, science, code, like using a computer terminal, etc. But, you know, one of the ways that we think about actually like building up these skills in the reinforcement learning process is not just about the domains, but about the actual like skills, like the cognitive skills that are required to um support like good behavior in those domains. So, you know, for example, um

33:48 the model needs to be able to reason over an extended amount of time, and in that reasoning process, you know, it should be exhibiting like sophisticated behaviors like self-verification and backtracking and trying new strategies. Um you know, and so, in order to encourage this type of behavior, we try to come up with reinforcement learning environments and questions that are not just hard because they might, for example, require like niche domain knowledge, but because they really force the the model to think through a problem logically. Um and then, you know, in reinforcement learning, if if the the question, in

34:20 order to come up with an answer to the question or the solution to the problem, if the model does have to engage in this process with these like kind of more sophisticated cognitive skills, if it gets the answer right, then that gets reinforced, and that’s kind of, you know how we end up play shaping that behavior. And you know and I think um you know similar to what Alex was saying like we don’t just try to get all of the data that we can or all the environments that we can. We hold ourselves to a really high bar in terms of quality. Um so you know that means that the questions don’t you know it’s not just

34:50 that they need to be hard and like practice the skills that we want it to learn but we have to ensure that um you know there’s basically nothing wrong with the environment so there’s zero noise in the reward signal. So that means that anytime the model like gets the answer right it has truly gotten it right and truly the reward truly corresponds to like good behavior and anytime it gets penalized that truly corresponds to bad behavior. So that means you know we have to avoid like the you know the the model shouldn’t be able to guess the answer. The verifier shouldn’t be you know sometimes

35:22 rewarding the same answer in different ways. Um Obviously you know we need to avoid things like reward hacking. Um there shouldn’t be multiple possible answers um that you know some get marked correct and some get marked incorrect etc. So um you know we um put a lot of effort into doing sort of like an iterative like you know sourcing data, reviewing the data, and then testing it in RL. Uncovering new problems and then using that to like inform this loop in order to build the

35:54 highest quality data set possible. >> That’s cool. Thank you for that. I mean I’m really excited by this this domain as as well um and uh so one thing that I’ve heard about I think I think Nathan Lambert shared about this. He described like post training data sets as like wrangling an org chart as much as the data itself. You need to bring people together in order to work on a domain and and they need to like understand the domain and the environment and they need to know what it is the agent’s doing in that in that environment and then um,

36:26 you to to train a model on it, and then you need to evaluate that model and kind of iterate on that cycle like you’ve already said. Could you kind of give us a picture of of what that looks like inside a team? Like, um, how you collaborate and and how you kind of take ownership of of the skills that you spoke about and the environments and and yeah, um, just your kind of way of working in that way. >> Yeah, that’s a great question. Um, I mean, we have a pretty small team, I think all things considered. Um, and um, so, uh, you know, we kind of like identify the like, um, the highest

36:59 priority domains that we really want to make progress on. Um, and then with it for each of one of those domains, we’ll have like a small team that specializes in that and gets really familiar with the particular type of data, we’ll say whether that’s like math or like software engineering or or so on. Um, and um, and that team is in charge of sort of like everything about that domain. Whether that’s like the evals that we’re tracking or like, you know, talking to data vendors, um, we do we purchase some of our data from vendors, um, or like, you know, going out and figuring out how do we generate some of

37:29 that those environments synthetically, um, and um, and so those those, you know, work streams or teams like work really, you know, tightly together to, um, you know, build the environments and then we like work very closely also with like the training team, um, which Barry can talk to more about, um, in order to, you know, put [clears throat] the data through RL and then, you know, see what behaviors come out of that, what things do we want to, um, you know, what things are we do we like that we’re seeing and we want to, you know, make sure that we continue reinforcing, what things, you know, what are the surprises that we we

38:01 want to fix going forward, um, and, you know, and that’s also like a really important, um, to have like a tight loop there because, uh, you know, some of the, um, uh, you know, some of the things that you you might not be able to necessarily tell from the problem itself will come out in reinforcement learning. And so, we really have to, um, you know, have that tight loop there. So, you know, I think one example of this was we had earlier on in the development of the model we had some um like science questions that were

38:32 um it seemed like that they were difficult questions. You know, we would like you know, you know, test them out with like you know, taking multiple samples from the question and then okay, like the model’s like you know, sometimes solving it sometimes not solving it. Um but then we put it through RL and the um the reinforcement learning algorithm uh basically like caused the reasoning um process of the model to like drop to zero. And the reason was because these were actually tended to be more like factual knowledge questions that the model didn’t really have to reason about and that wasn’t so

39:03 obvious to us until we actually put it through the reinforcement learning process. >> Okay, yeah. So, that that’s really cool. So, that paints a picture that yeah, most people in the community um wouldn’t necessarily have uh that much experience around, but people will be taking models and fine-tuning them for their domain. So, that that’s kind of one thing that like I think is quite exciting there that um if you are working with Beam on kind of a specific use case, you can say okay, let’s set up this kind of like domain-specific group

39:34 in the same way that you that you did internally, but like we’re going to wrap around our problem and we’re going to kind of work together on on uh say data and then RL and then eval kind of loop like that. Um which leads me to the question like which domains like which skills do you think were kind of underexplored? Like like where do you see kind of um potential uh for uh for Beam? >> And to to tie into that, there’s a question from the community that I’d probably add to that. What do you think belongs uh

40:05 in the environments uh versus in the weights and any things that didn’t work during this process? >> Um yeah, all good questions. Okay, let me see if I can answer all of them. Um, so the I guess in terms of skills that fell under explored, um, I think and so I would say actually like on a lot of the benchmarks that we were tracking, like the scores were still going up at the end of training. So to some extent we just need to train more on the stuff that we were already training on. Um,

40:36 uh, I you know, I think that that’s kind of just a limitation of we didn’t have enough environments, um, to to keep the run going. Um, I think, um, maybe like, you know, more forward-looking, um, one thing that I’m really excited about is is like the ability to train the models to engage in a little bit like more robust like human agent collaboration explicitly. Um, you know, I think especially like when you’re considering like integrating an agent to your workflow like, you know, some sometimes you want it to go off and do something completely autonomously, but

41:07 sometimes you also want it to, I guess, you know, bring you along for the ride, right? You want it to, um, do things in a way such that you by the time it’s finished doing what it’s doing, you as the user understand like what it what the model has done and why it made the choices it did. Um, and then that enables you to kind of like continue building on that work like going forward. Um, and um, so that that’s a particular direction that I that I’m quite excited about. Um, let’s see. Uh, what was it what was the second question? Can you repeat that?

41:37 >> Um, things that worked and things that didn’t and what belongs in the environment versus >> Yeah, what what >> data sets? >> Yeah. Um, so yeah, what belongs in the weights versus the, um, environments? I mean, well, to some extent like the environments are what helps shape what’s in the weights, right? So it’s it’s less that it it belongs in the environment versus the weights, but more that the environments are means towards like shaping the weights in the way that

42:07 we want, right? Um, we want the the to contain these like, uh, you know, as I was saying, like these cognitive behaviors or cognitive skills for reasoning, um, for tool use, you know, for operating autonomously for long stretches of time, and like and that, you know, that behavior doesn’t, at the beginning of training, doesn’t really like fully live in the weights yet. There may be the seeds of it are there, but then the environment is what allows us to sort of like pick that pick pick those behaviors out and sharpen them and strengthen them, so

42:39 that when we like get to the end of training and we release the model, it’s a good, you know, coding model. Um, and, um, yeah, I guess like things that worked and things that didn’t work. I mean, I think again, like, you know, kind of coming back to like the data quality point, I think that is something that is, I don’t know, everyone loves to really talk about like, um, you know, the reinforcement learning algorithms and so on, but like fundamentally, like, you know, if the data quality isn’t there, you’re you’re going to have trouble training your model. And so, you

43:09 know, anytime that we like overlooked something or cut a corner and then we tried to train the model, we would always find that, you know, that would result in problems. Um, and so I think, you know, again, like, having that attention to detail and always coming back and like looking many, many times at the environments that we have and making sure that, you know, they’re they’re really fully there is like one of the most critical aspects to getting this to work. >> Um, maybe it’s a skill issue of mine, and

43:39 maybe Behrouz will never talk to me after I say this, but I tried setting up RL and it collapsed completely, even though it was a very small run. So, what’s the secret to making it successful at the scale we did it, and can you talk to the scale at which we run our, reinforcement learning run? >> Yeah, I mean, RL algorithms are notoriously, you know, unstable and kind of like really takes, you know, a good amount of

44:10 time to to kind of achieve that stability and um especially I mean at at this scale of let’s say, you know, one one month of a like a continuous RL run on just, you know, like a you know, 10K GB200. So, this is a you know, something that is like a frontier kind of a lab scale of of a of an RL run. Um and for us

44:41 it was both an infrastructure kind of a build-out uh process and an engineering challenge and at the same time like a scientific challenge. We had to kind of go deep into the algorithms. And you know, also build out the partnership between these these these two arms. Um so, you know, oftentimes the two things that really become problematic is that, you know, as you optimize your policy, you

45:11 also optimize against the you know, small numerical divergences between your inference engine and your training engine. And you know, in a kind of a short RL run, this this is, you know, completely fine, but as you’re putting more compute on it, more optimization pressure, this becomes catastrophic. And at some point the the policy just learns to optimize against this divergence and and then you you you just see that, you know, suddenly your reward either kind of, you

45:42 know, go like it drops or your you know, completion lengths blow up. Um so, that requires uh you know, like a kind of some sort of a consistent uh study of the dynamics and you know, putting the fixes in into the stack. Um and on the engineering side, so you’re suddenly you know, handling um, an async system with different kind

46:13 of fault tolerance behaviors. And, you know, these chips or like, you know, your inference engines come in, uh, you know, like sometimes they they die. Um, and as you just, you know, scale these these up, uh, it’s it just the the, you know, scope of these engineering challenges increases, the failure rate goes up. So, you really want to have a system that is kind of a fault, like has has a like a strong amount of fault tolerance built

46:44 into it. At the same time, it can, um, it has enough observability so that the folks who are working on the like a science aspect of it and the stability aspect of it can actually debug it and see that, okay, you know, this this kind of a component here, um, you know, caused issues. Um, you know, in in our own journey, we started from our L runs with maximum context size of, you know,

47:17 16k, which, you know, is is is rather limited. We don’t like it’s it’s very hard to do any kind of a real coding, or like the long horizon agency tasks there. And, you know, we just built out the infrastructure and science together. We went to 32, we went to 64, and, um, you know, just just extended to 256 and and beyond in in the, um, you know, like a run up to the run.

47:49 Um, so all those things kind of need to come in week after week, month after month, and, uh, the lessons need to get absorbed into um, you know, our stack and our algorithms. >> Okay. That Yeah, that’s really cool. We’ve gone really deep into into RL and and and and heard about that run. And and if anyone wants to kind of learn about that, they should definitely go and check out your blog post uh, as well as all of the RL integrations that you’ve done, which are really cool. So, I think there’s TRL, there’s

48:21 uh, Miles, there’s um, Sky RL, uh, there’s Nemo RL. So, really like like pick your your flavor. Um, and and you can try out Beam. So, so that’s pretty good. That you’ve also, which I thought was great, is actually uh, collaborated with the community to do those runs. So, people can use the frameworks and then they can also see the results that you got on a specific example, which makes things a way easier to start from. May- maybe if we come outside the last uh, one or two questions, oh, I’d I’d want to ask like I saw in the plots and

48:53 you already mentioned it that there there isn’t really a plateau. Like it doesn’t look like um, necessarily the end of a run. Uh, looks like there there’s potential there. What does that potential look like? Like what’s the next step for Beam uh, in in its forms? Like is there another model? Have you considered other sizes? Like like where are you going with with Beam? >> Let me maybe take a bit [clears throat] of this. So, like obviously we are already working on the next models in the overall Beam

49:27 family. Without sharing the specifics on the sizes, the specifics on the data and where exactly we go from here, but that’s already the case. As people mentioned, even uh, for instance, if you take the Beam as currently is and you continue doing RL on it, we have not seen the signs of that stopping. Uh, so you could continue them going. But also, so like we’ll share a little more uh Luke and talking about the model architecture, but there are so many things uh in the architecture, in the

50:00 optimization regime, in the pre-training data that we would be able to rerun from scratch and get a substantial more powerful base on top of which you can apply the again. And at that point, let me just keep pushing the frontier further and further. So, that’s already underway. Uh I don’t know, Jess, do you want to say anything about the uh skills data as well? I think there’s I don’t know this >> Yeah, I mean, I think for us, like, um

50:32 you know, uh we need to kind of continue to build out our like pool of environments that we can train on. Like, you know, as we build a more more and more capable models, like, the the problem with reinforcing learning data is like it gets exhausted, right? Like, the model improves and the data becomes too easy and you can’t really use it anymore. Um which is uh you know, unfortunate means you always got to be getting more and more environments that are harder and harder. So, that’s a you know, a big challenge for us is to you know, keep raising that like difficulty bar. And um

51:02 you know, I think also like expanding into um you know, like other domains that we’ve invested in a little bit less. Um so, you know, I think that there’s like a whole um you know, range of like agentic tasks like that you know, are kind of like increasingly more complex and longer horizon that we haven’t yet. Um uh or we we’ve sort of scratched the surface with I would say and and we haven’t like, you know, gone deep enough on yet. Um and you know, uh and so that’s something that I’m really excited about. Um and um

51:33 uh yeah. So, you know, onwards and upwards. >> Cool. Yeah, I I mean, we’ve we’ve got a wealth of of knowledge from this. It’s really cool. Um I’m going to do my best to share everything in the comments uh in the description as well so that this video in itself can be really educational. People can use it as a springboard. Um, but as I’ve said already that this kind of knowledge is is super useful to the to the community. Something that you can really learn from that you can’t necessarily always extract from a paper or a technical report. You know, the kind of like the

52:04 intuitions that that you’ve built through your careers. Sanyam, did you have a last question or do you want me to kind of talk into the demos and these sorts of things? >> Uh yeah, I mean for me, I’m never going to talk to all of them because they’re so busy. So, I’ll I’ll take this chance until the next release. Um if you want people to like go home with one takeaway about Beam or one thing that you’re most proud of uh that you want them to know or want them to try, what would it be? And question for all all three of you.

52:41 >> If I keep repeating the same things that I already said so I’ll just say it 10 times, it will be boring. So, the maybe the takeaway for everyone for me is why I joined this in the first place about a year ago, which is the the Western world been increasingly lacking a model on top of which everyone could build adapted to owning AI in their own application, in their own ecosystem, with their data, with their

53:11 environment, but end-to-end controlling it and knowing that it’s been built end-to-end with uh the right data and patient provenance here. And there’s of course a lot more to do and furthermore in the coming months, but this is now something that we built in less than a year that we can be proud of. So, people can own the deployment of their AI stack on top of a

53:42 model built here that is advancing the frontier of the, uh, artificial intelligence. This is something I’m incredibly proud of. That’s why I joined Reflection AI in the first place, and I’d love to see what people actually do with it in your own startups, applications, home projects, you name it, because it’s open it’s open for all. >> Um, I guess I can go next. Um, I I mean,

54:14 actually to echo what Alex said, I think it’s, um, yeah, something that I’m also really proud of is like I think, um, you know, I I think that, uh, you know, everyone should have access to, um, cutting-edge intelligence and be able to use it in the way that they want to use it, um, and have control over it. Um, you know, I I think that that’s like just such an important like aspect of, um, this era that we’re entering into. Um, and, um, I you know, I personally feel very like

54:45 proud to have been able to be a part of that. Um, and I’m really excited to see also how everyone is using the model, um, in like, you know, all so many different ways that we wouldn’t even be able to think of. And I think that that’s like one of the the really cool things about like working in this space of open models is to be able to see like the diversity and the creativity of the community building on top of it. And I also wanted to give a shout out to our team. Like everyone who’s contributed to building the model is like, um, I don’t know, working with this team has just been phenomenal, and it’s a small team,

55:17 and like what they’ve managed to build over a year is like I just extraordinary in my opinion. Um, I’m I’m so proud of like everyone who’s worked on this, um, for putting together, um, such a great model. So, that that’s, yeah, what I’m really proud of. >> Yeah, I also like really want to yeah, all the points that Alex and Jess raised. Um you know, we are entering you know, like a an era of stronger and

55:48 stronger kind of agentic AI models and you know I’m personally really excited about democratizing some of these capabilities and really allowing people to you know, to to interact with these things and kind of reducing some of these you know, like these these these gaps that people just just hear about as like okay, you know, these frontier labs that are closed have access to you know, some

56:18 some magical you know, whatever that is inside and you can never know about it. Um So you know, I I think I mean, this is one of those wonderful things that um we can we can bring into this community where people you know, have access to these tools can integrate it into their systems. Um and uh you know, feel connected to it. Um And you know, also really want to echo the the team aspect of it. Um

56:49 it’s really amazing to see all these things come together in less than a year. Um And you know, these these models have not plateaued. So, it’s it’s just we we’re just like moving towards like the next thing. Um and and and just seeing that you know, the these guys came together just built it. The scale of of these operations. Um and also the the energy inside it it it is just

57:19 amazing. >> Cool. So, thanks for that. Thanks for everything that you shared. Um it’s been a really cool experience for me just just myself. Uh and so I I hope that everyone we that we shared it with and that gets to watch this video as it’s recorded on YouTube, X, LinkedIn can like kind of extract this and learn a lot from it. It’s really cool. So, good luck with the with the next model. Good luck with the next releases. Um, next I’m going to take people through how to use it. Um, but I’ll let you guys drop off the screen and then

57:50 Sania and I will stay around and and I’ll go through all of that. So, thanks everyone. Bye. >> Um, while you while we transition, let me introduce our open source lead. Um, he’s an incredible runner. If if he if he was just running, he would probably set the world record in running speeds. But, he set a record with our launch, I believe. Um, he really sprinted and led our team towards the launch. So, I’d love to introduce Florent to the stage. Uh, I always mess up your name. I’m sorry

58:21 about that. But, Flo, please tell us about yourself and what’s your favorite feature for of Beam. >> Hey. Hi. Um, >> uh, sorry, Florent. You’re muted there. That’s cool. Don’t worry about it. Um, if you just unmute yourself, we’ll start it back up again.

58:52 Uh, >> I can I can actually tell you. Are we good now? Okay, that’s good. All right. So, yeah, so as Sania introduced me, I’m Florent working on open source uh, with him and like a big team and the goal is really like to uh, take the model and and bring it to the community and it’s been great uh, working closely uh, with the research

59:24 team, with the engineering team at Reflection. Uh yeah, to make yeah, to make sure that we can build like great integration uh with Beam, with the rest of the community, and the and the the tool that powers the ecosystem. So, yeah, what’s great working on open source at Reflection is that you get to work with really smart people uh that we have internally, but also you get to work with the partners. Ben is one of them. Uh we’ve worked uh together

59:57 on uh integrating Beam into the Hugging Face stack, uh Transformers, uh TRL. Um and yeah, um building great content about Beam and making sure it’s adopted by the community. And we in the open source team, we also work with like uh dozens of other uh partners from uh different projects. Uh we work with uh also inference provider, cloud provider, to make sure uh Beam is actually distributed and actually uh used by uh

1:00:28 the community. >> Cool. Amazing, Florent. Thank you so much for that. Um that’s uh yeah, it it’s really cool to see a kind of a new lab be born in the open, a kind of like uh like, you know, turn from the caterpillar to the butterfly or something like that. And like you can kind of uh spread it I I shared this tweet actually a few weeks ago when you first released Beam. It was like, “If you’re working in open source for the last few months, I think you’ve been down in the trenches. You’ve actually interacted with Reflection

1:00:59 engineers like on SD lying on the LM, on TRL, all these places.” Uh and and yeah, that that it really is true. And And if you go out into the docs of of all of those projects, uh you’ll find recipes and examples that are going to bootstrap your your projects, your startup ideas, uh yeah, your kind of hacker projects. Um let me take Florent off screen, and then I’ll um and I’ll move over to the blog post, and I’ll and I’ll show you how to start those projects. Cool. Okay. So, uh yeah, let me dig in here and kind of go down to the technical part of it.

1:01:31 Um So, as I said, we’ve got this blog post on um on Hugging Face. As we often do, this is a really um yeah, it it it’s a really deep uh blog blog post compared to some of some that we’ve worked on. And and we really invested time in kind of showing you like what Beam is like and and how you can use it. Uh as ever, you can like see every single model and data set that that we talk about on the right and then and your spaces, so you can just kind of um springboard off to those. Um but

1:02:03 let’s start off here. So, after the inference section, as I mentioned at the top of the video, if you just want to run the model, go to the inference section. That’s got real um SG Lang inference endpoints, inference providers. So, you can try it out on, say, uh Base10, Fireworks, and other providers. But, if you actually want to start building with with Beam uh in many ways, I’d recommend starting from this section. It’s called What We Tested. And this is this is uh my work. So, uh when I first got the model, uh we

1:02:35 actually always do this. We we set up some tasks that we think will um will kind of like elicit the the the capabilities from the model based on really what the what the model authors tell us. So, so they normally say what it’s good at. So, we we collect like a a series of of benchmarks, and we’ll benchmark the model. Uh and and we usually match the scores that that um we’ll get. And then, what I like to do, and and I know other people do this, is to then look through the tasks in the benchmark

1:03:05 and kind of understand in specific tasks how the model was behaving. So, um and and then you can really dig into a specific task. So, um for example, here you can see that that we uh we switch over to a trace. And uh this is a a trace on the hub which I’ve shared where I used uh the pie harness with beam. Uh and and I set up this task. So, it’s for synthetic order ORD 742, find the cheapest single warehouse that can fill

1:03:36 the entire order and deliver by order deadline. So, this is a a real-world uh reasoning problem and I used harness. Uh I used the pie harness for this. So, as you look in here, you can see like it’s all rendered like it uses the get order tool, get inventory tool. These are all custom tools uh that that I defined. I also shared those custom tools in a bucket. So, if you just want to like take this project and and set it up for your own uh you know, kind of use it for your own project, you can just take that bucket and download it. Uh and and then you see like, okay, it’s completed that

1:04:07 use case. So, let’s go back now to the blog post. Um and so, I I repeated that a few more times. So, then I did a kind of long context retrieval over a code base. Uh and you see here, okay, so in low reasoning, there’s a bit of an there was a a bit of an issue. So, low long context response got all code behavior. So, it got it right, but it didn’t reason. And this is how I always work. So, I’d always say take a a set of take a set of capabilities and then match those up to a to benchmark. And then

1:04:38 find tasks within the benchmark that correspond to those capabilities. And then start to see like which capabilities require reasoning and and which ones don’t. Because when we look at uh evals from a model on a on a certain benchmark, that will typically be with a a specific reasoning parameters. So, you use the highest reasoning parameter. And you might want to know, okay, so what is it that requires that reasoning? And what could I what could I actually reduce the reasoning on uh and kind of match up to my own use case and really unpick the model. So, you want to push it beyond

1:05:10 its limits to find out where it needs reasoning and and where it doesn’t uh and where it can get away with the lower reasoning settings. So that’s what I’ve I’ve done here. I won’t go into every single one, but I’d recommend that people that want to use the model come here and try that out and then kind of repeat that sort of study for their own their own project. >> Um maybe maybe I’ll put Flo on the spot here, but Flo can you tell us what mirror is? What’s it about and how can people try

1:05:41 it? I’ll bring you on stage. >> Yeah, so mirror is a is our agentic coding harness that is developed by the team like Alex mentioned that earlier on. And yeah, so mirror is great to be paired with beam if you want to yeah, run task in your terminal, coding task in your terminal. So I’ll suggest you go to the reflection OSS org, you download mirror and you start building great stuff, great project with this

1:06:13 harness. And this is definitely a harness which is made to be working well with the the model. So as Beru said previously, so mirror is kind of uh uh yeah, we had harness that we trained internally is called Daedalus and mirror is kind of the child of this harness. So it’s really made to be in distribution with how beam was trained. So we would recommend to use mirror with beam, but we also support

1:06:45 other open source harnesses. Yeah, so let’s give it a try and find some great things with mirror. >> And >> [clears throat] >> maybe a follow-up, how do I get to mirror? So what’s the user journey? How can they try the model? Where should they get it from? What inference engine do you recommend? >> Yeah, so how can we fine tune if you want to? >> Yeah, so if you want to get like let’s say an endpoint uh uh to be able like to plug mirror uh

1:07:15 and start coding with it. So, we have actually uh partnered with uh uh inference engines. So, uh SG Lang, a VLLM, Token Speed. So, this is uh when you whenever you want to deploy uh Beam on your own stack. So, we work closely with those teams and actually also collaboration with Nvidia to make sure you can get the best performance of your hardware when you deploy uh um on your on your stack. So, it’s

1:07:46 optimized for uh H200, uh GB uh B300s, uh and other Nvidia GPU. But, also we have partnered with uh um Unsloth uh to um uh bring kind of uh uh mirror Beam to uh local hardware. So, uh the Unsloth uh team will like create quants uh with our model and this can be run with uh engines like llama.cpp. So, you can try

1:08:16 to get Beam uh into your local uh uh computer uh and plug it with mirror. Um also um we made sure that uh uh Beam is integrated uh across like different uh inference provider, also cloud provider. They have inference services. So, you can also if you if you don’t want to uh do the hassle of like deploying yourself on your own GPUs, you can directly um go to our uh uh blog. They actually they

1:08:47 are listed there and you’re going to have like find links on how you can directly hit an endpoint um um powered by those inference provider. >> Cool. Okay, yeah. Thanks for that. Uh well, that’s really So, we’ve learned about integrations. Let’s drop back into the blog post um and show you about Beam Build again. So, Beam Build, as we mentioned at the top, is a competition. And uh it’s a competition to win $10,000

1:09:20 in cash and $10,000 in Hugging Face Hub credits that you’ll be able to use to train models, to to store models, to do whatever you want. And so, that can really bootstrap your your own project. If you want to submit a project, you have to go to the Beam model page on the hub, which you’ll see in the description to this video. And post a a comment on the on that uh model with your with your project. And your projects can be a space, a data

1:09:51 set, or a model. So, you could build a demo that just used Beam. It could be in in Beam’s API. It could be running on your own inference engine. Uh and you could ship it in a in a space. It could be a a local project. Um and that’s okay, too. Uh and then or you could have a a model. So, you could fine-tune Beam. You could quantize it, um obliterate it, you could merge it. Anything that can go into a model. Uh in fact, maybe you you even come up with something a little bit different. And maybe you fine-tune another model with Beam. As an example

1:10:22 uh where I did that, I’ll show you in a sec. Uh and the third thing you could do would be to create a data set. So, you could create a synthetic data set that um uh was maybe to fine-tune another model. You could create an RL environment that was maybe to fine-tune Beam or another model. Or you could create a a benchmark data set that’s maybe to benchmark Beam. All of these things, really whatever you can imagine, try it out and um uh post it on that. You don’t need to post any description in the comment, just the link. And then put everything

1:10:53 about the project in the um the repo itself. So, everything on the readme and so that explains what it is, why it’s interesting, these kind of things. And then we’ll judge that and the winners will get this cash prize. And credit prize. Cool. So, let’s inspire you a little bit and show you some of the demos that that I worked on. >> Um maybe maybe before we jump there, uh folks are asking if they can if they want to fine-tune the model and they’re interested in that. So, maybe I’ll again put flow on this what is sitting across from me and I like bothering him. Flow,

1:11:24 how can people fine-tune this model? >> Yeah, sure. >> Yeah. If >> If you’re >> not satisfied with like current uh Beam capabilities, we’ve also worked with uh a lot of different open-source partners to make it possible to customize Beam um on your own on your own stack. So, we partners with Nvidia to have a recipe supported uh in Nvidia automodel uh powered by Mega from Bridge if you want

1:11:54 to run SFT uh on your uh data. Um we also partner with uh Ray Spark team that uh uh uh produce Mice, that creates Mice, this library, which is for RL. Uh Hugging Face as well uh has TRL, uh which is made it’s a post-training library where we have like enable uh RL recipes. We’ve also partnered with Anyscale uh to have recipes for Spark

1:12:25 RL. So, yeah, we definitely we made kind of a lot of contribution out there uh if you want to customize Beam on your own data. And uh I would advise to uh go to our developer website. We have a lot of cookbooks recipe, so you can get started with like uh customizing Beams to to your own needs. Cool. Okay. Um Right. So, let’s say you’ve you’ve got the model and you now want to um

1:12:56 to do something uh inspirational. You want you want to work with it. We’ve got examples embedded here. So, SFT with TRL um as Florent just mentioned. So, this is an example that we ran and that we set up with Laura. We fine-tuned it for a a multilingual reasoning problem. So, we get the model to reason in a specific language, which improves performance in that language. Uh this all runs in TRL. It works on eight This This run was on eight uh B300 GPUs. And uh you can see it converges.

1:13:28 We’ve also integrated the example into TRL, so you can try that. If you want to go for RL, this Miles example, as Florent mentioned, is is a great place to start. There’s uh This is a is a math reasoning problem. You can run this in full with full parameters, so it’ll take uh in this case uh 96 H200s, or you could run a a Laura fine-tune, so 32 H200s. So, this is a really invested in an invested project. Um but the example’s right here that you could use to start that off. That’s also

1:13:58 merged into Miles, so you can check out the examples there. And the reflection team ran this and and tested this run. Uh and you can see how it improved performance on AIME um here What was it? Uh 9%, um which is pretty cool. Uh they also ran it, as I said, with Laura, uh which also improved performance. So, you can kind of scale that down. Okay, those are the two fine-tuning examples. You don’t need to fine-tune to enter into the Build with Beam project.

1:14:28 Like I said, it can just be a demo. It can just be a data set. So, let’s show you the examples that that uh I set up to to inspire you. Um but you can’t steal them, cuz if you use the same thing, then it kind of doesn’t count. Um but maybe you can uh just start from them. So, the first one that I did, um which is something that I I kind of try often. Doesn’t always work with with other models, um, but this time it did, so I was happy to share it. And it’s “Hey Beam, build a native local macOS application.” This is quite difficult, um, because you you really want the

1:14:59 model to to stick to the specifics of macOS and build something that’s integrated. Um, in this case, I wanted it to build a, uh, speech-to-text application that I could use >> Open Jeff models fine-tuned for my, uh, >> like where my cursor was. Uh, I wanted it to build an open-source application that ran on device using a specific model from the hub. So, it had to interact with the Hugging Face MCP. It had to download the model and build, you know, this machine learning project, but

1:15:30 then it also had to use MLX, which is a specific library for for that’s optimized for Mac, so that it could get the best out of, uh, the Mac hardware. And then it had to create a taskbar, and that taskbar had to map to to keys because I wanted to press, uh, I think it was command-shift-space to get the the application to pop up. So, it was a real real-world application, uh, and it one-shotted it, which was pretty great. Uh, I shared the source code, I shared the trace, uh, from from the run of how it one-shotted it, uh, and this was in the PyHarness. So, this is the kind of

1:16:00 thing that you could share in a in a space, uh, and other people can try. And, uh, and that would be a a really cool submission. Um, the next one was, uh, was a bit of a like a a research project with Beam. So, I’ve been interested in how different harnesses use multi-agents recently. And, uh, it’s something I’ve been exploring with with different models and trying to see which models, um, which models use sub-agents in different ways and and how they use their sub-agents, whether they’re better

1:16:32 with, uh, a single agent using the harnesses sub-agent feature, or whether they would be better with with five independent agents. So, I did a little research project, and I and I got Beam to set up Beam inside a pie harness to do this research project for me. Uh and it created this this plot here and we can see that actually Beam in in some in some latencies performs pretty well with an async sub agents and others performs better with separate agents

1:17:03 but gets to the highest gets the highest performance with a with a single agent setup. I spoke to the reflection team and they haven’t um they haven’t optimized Beam at all for this pie setup. So the the research project isn’t really related to to Beam’s research or any of those these kind of things. It was really just a personal research project and I thought it was pretty cool how I got Beam to research Beam and to show me how it worked and these sorts of things. So if you have a research project that’s a bit like that and you want to use Beam as your as your you know research intern that’s a great thing that you can set

1:17:34 up. You can share it as a space or maybe even as a data set maybe even all of them. Okay and then the last kind of project that I wanted to do was to fine-tune a decision model for my specific code base. Decision models are there all the rage at the moment everyone’s everyone’s fine-tuning them and and and releasing them because they’re super fast and they’re really practical for a lot of use cases where you don’t need these you know LLM capabilities, right? So I thought okay could I get Beam to help me with my decision model project.

1:18:05 And it it it could it worked. So um So it uh fine-tuned a decision model using Unsloth and it first of all created a synthetic data set of decisions based on on a code base and I evaluated it based on the accuracy on the synthetic data set and then I measured the latency and I found that okay it’s way faster which is what you would expect but it’s also way more accurate. Um

1:18:36 It’s way more accurate and so uh yeah, I could use this decision model uh in my project or in my application. Uh and that is a kind of like a, you know, pocket ML project. I could share that model, that decision model on the hub, and I could submit that to to build with Beam, and then maybe I win the prize. Uh maybe I could even go further. I could take my decision model and build a little demo around it and submit a uh space. There’s it’s really quite um like any ML project, you can take uh quite a few angles from this. You know, you could focus on the data set, the demo, or the model.

1:19:07 Okay. Um that’s that’s all really. That’s everything that we had planned for this session. Uh like I said, come to all the resources that we shared. Come to this blog post. Uh put some time aside. Uh there’s a lot to it. Go check out Reflection’s blog post and uh their developer resources. Um I’ll hand over to Sanyam. Uh so he can say goodbye. I’ll bring Florent back on to say goodbye. Um I’ll take this off the screen. >> Um maybe I’ll I’ll ask Florent to help the participants. He’s He’s really led our open-source team, so he has a lot of

1:19:38 insights on this. This $20,000 in prizes. Uh Flo, what’s what’s the secret sauce to winning them? How should people think about demos? Can you say some secrets to like help them with ideas? >> Yeah. What I can tell is that Beam actually internally >> Sorry, you’re you’re muted. >> What I can tell is like Beam actually internally operate uh a vending machine and can like generate profit with that. So, maybe it’s something that you can try to explore with this. But yeah, we uh if you want

1:20:10 to build great demos, uh yeah, you can go uh on Hugging Face. You can pull up a space and plug Beam there, and maybe you can uh make it operate uh your your own uh vending machine with that. >> Cool. Okay, yeah, that’s it. I think Yeah, that’s really cool. We’re full of ideas. Um I can’t wait to see what everyone builds. Uh, and I guess we’ll now call it a day and uh, say goodbye to everyone. >> Yeah, look forward to your task. >> Thank you so much, Ben. It’s been an honor doing this with you.

1:20:41 And thank you so much, Flo. It’s been an honor going on this journey with you. So, I’m really excited for to see what the community does with this. >> Nice. Yeah, thanks for coming and thanks for inviting me into your office and letting me uh, experience the vibes of the uh, release moment. Super excited. Okay. See everyone. See you soon. Bye.