Step 5 Preview (Tested): This MODEL BLEW MY BRAINS AWAY!

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Step 5 Preview (Tested): This MODEL BLEW MY BRAINS AWAY! · Channel: AICodeKing · 11:16 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Software engineering with AI, AI models, Local and open-source models

Summary

AICodeKing tests Step 5 Preview, Stepfun’s flagship model, through eight coding and reasoning tasks in Open Code, using fresh GLM and MIMO runs as demonstrations alongside historical comparison scores. Step scores 67 out of 80 (83.75%), with its strongest results coming from a playable archery game and a local fine-tuning workflow that produces saved model weights and a working web app. The review also examines interactive simulations, 3D objects, an SVG illustration, a counting problem, and a wristwatch, noting both strengths and specific flaws. The host concludes that Step 5 Preview is worth trying for interactive prototypes and local development experiments.

Key points

The review uses Stepfun’s API for Step 5 Preview and runs all eight tasks in Open Code, with each task scored out of 10.
Step 5 Preview supports up to one million context tokens and 64,000 output tokens; the video says Stepfun describes it as a 600-billion-parameter mixture-of-experts model with 27 billion active per token.
The elevator simulation, contact lens case, folding table, and panda illustration each receive 8 out of 10, with specific issues noted in their behavior or presentation.
The archery game scores 9.5 out of 10: the host completes a round, hits all four targets, records a timed result, and verifies the replay loop.
The counting task receives 10 out of 10.
For the fine-tuning task, Step creates a panda-fact dataset, runs a LoRA training job, saves and fuses the adapter, and serves responses locally through a web app; it scores 9.5 out of 10.
The wristwatch task receives 6 out of 10.
Step’s total is 67/80, or 83.75%; the video cautions that comparison-chart scores for other models are historical references rather than a fresh controlled ranking.
The host recommends trying Step on projects whose results can be directly checked, such as a playable game or a local app.

Chapters

Start	Chapter	Summary
00:02	Model overview, setup, and scoring	The host introduces Step 5 Preview, its capabilities and pricing, and the eight-task Open Code evaluation setup.
02:42	Elevator simulation and interactive 3D projects	Step earns 8/10 for the elevator simulation, contact lens case, and folding table, with the host demonstrating their interactions and noting shortcomings.
04:50	SVG illustration and archery game	The panda SVG receives 8/10, while the playable archery game earns 9.5/10 for its targets, leaderboard, and replay behavior.
06:58	Counting, local fine-tuning, and wristwatch	Step gets full marks for counting, 9.5/10 for a locally trained panda-fact app despite data errors, and 6/10 for the wristwatch.
08:34	Final scores and verdict	The host presents Step’s 83.75% total, explains how the other models’ reference scores should be interpreted, and recommends trying Step for interactive prototypes and local development.

Quotes

This is one of my favorite results from step in the whole review. — 05:53

The correct answer is 20,460. — 06:58

STEPPEP got that chain working and we verified the local interface using the trained model. — 08:02

I think step five preview is well worth trying for interactive prototypes and local development experiments like these. — 10:44

Concepts and entities

Step 5 Preview, Stepfun, OpenCode, KingBench 3, GLM-5.3, GLM-5.3-Flash, MiMo-V2.6 Pro, MiMo-V2.6 Flash, Gemma 2B, MLX, LoRA

Full transcript (original language)

00:02 [music] Hi, welcome to another video. So, today we’re testing Step 5 preview, Stepfun’s new flagship model. And the results that got my attention were a playable archery game and a locally fine-tuned model with a working web interface. Those are two very different jobs, and Steep delivered both through open code. So, let me show you what it built and why I think this preview is worth paying attention to. Stepfun is sponsoring this video. I’m using Steppp’s actual API directly in open code and we’re putting it through all eight tasks in Kingbench 3. MIMO

00:34 v2.6 Pro, MIMO v2.6 flash, GLM 5.3 and GLM 5.3 Flash are our reference points. The main focus today is what you can get out of step and how its projects work when you actually use them. Now, step 5 preview supports a million tokens of context and up to 64,000 output tokens. It accepts text, images, and video. Today’s tests use text prompts for coding and reasoning inside open code. Stepfund describes the architecture as a mixture of experts model with 600 billion parameters in total and 27 billion active for each token.

01:06 Basically, the model activates a portion of its parameters for each token. That’s an interesting technical setup, but the part I want to spend time on is what it produces inside a coding agent. And open code gives us a useful way to see that. Step can work through the task, create files, run commands, and check its own output. Using the tools Open Code provides, we get actual projects to open and inspect. The game and the local training app are particularly good examples of why that workflow is useful. The published international API price is $1 per million uncashed input tokens, 5 for cached input, and $2.70 per million

01:39 output tokens. Reasoning tokens are included in output billing. That cached input rate is worth looking at if your workflow repeatedly sends eligible shared context, although your actual bill still depends on how much the agent generates and how many calls it makes. Stepfun also says the weights are planned for October 15th. The version we’re using today is the API preview, and I’ll be watching that planned release after seeing what it can already build here. So, let’s get into the setup. Step connects directly to Stepf funs API while the four reference models use Open Router. All five run inside

02:10 Open Code. Every task has its own folder and session with the original King Bench prompt and no other model solution or answer key. There are eight tasks, each worth 10 points for 80 points in total. I requested high reasoning effort and allowed up to 64,000 output tokens per generation. We check the delivered projects and use the updated results for the final scorecard. The GLM and MIMO figures on the comparison chart remain their historical Kingbench results. Their fresh runs give us reference demonstrations while STEPP receives its score from this review that keeps the earlier scores intact and makes it clear what the chart represents. The first task is the elevator simulation. We need

02:42 three elevators, each taking only one person with people waiting on different floors and traveling to randomly assigned destinations. There’s quite a bit of behavior to coordinate here because the simulation has to keep track of both the cars and the people waiting for them. So, let’s send it and check. Step gives us a working simulation. I spawned 12 people and the delivered count reached 12 with nobody left waiting or riding. The elevators carry one passenger at a time. People stay in the cues until they can board and the destination tool tips are there too. I like that you can follow the whole

03:14 process on screen. Spawn people, watch the cars collect them and see the cues drain as they reach their floors. Step has implemented the central behavior of the prompt and we can verify it by actually running the simulation. The overlapping floor controls and a reset during pickup bug costed points. So this gets 8 out of 10. Those are specific things to correct in a project that already demonstrates the requested passenger and elevator behavior. Now, let’s try the contact lens case. This asks for a 3JS model with left and right labels and caps that open when you click them, all in a single HTML file. Step

03:47 delivers an interactive 3D case. I can open either cap independently, close it again, and use the camera controls to inspect the model. That’s a useful result from a short prompt, especially when you want to experiment with how an object behaves on screen. Watch what happens as I open one side and then the other. Each cap responds separately and I can move the camera to inspect the interior. Having those interactions working gives us something much more useful to evaluate than a single still image. The geometry and default framing need another pass and the liquid and lens should stay in the wells instead of

04:17 moving up with the caps. I’m giving this 8 out of 10. The working cap controls and camera interaction are the parts I like here. The third task is a folding table. The prompt asks for a slider that unfolds the table as you move right and folds it as you move left with smooth 3D animation. step creates a recognizable table and the leg frames move smoothly underneath the top. I can scrub through the middle positions, stop partway and reverse the movement. So, we get a continuous animation that responds to the control, which is exactly the kind of interaction I wanted to explore with this test. It does reverse the requested

04:50 slider direction. The left side extends the legs and the right side tucks them underneath while the labels describe the opposite. That keeps it at 8 out of 10. The folding motion itself works and it’s worth showing those intermediate positions because that’s where you can really inspect the animation. Next is the panda eating a burger. This is an SVG task, so we’re asking step to create a vector illustration with a recognizable character and a clear relationship between the panda and its food. The result reads immediately as a panda holding a burger near its mouth. It is valid SVG, so we have an

05:23 illustration we can open and scale while inspecting the shapes. I think this is a nice example of the range in these tests. Step moves from coordinating a simulation to making a small visual asset. The burger layers and mouth placement need some composition cleanup. So, this gets 8 out of 10. The subject and action are clear, and the illustration gives us a usable starting point for further art direction. Now, let’s make a bow and arrow game. There should be four targets, and hitting them in less time should move the player higher on the leaderboard. This is one of my favorite results from step in the

05:53 whole review. You draw back from the bow, set the shot, and release the arrow. Step also adds wind and moving targets. So, there’s a bit more to the game than simply clicking four stationary objects. You have to adjust your aim and see how the shot behaves. I played a complete round in the browser, hit all four targets, and got a timed result on the leaderboard. Play again resets the targets, arrows, and timer correctly. So, the game carries us through aiming, hitting the targets, recording a result, and starting another round. That’s what makes this demo convincing to me. We can play the game

06:26 and inspect the behavior as it happens. The lower times sort above the slower ones. Misses don’t count as hits, and hitting the same target repeatedly doesn’t falsely finish the challenge. Step delivered a more complete game here than the fresh GLM 5.3 flash demo, which failed when saving its leaderboard result. That’s a comparison of these delivered demos. The historical chart scores stay unchanged. There’s a small edge case where a slow result that misses a full top 10 can display rank zero. With that deduction, step gets nine and a half out of 10. I like this result a lot. The playable round and

06:58 working replay loop make it one of the strongest demonstrations in this test. The sixth task is the counting problem. The correct answer is 20,460. On the updated scorecard, I’m giving step 10 out of 10 for this question. That adds the full 10 points to its total. The seventh task is more involved. Generate a data set of panda facts. fine-tune a Gemma 2B model and create a local web interface that produces a fact when I refresh the page. This is the other result that makes STEP especially interesting to me as a coding agent. Step generated the data set, ran an actual Laura training job, saved the

07:30 trained adapter, and fused it into a local model. It used Gemma 22B instruct through MLX. The web app then ran inference locally with those trained weights. The run needed one continuation after I corrected a cache setup issue in the test environment. After that, step completed the workflow. So, here’s the app. It generates a Panda response locally and refreshing the page produces another response. There’s also a next fact button which makes it easy to demonstrate. This task involves several pieces that have to fit together. The

08:02 data has to be prepared. The training has to run. The resulting weights have to be saved and the app has to load them correctly. STEPPEP got that chain working and we verified the local interface using the trained model. For this fresh demonstration, STEPP delivered a more complete local training workflow than either GLM run. The working app and saved weights are the reasons I would spend more time exploring STEP for this kind of project. The deduction is for factual errors in the generated training data, including a wrong claim about the first pandas in a US zoo. The data needs checking before you rely on the generated facts. I’m giving this nine and a half out of 10

08:34 for the delivered workflow with that data quality issue included in the assessment. The final task is a full 3D wristwatch with smooth hands, the current time and two time zones. I regenerated this one and step finished the watch. It was fine, but nothing extraordinary. So, I’m giving it 6 out of 10. So, this is the final chart. Step five, preview gets 67 points out of 80. That is 83.75% or an overall grade of 8.38 out of 10. The individual scores are eight for the elevator, eight for the lens case, eight

09:08 for the folding table, eight for the panda, 9 and 1/2 for archery, 10 for the counting problem, 9 and 1/2 for fine-tuning, and six for the wristwatch. Seven of the eight tasks now score at least eight out of 10, including two 9 and 1 halfs and a 10. The updated total is 83.75%. The game and the local training app remain my favorite demonstrations. And together with the simulation, 3D objects, and illustration, they show the range of projects STEP can handle. For reference, GLM 5.3 is at 91.25%

09:40 and GLM 5.3 Flash is at 78.75% on the retain chart. Myo V2.6 Pro’s original result is 69.38% and Myo V2.6 Flash’s original result is 7250%. Both myo models retain their earlier 10 out of 10 for fine-tuning. These are historical reference scores rather than a new controlled ranking from the fresh demonstrations. There’s also the separately labeled Myo flash math retest with a 131,072 token allowance. Using that math grade gives it 80% and it stays separate

10:13 because it used a different output budget. steps updated 83.75% puts it above the myo and GLM 5.3 flash reference figures on this chart with GLM 5.3 still ahead. Its strongest demonstrations make me want to put it to work on another interactive prototype. Personally, the archery game and the local fine-tuning app are the clips I’d spend the most time showing you. They give us complete interactions to try and the local training task leaves us with saved weights and an app that uses them. That’s substantial work for a coding agent to deliver. I think step five preview is well worth trying for

10:44 interactive prototypes and local development experiments like these. Start with a task where you can check the result directly as we did by playing the game and refreshing the local app. The two 9 and 1/2 results give me a concrete reason to explore those workflows further. So if you’re curious about step, I’ll leave its documentation and the direct API setup for open code in the description. You can use the same route, open a fresh project and see how it handles the kind of work you want to build. I’m particularly interested in what we can make next by extending the game or building on the local model app

11:16 from this review. Overall, it’s pretty cool. Anyway, let me know your thoughts [music] in the comments. If you like this video, consider donating through the super thanks option or becoming a member by clicking the join button. Also, [music] give this video a thumbs up and subscribe to my channel. I’ll see you in the next one. Until then, bye. >> [music] [music]