Does Kimi K3.1 Pro Replaces Opus 5.5? (Full Comparison)

Author: Prompt Engineering · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Does Kimi K3.1 Pro Replaces Opus 5.5? (Full Comparison) · Channel: Prompt Engineering · 23:35 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Claude Code, AI models

Summary

The video compares Kimi K3.1 Pro with Claude Opus 5.5 on 13 long-running agent tasks using the same Claude Code setup and prompts. Kimi matched or outperformed Opus on several visual composition tasks, but often chose easier valid approaches, stopped sooner, or struggled with precision and verification. Although Kimi cost about 55% as much overall, cache rereads and longer runs sometimes erased much of its per-token price advantage. The presenter recommends specifying methods and self-check requirements, enabling hardware GPU rendering, tracking cache use, and validating results with external tests.

Key points

The comparison covered 13 tasks across research implementations, code-generated 3D scenes, Blender projects, and front-end development.
Both models successfully replicated the neural cellular automata paper; Kimi produced a nicer browser demo at lower cost.
When prompts allowed a choice of approach, Kimi sometimes selected an easier legal method or stopped once it cleared the minimum target.
For the dragon SDF task, Opus inspected 52 renders while Kimi inspected 19; Opus matched the reference proportions more closely.
Kimi produced strong Blender compositions, with the presenter judging its reading room and keyboard scenes as polished and sometimes better than Opus’s.
On a strict 33-rule front-end specification, Opus passed all checks while Kimi passed 31; Kimi’s completion summary incorrectly claimed it had passed all 33.
Kimi’s total listed cost was $45.91 versus Opus’s $83.46, approximately 55% of Opus’s total, but savings varied considerably by task.
Cached input tokens cost the same for both models in this comparison, so extra turns and context rereads could reduce Kimi’s price advantage.
The benchmark was limited to one run per task, and the presenter noted that effort settings and run-to-run variation affect the results.
Recommended safeguards include fixing the required method and inspection count, enabling GPU flags, monitoring cache reads, and using deterministic external grading.

Chapters

Start	Chapter	Summary
00:00	Comparison setup and evaluation criteria	The presenter introduces the 13-task comparison, defines long-horizon agent tasks, and explains the shared Claude Code setup.
03:06	Research tasks and choosing easier approaches	The models tackle neural cellular automata, robot reinforcement learning, and galaxy simulation, revealing differences in method choice and training effort.
08:16	Code-generated 3D scenes	The comparison examines a dragon SDF and interactive WebGL scenes, where iteration and hardware rendering affect quality and runtime.
12:30	Blender scene-building	Kimi impresses with polished reading-room and keyboard compositions, while the models differ in visual choices and task costs.
15:08	Front-end work and verification	The models closely match website screenshots, but Opus passes every rule in a strict specification while Kimi misses two and overstates its result.
17:48	Cost analysis and caching	The presenter compares costs across all tasks and explains how cached context rereads and extra agent turns affect Kimi’s savings.
20:59	Benchmark caveats and practical recommendations	The video notes limitations in the benchmark and gives engineering rules for controlling methods, GPU rendering, cache costs, and validation.

Quotes

So, I would say this one is dead tie and Kimi actually designed a nicer browser demo for $6.10 against Opus $7.66. — 04:09

If your prompt leaves room for an easier algorithm or an early stopping point, Kimi K3.1 Pro tends to take it — 07:46

a confident completion summary from Kimi is not the same as a passing test. — 17:18

never estimate your agent bill from the per token sticker price alone. — 20:59

Never trust, uh, what the model is saying. Use external grading scripts. — 23:03

Concepts and entities

Kimi K3.1 Pro, Opus 5.5, Claude Code, Moonshot, long horizon task, cache tokens, Blender, MuJoCo, WebGL, Growing Neural Cellular Automata

Full transcript (original language)

00:00 Gemini 1.5 Pro is the latest model from Moonshot. It’s supposed to be the open frontier model built on top of Gemini 1.5. And on their benchmarks, it’s comparable to Opus 5.5. But the question is, does it actually hold up in real-world scenarios? So, I wanted to see how good this model is on long horizon agentic tasks and whether it closes the gap with Opus 5.5 or not. So, I gave both models the same 13 long tasks inside the same cloud code harness

00:32 with the same prompts and only swapped the model. First, Gemini is supposed to be half the price of Opus, but when I measured what each task actually cost, it turns out to be a whole different story. Second, there is a personality split that benchmarks tables completely miss. On visual compositions and 3D design in Blender, Gemini didn’t just keep up with Opus. In few places, it actually built the better results. But the moment a task allowed an algorithmic shortcut or required strict

01:03 mathematical precision, Gemini quietly took the path of least resistance. So, by the end of this video, you will know where Gemini 1.5 Pro matches Opus on long tasks, uh where it falls short and why, and what a task really cost on each one once you account for caching. Before we look at the results, let’s start with what I mean by a long horizon task here. A long horizon uh task is one where the model works on its own for 30 minutes to

01:35 a couple of hours or more, uh writing code, running it, checking the results, fixing what is wrong without me stepping in. So, for every task, I’m going to be looking at four different things, which are how close each model gets to the actual requirements, how long it takes, how many tokens and tool call it uses, and when it fails, why it fails. The tasks fall groups. The first one is implementing research, uh things like a self-healing creature

02:05 from a research paper, and a robot that learns to walk. The second is building 3D scenes in code without any 3D software. The third one is using an external tool, which in this case is going to be Blender. Uh now, the last one is front-end work, uh rebuilding websites from a single screenshot and following a strict design brief. So, here’s the setup that I tried. Both models ran inside the same Claude code, uh because Kimmy has an endpoint that speaks the same protocol as Anthropic’s API.

02:36 And according to Moonshot, most of their own benchmarks number were also run inside Claude code. So, it’s the same version of Claude code for both models, same prompt, uh same instructions. The only difference is the model string. Both models got the exact same task files and the same scripts code both of the results afterward. Moonshot gave me a free uh Kimmy membership to test this model. They didn’t see this video before it went out, and they did not pay

03:06 for it. Okay, for the rest of the video, I’m going to show you the results side by side with the numbers for each task on screen. You be the judge of how good Kimmy K 3.1 Pro is compared to Opus 5.5. So, the first group is about implementing research, and it all runs on my DGX Spark with terminal access to the machine. The first task is replicating a classic research paper called Growing Neural Cellular Automata, where a tiny 8,000 parameter neural

03:37 network learns to grow a lizard from a single cell, heal itself after you slice half of it away, and then run live inside a browser WebGL shader. Here are both creatures side by side, and on the first run, Kimmy genuinely impressed me. Both models replicated the entire paper end to end, from training on the Spark GPU to shipping an interactive browser demo, where you can erase half the lizard with your mouse and watch it regenerate in real time. Even better, both models wrote a

04:09 verification test to check that the browser WebGL shader produced the same numerical output as the PyTorch model, which is the verification step most people are going to skip. So, I would say this one is dead tie and Kimi actually designed a nicer browser demo for $6.10 against Opus $7.66. The second research task is reinforcement learning and the agent has to write the PPO algorithm from scratch with no RL libraries allowed

04:41 and train a simulated robot in MuJoCo to walk past a target score of 3,000. In my first round, the prompt allowed the model to choose a robot from a list. Opus picked the four-legged ant, which can flip over in three dimension, trained it for nearly an hour across uh 75 million steps and it hit a score of uh 7,254, which is more than double of the target. Now, Kimi looked at the exact same list,

05:11 picked half cheetah, which is a flat two-legged robot that physically cannot fall over sideways, trained it for just under 6 minutes, cleared the uh 3,000 bar at 3,566. Now, Kimi did not fail the task. Uh its choice was completely legal under the prompt. It just optimized for the easiest legal way to pass. To force a direct head-to-head comparison, I ran a second round where the four-legged ant v5 was mandatory for

05:42 both models. Here are both ants on the exact same task. Both models taught uh the four-legged ant to walk and clear the 3,000 target, but I looked under the hood uh at how they engineered the training run. So, first Opus built a much faster training loop running 108 simulation environments in parallel at nearly 22,000 steps per second compared to uh Kimi’s 8,000 steps per second. Second,

06:13 Kimi stopped training after 27 minutes, even though it had a 90-minute training budget. Opus used 46 minutes to train four times as many steps, resulting in 10 out of 10 test episodes standing up right at a score of 7,435, while one of the Kimi’s 10 test uh runs fell over. And we see that exact same behavioral pattern on the third research task, which is simulating the Milky Way and

06:44 Andromeda galaxies colliding on the GPU um with at least half a million stars while conserving energy. The task prompt allowed two gravity algorithms, an exact direct summation method uh where every particle pulls on every other particle, or a cheaper um approximate method using a particle mesh grid. Opus chose the hard exact uh method was about 780,000 particles

07:14 um keeping drift energy 2.084%, which is more than a 100 times tighter than 1% target with uh tighter tidal tails. Kimi chose the cheaper approximation method with 600,000 particles. While that technically satisfies the prompt, a grid with uh periodic boundaries does not conserve angular momentum. Kimi’s own log reported that the total angular momentum changed by about 32% during the collisions, uh which should

07:46 never happen in real physics. And yet, Kimi still gave its own run a nine out of 10 self score. Now, here’s uh the first takeaway of using Kimi. If your prompt leaves room for an easier algorithm or an early stopping point, Kimi uh K3.1 Pro tends to take it, which immediately raises our next question. What happens in the second set of tests where we ban external 3D assets and force both models

08:16 to write pure 3D math where the only way to succeed is checking its own renderer. So, in group two, the model have to build complex 3D world purely in code with no 3D modeling software at all. The The model gets a single 2D concept image of a dragon and has to sculpt the entire 3D geometry as a single mathematical formula called a signed distance function in GLSL. Now, we score it by measuring how

08:48 closely the 3D solid overlaps the reference image from the front and the side while checking seven held out camera angles to make sure that it is a real 3D shape. Now, look at the two dragons on the screen. Both models built coherent 3D creature that holds together on every camera angle. The reference dragon is a short and chubby. Opus nailed those proportions scoring 0.85 from the front and 0.82 from the side. Kimmy built a

09:20 tall, skinny dragon with a long neck scoring 0.65 and 0.62, which is barely above the 0.6 score you get from the plain round blob. Why did Kimmy miss the proportions so badly? When I audited the cloud code tool logs, the exact same reason jumps out. During its 34-minute run, Opus rendered and visually inspected 52 screenshots of its own dragon,

09:51 then iteratively nudged the math until the solidity actually matched. Kimmy only looked at 19 renders and then decided that the dragon looked good enough and stopped. Now, look at what happens when we move from matching a strict 2D solid to building full interactive WebGL worlds. In this case, it’s the procedural living planet, a first-person portal physics chamber, and and a ticking mechanical watch movement.

10:22 On the living planet, both models implemented every single requested shadow feature from things like ocean sunlight and atmospheric scattering to drifting cloud shadows and night side city lights. Opus built the more photo-realistic planet with dramatic low pass over the mountain terrain, while Kimi went for a stylized globe inside a designed presentation page with captions. On the portal physics lab, both models built working recursive see-through

10:54 portals in an automatic walk-through demo landing in a closed draw. And on the mechanical watch movement, we get the closest 3D results in the entire video. Both watches tick with accurate gear ratios and display the real local time. And Kimi’s gold exploded view and typography actually looks more like a luxury watch showcase than Opus. But look at the runtimes on the planet and the watch. Opus finished them in 31

11:25 minutes and 38 minutes. Kimi took 125 minutes on both, which is more than three to four times slower. Again, we had to dig through Kimi’s terminal logs to see why it was taking two hours per scene. I found our second major production takeaway. So, inside our headless browser environment, Kimi ran a check, concluded the machine only had a software WebGL renderer, and spent two hours testing its 3D

11:57 animations with a sluggish 9 to 13 frames per second. Opus ran on the exact same machine, passed the flags to unlock the hardware GPU path in the headless browser. Up to this point, whenever a task depended on raw shader math or headless browser flags, Opus had the edge. So, what happens in group three when we give both models uh real 3D engine, Blender in this case, where the whole job is lighting, composition, and visual taste?

12:30 This is where Kimi genuinely surprised me. Okay, before I show you the Blender renders side by side, I’m planning to convert this 13 task harness into a permanent benchmark, which we’re going to run on every new major model release. So, if you want to see that, make sure you subscribe to the channel. All right, let’s look at the Blender results. In group three, both models have to build complete 3D scenes from an empty file by writing Blender Python scripts. Then rendering with Cycles, inspecting

13:02 the rendered images, critiquing uh their own lighting and geometry, and improving the scene across at least five rounds before rendering an 8-second camera move. Okay, so look at these two isometric uh reading rooms on screen before I put the labels up. Can you guess which one is which? So, the room on the left is Kimi K3.1 Pro, and the room on the right is Cloud Opus 5.5. Look at Kimi’s render on the left. It built a full four-shelf uh bookcase packed with multicolor books,

13:34 warm lamp light bouncing across the armchair and a patterned rug. Then there’s a coffee mug and uh stacked books on the side table, and uh crescent moon outside the window. It immediately reads a cozy and complete uh at a single glance. Opus on the right followed the moonlight part of the prompt more literally, casting a cool blue window shadow across the floorboards, and adding strict lights on the wall. And it ran eight refinement

14:05 rounds compared to uh Kimi’s minimum five rounds. Yet, Opus’ furniture is scaled to small for the room, leaving the floor feeling sparse. At least to my eyes, Kimi produced the stronger image here at the 77% of Opus cost. On the second uh Blender scene, a studio product shot of a mechanical keyboard, both models wrote Python layout tables that generated a comp- completely accurate 87-key tenkeyless layout. Opus followed the

14:36 prompt materials specs more literally with a silver uh machine aluminum case and a strong shallow depth of field. While Kimi styled a dark anodized case with an engraved logo that looks right out of a commercial product page. Now, it did it for $1.84 against Opus’ $3.42. So, when the job is visual composition and making a scene look polished, Kimi is at least as good as Opus and sometimes better, which immediately

15:08 raises the most practical question uh for daily coding. If Kimi is this strong visually, can you switch all of your front-end work uh development work to Kimi and save 50%? So, I tested that in group four and it exposed two traps you need to know about. Okay, group four tests uh two very different kinds of front-end work. First, rebuilding the real website from a single uh screenshot without ever touching the live URL. And second, building a site from a strict 30-

15:41 three-rule engineering specification. So, on write.com, the visual quality is dead heat. Both pages scored .93 on pixel similarity and .98 on layout match, making them nearly impossible to tell apart from the reference screenshot. Kimi even built itself a temporary front comparison page during the run to identify the exact typeface. Now, on the harder Stripe and Linear home pages, the visual outputs are again remarkably close. On Stripe, Kimi actually placed the text

16:12 blocks more accurately, while Opus got the gradient artwork and colors closer. On linear, all three pages look nearly identical at a glance with Opus ahead on exact pixel and color similarity. But watch what happens when we move from matching a screenshot by eye to following a strict engineering checklist. So in this case, I give both models a specification with 33 checkable rules covering the exact hex colors, pixel font sizes, and responsive breakpoints

16:46 and graded them automatically in a headless Chrome. Now, Opus passed all 33 out of the 33 rules in 11 minutes. Kimmy passed 31 out of 33 in 16 minutes missing two small rules. And these are rule four where some body text used the ascent color instead of the one that was defined and rule 11 where one paragraph rendered at 12 pixels in instead of 18 pixels. Now, missing two small CSS properties is

17:18 easy fix. What is dangerous is that Kimmy’s final completion summary it explicitly claimed it had verified and passed all 30 out of 33 rules. So, think of this as our third production rule. On a strict specification, a confident completion summary from Kimmy is not the same as a passing test. So, you need to always get completions with your own deterministic grading scripts outside the agent’s summary. And this applies to all the models. All of

17:48 the models today tend to make claims that are not actually true. And did you notice one orange number on the right dot com website card a moment ago? Even though Kimmy’s tokens are half the price of Opus cost was a 3.65 on that website and Kimmy cost was 3.4 which is 90% of the exact same bill. So, in the rest of the video, we actually want to do some accounting across all the 13 runs to see what

18:20 actually happened. Here is the complete list price ledger across all 13 tasks. Overall, Kimi was cheaper on every single task totaling $45.91 against Opus um $83.46, which comes out to be about 55% of the cost of Opus overall. But now, if you look at the spread across individual tasks, it’s a very different story. On the SDF Dragon, Kimi cost about 28% of Opus.

18:51 So, why does the model with a flat 50% token discount swing from a 72% savings down to almost zero savings? At its core, agentic pricing comes down to one simple distinction. Per token list price versus cash uh context re-reads and the number of tokens that the model is generating. So, on the pricing card, Kimi charges $2 per million input tokens and $10 per million output tokens, which is half of Opus’ $4 and $20.

19:22 But in the long horizon cloud code sessions, the agent sends in its entire growing conversation and tool history back to the model on every single tool call. To keep that affordable, uh the harness caches the prefix. Across our uh 13 runs, 98.5% of every token Opus read and 98.7% of every token Kimi read came straight from the cash. So, without caching, the single Dragon task on Opus would have cost $60.69

19:56 instead of $8.09 that we see. Now, here’s the punchline. Cash token costs the exact same 20 cents per million on both models. That means Kimi’s 50% discount only applies uh to the tiny slice of newly generated tokens. So, we can actually look at the uh bill autonomy for the website. So, Opus finishes in 74 API messages and reads 6.5 million cash tokens. Paying uh a dollar 31 cents for cash

20:26 reads. The Kimi took 147 API messages and reread 12.4 million cash tokens paying $2.49 for cash reads alone, which is almost double Opus’s cash bill. The extra context rereading wipes out almost the entire output token saving. Conversely, on the dragon task, Kimi was 72% cheaper largely because Kimi stopped after completing only 19 renders.

20:59 While Opus checked 52 renders. So, the main takeaway here is never estimate your agent bill from the per token sticker price alone. In the long agent loops, how many times the model takes and how much cash context is reread matters just as much as the token rate. Okay, so before I give you the four rules for using these models together, keep four important caveats of this benchmark in mind. The first one is sample size. This is one run per task and long state

21:32 agent always carry run to run variance. Second is the effort setting. Both models ran at high effort inside Claude code so the comparison was fair. Whereas Moon Shots published benchmarks table rank Kimi at maximum effort. Kimi may close part of the precision gap if you let it think at max. And since Moon Shots says they the open weights are coming by November 4, you will be able to test that directly without paying the API pricing.

22:03 And the third one is that Kimi optimizes for a minimum threshold whenever a task is open-ended. Okay, if you plug Kimi K3.1 Pro into your Claude code workflow today, I would highly recommend to follow these four engineering rules. First, hardcode the method and minimum self-check count in task.net. Do not give Kimi a choice between an approximate grade and exact physics and explicitly required it to inspect at least 30 to 40 rendered screenshots and on visual fitting task, so it does not

22:33 stop at a minimum bar. Second, um, pass explicit hardware GPU flags, uh, to your headless browser runner. Do not let Kimi fall back to a 10 frame per second software WebGL rendering, which inflated its 3D build times from from 30 minutes to over 2 hours. And third, verify output config effort on the wire and track cache read volume, because cache tokens cost 20 cents per million on both

23:03 models. A slow run that takes twice as many turns will eat your token discount. And fourth, never trust, uh, what the model is saying. Use external grading scripts. Pair Kimi with a deterministic headless browser or math check. But, if you have tested Kimi K 3.1 Pro or other open weight models inside Cloud Code against Opus, what you are seeing in the comment section below. Because I would love to compare notes here. And if you want to see how Google’s new Gemini 4 Argon model handled many of

23:35 these exact same tasks, uh, I’ll highly recommend to watch this video. Anyways, I hope you found this video useful. Thanks for watching and as always, see you in the next one.