ByteDance's World Model Re-Shoots Movies, Sora Returns? & Krea 3 Is Here!

Author: Theoretically Media · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: ByteDance’s World Model Re-Shoots Movies, Sora Returns? & Krea 3 Is Here! · Channel: Theoretically Media · 16:28 · Watch on YouTube

Topics: AI content generation, AI models, Multimodal models, Open source and software ecosystems

Summary

The video covers ByteDance’s Omni Human 2.0 avatar technology and an early demonstration of its unnamed world model, including camera re-tracking in scenes from a licensed film. It reviews Krea 3 as a fast image editor with some generation features still forthcoming, and introduces LTX 2.5 post-production tools for tasks such as matting, HDR processing, and upscaling. The host also discusses a report that OpenAI may provide Sora-like technology to a startup linked to former Sora research lead Bill Peebles, though a return does not appear imminent or aimed at general consumers. The video ends with a look at an open-source tool for generating and exploring fictional city maps.

Key points

Omni Human 2.0 adds camera movement, emotion, and improved lip-sync to avatar performances, but the host notes issues with teeth and some speech sounds.
Omni Human 2.0 is image-sourced rather than driven by video, and the host sees potential for using it to patch specific dialogue into AI films.
ByteDance’s unnamed world model demo shows mouse control, prompted objects that characters can interact with, and re-tracking camera movement in a scene from Stephen Chow’s King of Comedy.
Krea 3 is presented primarily as a fast image editor; features such as LoRA generation and style transfer were described as coming soon.
The host demonstrates Krea 3 edits for cleaning image grain, changing an outfit, and preparing a last frame for a generated video shot.
LTX 2.5’s post-production tools include alpha matting, 8-bit-to-16-bit HDR expansion, native HDR processing, and detail-focused upscaling.
The LTX tools are available through open weights, ComfyUI workflows, and LTX Desktop, which the host says runs locally without ComfyUI setup.
A report says OpenAI is in talks with a startup to provide access to Sora-like technology in exchange for equity; the host says it may target professional film users rather than consumers.
Sharp Maker’s open-source tool generates fictional city maps that users can explore and potentially download in 3D formats.

Chapters

Start	Chapter	Summary
00:00	ByteDance’s Omni Human 2.0	The host reviews the avatar system’s improved performance and cinematic camera movement, while noting limitations and possible use in AI film edits.
02:37	ByteDance’s world model	An early demo shows interactive world-building and camera re-tracking, with the host considering how the technology could connect video, 3D assets, and world models.
05:44	Krea 3 as an image editor	The host tests Krea 3’s image editing, discusses generation features that are still forthcoming, and shows edits for character images and video frames.
09:49	LTX 2.5 post-production tools	The sponsored segment introduces LTX tools for matting, HDR processing, clean plates, and upscaling, available through local workflows and LTX Desktop.
12:53	Sora’s possible return	The host discusses a report that OpenAI may license Sora-like technology to a startup, while noting that a release does not seem imminent or consumer-focused.
14:56	Exploring fictional city maps	The video closes with an open-source tool for generating and exploring cities, including the possibility of downloading 3D assets.

Quotes

I was really happy to see that it has mouse control — 03:08

Krea has clearly been drinking fake Carl Urban’s coffee — 05:44

I do think that this is going to find its way into my pocket as my primary image editor. — 09:49

This all runs on your machine. — 12:22

I would love to see Sora back in action. — 14:25

Concepts and entities

ByteDance, OmniHuman 2.0, Dreamina, world model, Krea 3, LTX 2.5, Alpha Gen, LTX, Sora, OpenAI, Sharp Maker

Full transcript (original language)

00:00 Today we’ve got some cool stuff coming out of ByteDance, including a look at their upcoming world model and the return of Omni human. Plus Krea is back and actually super quickly on top of it with the launch of Krea 3. All that and the return of Sora. Yeah, I never thought I’d say that. Kicking off yeah, some interesting news coming out of ByteDance. Well, first off we have the return of Omni human. This is the 2.0 version of their like avatar lip sync technology. But now with more camera movement, emotion, and better lip

00:31 syncing than like your standard avatar. Let’s take a quick look. >> I’m still trying to wrap my head around this. This is absolutely unbelievable. This means the world to me. Thank you all so much. >> Hey, you’re welcome fake AI actress lady. You earned it. I don’t think it looks bad and we’ve certainly come a long way since like the early days of avatar technology. You know, that kind of like bobblehead like hand motion thing that it used to I mean, you remember what I’m talking about. But you know, I mean obviously we still do have some problems with the teeth and you

01:02 know, while I don’t read lips, it does kind of feel like it’s not hitting all of like the diphthongs. As a quick side note, diphthongs would be an incredible album name for my fake punk rock band, The Gossian Splats. Now in terms of actual performances, I mean I do have to note that these are ByteDance’s examples and one thing that I’ve just noticed about ByteDance examples is that they tend to not ever look as good as when you know, we get a hold of them. But I mean, I do have to say in particular this one does feel a bit over the top. >> Target is not here.

01:34 Move to the next intersection immediately. Block all exits. He’s still inside this perimeter. >> I mean, fake Karl Urban really needs to chill out on the coffee. You know, that said, some interesting notes here including the fact that it’s actually for what is essentially an avatar, this does clearly showcase that Omni human 2.0 is you know, clearly able to do cinematic camera movements. And while yes, it does feel a little south of C-dances 2 and 2.5, uh and I understand that you might not be overly wowed by this, but the the thing about these

02:05 types of products is that they do come in very handy uh for patchwork in your AI films. When you need that like specific line of dialogue. So, despite the fact that Omni Human 2.0 does seem to have some limitations, like you can’t drive video with it. Uh it’s only image sourced, uh but that said, I mean it won’t change the dialogue like C-dances often will. So, I do think that this is one to keep an eye on. I’ll certainly take it out for a test drive once it becomes available. Currently for me, it is region locked, so I haven’t gotten the chance to test it out. That said, I did check in with the Dreamina team, and

02:37 I mean eventually it will be on that platform, so you know, we will all have access to it at some point. Now, in kind of more exciting ByteDance news, we did get our first look at the world model that they’ve been building. So, yeah, I mean essentially we are looking at like bootleg footage here, but at the Wonderful Tonight event earlier this week in Beijing, ByteDance showcased their uh their world model, which currently is unnamed, but I mean it does look really impressive. So, initially it does kind of look like the standard world model thing. Uh actually one thing

03:08 to note here is that uh I was really happy to see that it has mouse control, so it’s not you know, all of the other uh world models are all like WASD for moving around, and you have to up down left right uh with your keyboard controls. This one actually looks like it has mouse control, so thank you for that. We can of course prompt objects into the world and then have characters within that world react to that object uh as we see here with this spoon popping to existence, then our character climbing up it. But the part that got me doing like the like the Leo like pointing at the screen meme, um was camera re-tracking within a video.

03:40 Showcased was a scene from Stephen Chow’s King of Comedy, which actually shouldn’t be a surprise considering that Chow recently signed a licensing deal with ByteDance uh you know, for AI generated video. And as we can see in this demo from the film, uh we can now take like environmental details and performances of that original shot and re-track them. Now, it’s kind of hard to tell from the footage again, it’s like bootleg footage, but it does look like that UI in the corner there does seem to like have some like 3D controls to it. And then on the

04:10 second shot, I don’t know I’m still again not entirely sure, but it looks like it might be a Gaussian splat that’s in there. Again, really hard to tell. Some more interesting UI stuff in this kind of like portal effect demo. It’s a very nice shot, but what struck me is when it cut out wider kind of again getting another peek at the UI here. You know, we have these three environment planes basically and what I presume this line is is the camera track path through. So, yeah, it’s I mean it’s a really cool way of looking at it. I don’t know if this is going to obviously be you know, the the

04:41 final design for the UI. It’s it’s probably something that they’re working on, but yeah, I mean interesting and I think it does I think it’s this is easy to wrap your brain around. So, all of this looks really interesting and definitely opens the door to a number of use cases outside of like the current world model template of like walk around an environment for 60 seconds as things decohere around you, but more than that we’re definitely starting to see a lot of the modalities come together. Everything from like you know, video generation in C dance to like 3D asset generation to world models. So, yeah, I

05:12 expect 2027 to to really be the year that brings it all together. Interestingly, all of this might have something to do with ByteDance’s upcoming Pico Space Pro VR headset. This is kind of like their version of the Apple Vision Pro. Not yet released though it is like rumored to be significantly cheaper than say well like say the Apple Vision Pro. So, I will definitely be keeping an eye on all of this and obviously let you know as it develops. Moving back to home turf, Krea has clearly been drinking fake Carl Urban’s coffee and like grinding away

05:44 because they have released Krea 3. That’s just 5 months after Korea 2. Now, importantly, I think Korea is actually focusing Korea 3 to be more of an image editor than it is an image generator. There definitely does appear to be plans for generation coming down. We’ll take a look at that in a little bit. But for the most part, they are, you know, hyping this is like the fastest at 2K edits. And you know, it’s actually pretty good. So, again, if you’re looking at it as just a straight image generator, I mean, whatever. It It gets the job done. You know, as we see here

06:14 are four images of our man in the blue business suit walking down a busy city street. What’s kind of funny is that even Korea kind of seems to know that, considering that if you try to generate an image in Korea 3, it’s like, “Are you sure you want to do that?” And I think that is because a lot of the bells and whistles of like say Korea 2 aren’t here yet. So, like if you try to generate with a Laura, that is coming soon. There’s a waitlist that you can join up there. And then if you want to do style transfers, which is actually a very strong part of Korea 2, it’s once again coming soon. So, again,

06:44 as a generator, maybe not so much. As an editor, I do think it does some very nice stuff. For example, we we all know that grain pattern in GPT images. So, handing that image over to Korea 3, and just giving it the prompt to like remove the grain and improve the skin texture, we end up with this, which is significantly better. It’s cleaned up. For something that’s a little more me specific, and well, I mean, specific to fans of flamethrower girl. So, one issue that I do keep running into from her original outfit, kind of like the Mad Max apocalypse thing, is that anytime

07:16 that I try to do like any kind of image editing on it, a lot of times between like Nano Banana and GPT, like I I end up getting pushed back for inappropriate content. As a quick side rant on that, look, I get it. Yeah, she’s, you know, fairly cheesecakey in this, but I’ve always considered her like pretty much a superhero, and like to be honest, there are iterations of Wonder Woman that are wearing less clothes than she is. Thankfully, K3 is like, “You want an outfit change?” Sure. Let’s Let’s do an outfit change. I mean, I know I know Claude doesn’t do images, but if I tried to do this in Claude, I’m 90% sure I’d

07:46 be getting a lecture. So, yes, it is nice to have an image at model that will actually do the job. Uh otherwise, we’d basically be stuck with this as her new outfit. So, the way that this works is you have this precise edit mode where you can basically paint boxes on what you want to change. Uh in this case, let’s go uh change her to business attire. And there you go, business shark flamethrower girl. If you ask her for those TPS reports, she’s going to burn down the office. Now, where I think that this comes in really handy is uh you know, if you’re using any first frame last frame stuff. Uh for example, taking our FBI agent drinking coffee in a

08:16 Pacific Northwest diner, and then we generate an edit of the waitress delivering the pie and use that as a last frame. Well, we end up with this. This is a damn fine cup of Joe. But how’s the pie? >> It’s really good. It’s an old recipe from the log lady who got it from the owls. >> So, yeah, overall, a pretty solid

08:47 result, and it actually solves a few problems that we’ve run into a number of times in the past. Like, sometimes this water or cup would disappear, and for sure, we’ve definitely seen some insanely sized slices of pie. So, for sure, something to play around with. I think this offers you a lot more control. Now, that said, if you’re trying to take multiple elements and kind of combine them together, especially for cinematic work, um you know, I I don’t think this is necessarily the model, at least right now. I mean, it gets the idea and it gets the job done. You know, that said, you know, GPT image 2 ends up giving us

09:19 this, which, you know, despite the grain, does I mean, it does look a lot more dynamic. Now, that said, when you take all of those elements and bring them into an Omni model, it kind of like yeah, I don’t know, like like soups them all together anyhow. So, uh for example, a Planet Hell work in progress, >> The other thing I should notice that yes, it is very fast. Um so yeah,

09:49 overall I would say again, not going to be your primary image generator, but I do think that this is going to find its way into my pocket as my primary image editor. Moving on, uh Sora is coming back. I know that’s something that a lot of you have been hoping for. Like I to be honest, I never expected it to happen. Uh but first a quick word and you’re definitely going to want to check this out because uh our friends at LTX have some new open-source tools for us. So, we have definitely come a long way and AI video and images, I mean, they look great now. I mean, really great. But, one thing that we Well, we don’t

10:20 talk about too often is that they’re always in lossy formats. They’re like JPEGs and MP4s. They’re generally 8-bit and baked in. And those are formats that don’t really work in a professional context. So, yes, you can post it, but I mean, you can’t really grade it, key it, or blow it up. Well, good news on that front. LTX 2.5, the open weight model that, you know, you can run locally, just shipped post-production tools. As a quick FYI, LTX was kind enough to sponsor today’s video. And hey, look,

10:51 this one is relevant to your interests. Taking a quick tour, we have Alpha Gen. Uh to note, this one is in beta. So, yeah, I mean, beta, but you hand it a shot, be it AI generated or not, uh and it hands you back the matte. Frame accurate, like hair, smoke, fire, motion blur, glass, it all comes through, soft edges and all. it’s kind of a big deal. Some of you might remember the old days of rotoing in After Effects. So, yeah, this is a bit of a dream. On the color side, there’s actually two updates here.

11:22 Uh SER to HDR takes your 8-bit footage and expands it to 16-bit EXR, uh scene linear. So, you’ll get real color range in the shadows and highlights and less color drift and fewer artifacts. And then there’s native HDR for footage that’s well, it’s already HDR. So, 16-bit in, 16-bit out. And basically the edit here ends up happening without flattening it. So, this is great for like day-to-night in-painting or clean plates. As for blowing it up, I mean normally you’d upscale and watch it go

11:52 soft. LTX actually does the opposite. So, native resolution processes the footage at the size you’re actually delivering. So, 4K or 8K. Then refine rebuilds with fine detail on top. So, you end up with a bigger frame that’s actually sharper, not blurrier. Now, in terms of where this all lives, well, there are a couple of different places. The open weights of course are on hugging face. There are comfy UI workflows. And I mean, if you if you’re a node hater, which I know I know you’re out there, the LTX desktop, which just

12:22 got a big update that bundles Alpha Gen, Restore, Clean Plate, and Layout to Render. And that’s all with no comfy setup at all. Like not even the API. This all runs on your machine. The weights, the workflows, and LTX desktop are all linked down below. And hey, tag LTX if you make something. I can tell you for sure they do look at them. My thanks as always to LTX for sponsoring today’s video. So, Sora, which I don’t think needs any introduction. Interestingly actually the site itself actually still exists. You can still

12:53 download like all of your assets here. And then like you can see all of the icons. It just doesn’t do anything anymore. So, yeah, pretty sad. So, yeah, Sora, which had its plug pulled earlier this year when it was decided that OpenAI was done with SideQuest and instead moving on to much more serious projects like dots. I’m kidding. I love dots. I love my dot. Dot is great. Well, in the wake of all of that, Bill Peebles, who was the Sora research lead, ended up leaving OpenAI. You might not have known about that because he didn’t

13:23 post a manifesto declaring that this was Armageddon. Boy, I’m a little saucy today. Well, interestingly, according to a report from The Information, it does look like OpenAI is in talks with an unnamed stealth startup to give it access to the Sora-like technology, the the weights in the model. And in exchange for that, OpenAI would take equity stake. Now, what’s interesting is that this stealth startup, which I guess apparently isn’t so stealthy, because we do know that Bill Peebles is part of this stealth startup. Additionally, Jeffrey

13:54 Katzenberg, the former head of DreamWorks Animation, is also a co-founder. Also in there is Sergey Yashin, who is the former Dropbox CFO. So, I like I don’t know again, I don’t know how stealthy I think you’re being about as stealthy as the guy that got caught putting microphones in the OpenAI conference room. Which is to say, not very stealthy. By the way, do you know if this guy actually used ChatGPT to help him build the device to spy on OpenAI? Look, I am not an international spy, but I like apparently this guy isn’t either. Anyhow, back to Sora. It

14:25 doesn’t look like this is like set for eminent release or anything like that, and it does look like this is probably is not going to be consumer-facing this time around. The information also reports that this probably is going to be more of a narrower professional film industry, you know, type audience as opposed to you know, general wide release. So, yeah, I mean we’ll see how it plays out. So, I will definitely keep an eye on this. I I would love to see Sora back in action. Rounding out with just like kind of a cool little open-source thing that I discovered from Sharp Maker. So, what

14:56 this does is it it’s a it generates fictional city maps and then builds cities, and then you can like wander around in them. I mean, overall, I just think that this one is really neat. It is all in Japanese, so if your browser supports translation, if you so I mean, I I’m terrible at reading kanji, hiragana, and katakana. But, you know, you can kind of move around, zoom in with your mouse, or if you’re on mobile, you can pinch in. There are some other, you know, choices in terms of like how you want to render your city. And there are actually a

15:26 number of other, um, cities that basically you can randomly generate as well. Um, so yeah, I mean it’s it’s pretty cool. I mean, and you can get down to street level with this as well. So, um, let’s actually go back into the color for this one. Um, so this might like create some interesting opportunities, uh, depending on, you know, use cases, um, for, you know, essentially generating a city and then populating it out for references. The other interesting and cool part about this, too, is that you can actually down I haven’t tried this yet, um, but you

15:57 can actually download, um, uh, GLBs on this, uh, FBXs and, uh, OBJs. So, you potentially, uh, I don’t know how big it is, but like you can, you know, generate probably a section out and then import it into Blender and then do all of your, you know, Claude, uh, GPT, um, you know, Blender MCP stuff with it as well. So, definitely worth a bookmark and if anything else, just like worth like 5 minutes of your time just like just wandering around and zooming in and zooming out on a city cuz that that’s that’s what I that’s what I’ve

16:28 been doing for the last 5 minutes. I guess that’s it for today. As always, I thank you for watching. My name is Tim.