Microsoft's TRELLIS.2 Is INSANE: One Photo to a Full 3D Model (I Ran It)

Author: Prompt Engineer 48 · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Microsoft’s TRELLIS.2 Is INSANE: One Photo to a Full 3D Model (I Ran It) · Channel: Prompt Engineer 48 · 09:23 · Watch on YouTube

Topics: AI content generation, AI infrastructure and hardware, Local and open-source models, Multimodal models

Summary

Microsoft’s TRELLIS.2 is an open-source image-to-3D model that generates 3D assets with separate PBR material layers. The video explains its sparse-voxel approach, reports the creator’s generation times and memory use on a rented L40S GPU, and walks through setup challenges. Results on five example images were impressive, though the meshes need cleanup for animation-ready use and the project has installation and production-readiness caveats. The creator recommends it for prototyping and rough game props, but suggests waiting for a cleaner release before building a production pipeline around it.

Key points

TRELLIS.2 is described as a 4-billion-parameter, open-source image-to-3D model that generates assets with PBR materials.
Rather than representing shapes as a continuous field, it uses sparse voxels to represent geometry and materials, which can support open surfaces and enclosed internal structures.
The creator’s L40S runs took about 70 seconds at 1024 resolution and 149 seconds at 1536; these are single-run ballpark results.
At 1536, peak memory use was 15.3 GB, below the readme’s stated 24 GB requirement in the creator’s run.
At 1024, raw meshes had 11–16 million faces before export remeshing and decimation; the exported GLBs were around 42 MB.
The setup requires Linux, substantial GPU memory, dependencies and version pinning, and access to gated model files.
The creator recommends a GPU with at least 24 GB of memory and notes that a 48 GB card provides more headroom for loading and exporting.
Exports are opaque by default, and the resulting scan-style meshes may need retopology for animation-ready use.
The creator found the results impressive for a research release, but advises waiting for releases and a cleaner install for production use.

Chapters

Start	Chapter	Summary
00:00	What TRELLIS.2 is and how it works	The video introduces the image-to-3D model, its PBR output, project status, and sparse-voxel representation.
02:04	Compression, speed, and output quality	The creator explains the model’s compression approach and compares reported H100 speeds with L40S timings, memory use, and mesh sizes.
05:16	Running the model and handling setup	The video outlines the example workflow, cascade options, hardware requirements, dependency issues, and installation steps that worked.
08:22	Limitations and final verdict	The creator notes export and evaluation caveats, identifies suitable use cases, and weighs the results against production-readiness concerns.

Quotes

real metal, real roughness, and a real backside it never saw. — 00:00

they turn a giant sparse object into a short list of tokens — 02:37

I rented an L40S with 48 gigabytes for about a dollar nine an hour. — 03:08

Every one of mine was a version or an access problem, not a broken model. — 08:22

The output is genuinely impressive, and it fits on a mid-range cloud GPU, not just a data center monster. — 09:23

Concepts and entities

TRELLIS.2, Microsoft, voxel, PBR, sparse VAE, Hugging Face, GLB, RunPod

Full transcript (original language)

00:00 This is one picture of a steampunk engine and this is the 3D model Microsoft’s trellis.2 made from it on a GPU I rented with real metal, real roughness, and a real backside it never saw. So, how does it actually do that? How long did it take on my run? And can you run it yourself? Let’s go through it. >> [music] >> First, what is it? trellis.2 is an open-source image to 3D model from Microsoft. The page calls it a 4 billion parameter model and it makes assets with

00:31 full PBR materials at resolutions from 512 up to 1536 cubed. PBR just means the model comes with base color, roughness, metallic, and opacity as separate layers. So, a game engine or a renderer can light it properly instead of getting a baked-in picture. The authors are from Tsinghua Microsoft Research, USTC, and Microsoft AI. And the paper is on archive number 2512.14692 dated December 16th, 2025. Now, the repo itself, because hype and reality are

01:02 different things about 12.3 thousand stars, 1.5 thousand forks, MIT license for the code and the model. But, look at the details. 11 commits, no releases published, and two contributors listed. One of them is co-pilot. So, this is a research release and the page even says academic use only. Keep that in mind when you think about putting it in production. But, people are clearly using it. On Hugging Face, the model shows about 2.25 million downloads last month and 100 spaces built on top of it.

01:32 That’s the Hugging Face page captured today, not my number. Here’s the actual idea in normal words. Most 3D generators describe a shape as a field, a smooth function where the surface is wherever the value crosses zero. That works great for closed blobs. It struggles with open surfaces like a leaf or a cloth with thin walls and with hollow insides because a field has to decide inside versus outside everywhere. Trellis.2 skips the field. It stores only the cells of a 3D grid that the object

02:04 actually touches and in each cell it keeps the geometry and the material together. They call it a voxel. Because there’s no inside versus outside rule, the page says it can handle open surfaces, non-manifold geometry, and enclosed internal structures. And converting between a normal textured mesh and this format is fast under 10 seconds on a single CPU one way and under 100 milliseconds on coda coming back. That’s the author’s number. I did not time that part. A grid like that is huge. So, the second trick is compression. A sparse VAE squeezes it 16

02:37 times in each direction. At 1024 cubed that leaves about 9.6 thousand latent tokens, which is small enough that the generator can be a plain diffusion transformer, no exotic architecture. Plain English, they turn a giant sparse object into a short list of tokens, let a normal transformer write that list from your photo, then decode the list back into a full model. Now speed. The readme claims on an H100

03:08 about 3 seconds at 512, 17 seconds at 1024, and about a minute at 1536. I don’t have an H100. I rented an L40S with 48 gigabytes for about a dollar nine an hour. My first run at 512 took 67 seconds, but that includes warm up, so don’t treat it as a clean number. 1024 took about 70 seconds. 1536 took 149 seconds, roughly two and a half minutes. So, on this card it’s slower than the readme’s H100 numbers, which

03:38 makes sense. It’s a smaller and cheaper GPU, but I only ran each setting once, so call these ballpark. Now the quality is the real story. Look at the normal pass, the surface detail on the gears is sharp and the back of the engine, which the photo never showed, looks consistent with the front. Memory was a surprise. The readme says

04:09 you need 24 GB. My peak at 1536 was 15.3 and 1024 stayed between 7 and 12. One honest note on inputs, these are the example images that ship inside the repo, not photos I picked to flatter it. I didn’t test it on a hard case like a transparent bottle or a person, so I can’t speak to those. On three of those images and 1024, generation took 70, 74, and 85 seconds. Peak memory was 7 and 1/2, 7.6, and 12.1 GB. So, the cost isn’t one fixed number, it moves with

04:41 the object. The busiest object, the one with the most fine structure, took the longest and used the most memory. The raw meshes were big, too. At 1024, I got between 11 and 16 million faces before cleanup. The export step then remeshes and decimates down to 1 million faces with 4096 pixel textures, and that gave me GLB files around 42 MB each.

05:16 If you do have the hardware, the code is short, and the readme example is only five step. One, load the environment map. That’s just the lighting for the preview. Two, load the pipeline from the huggingface model trellis.2 4 billion and move it to the GPU. Three, open your image and call run, and you can pick the pipeline type 512, 1024, 1024 cascade, or 1536 cascade. Four, render a preview video. Five, export a GLB, where you choose the face budget and the texture

05:46 size. The cascade options matter. Instead of making the full resolution in one go, it makes a coarse structure first and refines it, which is why 1536 was affordable at all on a card like mine. There’s also a texturing mode where you give it a mesh you already have and it paints the PBR materials onto it from a picture. I didn’t test that one, so treat it as a claim, not a result.

06:16 Can you run it at home? My laptop has an RTX 4060 with 8 GB on Windows and the readme says Linux only and 24 GB, so no. That’s why I rented a pod. And even on a pod, it was not one command. Two of the model files are gated on Hugging Face. My token worked for the background remover, but not for Dino version 3, so I used a community mirror of those weights, which you should be careful with. Then four version clashes, torch audio, transformers, five, OpenCV, five, that could not read the lighting file, all fixable, but pin your versions. The

06:49 first model load alone took about 5 minutes and exporting a game-ready GLB took two to three minutes per asset on top of generation. So, the full cost per model is more than the generation number. So, here’s the path that worked for me in order, so you can skip my mistakes. Step one, get a GPU with at least 24 GB. Mine was 48 and I’d say that’s the comfortable choice because the load and the export both want headroom beyond the

07:20 generation peak. Step two, use a Suo Day 12.4 image with the development tools because the extensions compile on install. I used a PyTorch image on Ubuntu 22. Step three, clone the repo with the recursive flag because it pulls in submodules like Eigen for the voxel code. Step four, install torch 2.6 with CUDA 12.4 and then run the setup script with the flags for basics, flash attention, and NVIDIFR NVIDIFRAG, CUMESH, OVoxel, and FlexiGAM. That’s the

07:51 one that builds the custom kernel, so give it time. Step five, deal with the gated models. Request access on Hugging Face for Dino version three and for the background remover, and login with a token. If access is pending, you’ll hit the same 403 wall I did. Step six, pin your versions. For me, that meant a matching torch audio and older transformers that has the layer attribute the repo expects, and an older OpenCV, so the lighting file loads. And step seven, run the example script and watch the first model load. Be patient.

08:22 It’s slow once, and after that, it’s fast. If a step fails, read the actual error before you reinstall everything. Every one of mine was a version or an access problem, not a broken model. Things to know before you get excited. GLB exports are opaque by default, so transparency needs manual wiring. Linux is the tested platform. And the project page shows a quality chart, but names no

08:52 competing models, so I can’t tell you it beats anything. I can only tell you what it did on my five images. So, who is this for? If you make games or prototypes, and you need lots of rough but well-lit props fast, a couple of minutes per asset on a rented card is cheap. At about a dollar nine an hour, one asset took me roughly 4 minutes at 1024, and about six at 1536 generation plus export, which is around 7 to 10 cents per model once the model is loaded. That’s my arithmetic from this run, not a promise. If you need clean,

09:23 animation-ready topology, remember what you get is a 1 million face scan style mesh, not something an artist built. You will probably still retopologize it. My verdict for a research release, the output is genuinely impressive, and it fits on a mid-range cloud GPU, not just a data center monster. For a production pipeline, I’d wait for releases and a cleaner install. If you’re looking for cheap and reliable GPUs, I use RunPod and the link is in the description. If you have a photo you want me to run through it, drop it in the comments. Thanks for watching.