Introducing Continuum: A Real-Time Video Operating System | Runway Labs

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Introducing Continuum: A Real-Time Video Operating System | Runway Labs · Channel: Runway · 05:09 · Watch on YouTube

Topics: AI agents, Software engineering with AI, AI developer tools, AI content generation

Summary

Runway Labs introduces Continuum, an early research application exploring how real-time video models could be integrated into an everyday operating system. Users can create persistent, interactive video portals from content on screen, with window resizing and clicks influencing the generated experience. Continuum also supports agents that use visual thinking to show their reasoning through real-time visualizations, and it can proactively suggest video interfaces based on what the user is viewing. The demo closes with an interactive Mars outpost and describes these features as early explorations toward generative video interfaces.

Key points

Continuum explores bringing real-time video models beyond the browser and into an operating system.
A prompt bar and tabs let users ask the AI questions and return to previous tasks.
Users can select content on screen to generate visual video portals that help explore a topic.
Generated portals persist across the experience and can be reopened from a side drawer.
Portals have dream-like, fuzzy edges to distinguish generated content from fixed content.
Portals are interactive: resizing a window changes the input to the video model, and users can click within the video.
An info tooltip reveals some of the prompt behind a portal so users can tailor the experience.
For agent tasks, Continuum can choose a suitable mode and open a chat that uses visual thinking to show the model’s reasoning.
Continuum can proactively suggest real-time video interfaces based on the content the user has open.

Chapters

Start	Chapter	Summary
00:00	Introducing Continuum	Ryan introduces Continuum as an exploration of how real-time video models might work within an everyday operating system.
00:31	Creating persistent video portals	The demo shows how users can select screen content to create visual portals that remain accessible while browsing.
01:33	Interacting with generated scenes	Portals can be resized and clicked, with those actions shaping the regenerated scene and its interactive experience.
02:33	Visual thinking with agents	Continuum demonstrates an agent creating a 3D-printable bike mount and using real-time visuals to communicate its reasoning.
04:08	Proactive interfaces and a Mars outpost	Continuum suggests video interfaces based on what is open, then demonstrates an interactive Mars outpost.

Quotes

we’ve been exploring how they might live beyond the browser and in an operating system that you use every day. — 00:00

these portals come with you throughout the experience. — 01:02

resizing a window is actually an input to this real-time video model. — 02:03

it’s really awesome and so much more rich than just a wall of text. — 03:35

Continuum actually generates suggestions for real-time video interfaces based on what I have open or what I’m looking at. — 04:08

Concepts and entities

Continuum, Runway Labs, Runway, Ryan, real-time video models, visual thinking, agents

Full transcript (original language)

00:00 Hello. My name is Ryan. I’m a designer and prototyper here at Runway Labs. And we want to share an early look at a new research application we’re working on called Continuum. The idea here is that as real-time video models become available, we’ve been exploring how they might live beyond the browser and in an operating system that you use every day. So, I want to give a quick demo. All right. You’ll notice along the bottom you have a familiar prompt bar where you can ask anything to the AI.

00:31 And along the top, [music] I have tabs of things I’ve been working on that I can jump back into. So, here’s an article I was reading earlier. And Continuum has context and memory of the types of tasks [music] that I’ve been working on. And so, as I’m reading through this, I can always have Continuum help me dive deeper on the topic. So, at the bottom here, I can select and then select anything on screen that I’m looking at. And Continuum generates a real-time

01:02 video portal. This portal can be 3D visualizations. It can be other content. Really anything that helps me dive deeper on a topic in a visual way. And what’s really awesome is these portals come with you throughout the experience. So, as I’m browsing on the web, for example, I can always scroll over to this right-hand drawer and expand and view portals I’ve previously generated. They have blurry or fuzzy edges, kind of

01:33 dream-like, to help distinguish between generated and fixed content that you’re looking at on screen. What else is awesome about portals is they’re fully interactive experiences. For example, I’ll jump into one that I was looking at earlier. This is all about photosynthesis and plant biology. And just like a traditional window, you can resize them to get a better look.

02:03 But what’s different here is that resizing a window is actually an input to this real-time video model. And so it completely regenerates the scene creatively in a way that gives me more context and control. And I can also click anywhere to interact with this video and get really rich experiences. Besides being able to interact directly with the video,

02:33 we also have an info tooltip in Continuum to expose some of the prompt behind the scenes so that you can really create and tailor these custom experiences that are super personal to you. >> [music] >> All right, I’ll minimize that one for now. Another thing Continuum is awesome at is working with agents. Um so I’m going to go down here to the bottom and I need a 3D mount that’s printed for my bike. So I’m going to say, “Can you make

03:04 a 3D print model for a mount for my bike?” And you’ll notice that Continuum is automatically selecting the correct mode to use for the type of intent that I’m trying to experience. And so here, I’m going to click go and it opens an agent chat window with a new paradigm we’re calling visual thinking. And so the coding model can actually use the real-time video interface to be able to create these 3D visualizations of

03:35 what it’s reasoning through. Uh it’s really awesome and so much more rich than just a wall of text. We think this is a really exciting way to have agents work real-time video models and to be able to visualize themselves in ways that create transparency and trust with these systems, especially when they’re doing things in the real world like 3D printing. All right. Another thing I wanted to share as a part of this is how these interfaces and

04:08 generative video moments can be created proactively within Continuum. And so let me look up something I was looking at earlier today on the web. And you’ll notice at the bottom that Continuum actually generates suggestions for real-time video interfaces based on what I have open or what I’m looking at. So since we’re looking up some of these space topics, let’s jump into a Mars outpost for

04:39 example. And so here, this is a real-time video that’s a whole world that I can step into. So I can actually full screen this right in Continuum. And just like you’d expect, I can click anywhere and interact with the video >> [music] >> in the same way. So here I can zoom into the outpost itself. That’s just an early look at some of the

05:09 new capabilities and features that we’ve been exploring in Continuum. [music] We’re excited to share more soon and keep building towards generative video interfaces.