How Relace gets month-to-month GPU allocation that scales

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: How Relace gets month-to-month GPU allocation that scales · Channel: Together AI · 01:04 · Watch on YouTube

Topics: Software engineering with AI, AI infrastructure and hardware, Markets and economy, AI models

Summary

Preston, founder of Relace, says long-term, million-dollar GPU reservations were not accessible to the company. Access to the Together YC cluster gave Relace compute that could scale with its needs and supported its work training small code-generation models. The month-to-month allocation lets the team plan for increased compute demand and count on reserved GPUs being available. Preston also says access to GPU nodes helps Relace attract engineers and is a must-have for a small company like theirs.

Key points

Relace could not readily access million-dollar, year-long GPU reservations.
The Together YC cluster provided compute that could scale up or down with the company’s needs.
Relace has been training small models for code generation.
The team is exploring small models for specific tasks in its code-generation workflow.
Month-to-month GPU allocation helps Relace plan for periods of higher compute demand.
Reserved capacity means engineers can expect a GPU to be ready when they need one.
Access to GPU nodes gives Relace a recruiting advantage.
Preston describes GPU access as a must-have for a company like Relace.

Chapters

Start	Chapter	Summary
00:01	Scaling compute for Relace	Preston describes the limits of long-term GPU reservations and how the Together YC cluster provides scalable compute for Relace’s small-model work.
00:33	Month-to-month reserved capacity	Relace uses flexible monthly allocation to plan for changing compute needs and ensure GPUs are available to its engineers.
01:04	GPU access as a must-have	Preston concludes that access to GPU capacity is essential for a company like Relace.

Quotes

The Together YC cluster really helped us get access to compute that could scale up and down with what we needed. — 00:01

The allocation is is month-to-month — 00:33

We knew that when we would click a button, there would be a GPU ready for us. — 00:33

It’s a must-have for a company like ours. — 01:04

Concepts and entities

Relace, Preston, Together AI, YC cluster, GPU, code generation

Full transcript (original language)

00:01 Million-dollar commitments of year-long GP reservations were not something that was very available to us. The Together YC cluster really helped us get access [music] to compute that could scale up and down with what we needed. >> Hi, I’m Preston. I’m the founder of Relace. We went through YC in winter ’23. And recently, we’ve been training a lot of small models for code generation. And so, we’re trying to find ways to reduce the cost either [music] through using small models directly or training small models to do specific tasks within this code generation

00:33 workflow. Since we got access to the YC cluster, we’ve had much more availability for engineers to build. The allocation is is month-to-month, and so we can say, “Okay, in the next 2 months, we’re going to have a lot more compute needs.” And so, having that reserved capacity was extremely helpful to us. We knew that when we would click a button, there would be a GPU ready for us. Together give us kind of the advantage that small companies need. We’re also trying to recruit engineers, and so being able to say, “Hey, we have like this many nodes of GPUs that you can run experiments on.” [music] It’s a

01:04 must-have for a company like ours.