How to Test Your Code (Even When AI Writes It)

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: How to Test Your Code (Even When AI Writes It) · Channel: ByteByteGo · 04:08 · Watch on YouTube

Topics: AI agents, Software engineering with AI

Summary

The video explains how unit, integration, and end-to-end tests catch different kinds of software bugs. Unit tests are fast and isolate small pieces of logic, while integration tests check interactions between system components and end-to-end tests verify complete user workflows. The testing pyramid recommends many fast tests at lower layers and fewer broader tests, with each test placed at the lowest layer that provides enough confidence. AI can speed up testing, but tests based on the same assumptions as AI-generated code may pass even when the code is wrong, so expected behavior needs an independent source.

Key points

Passing unit tests does not guarantee that the product works as a whole.
Unit tests check small pieces of logic in isolation and are typically fast to run.
Integration tests verify that components work together, such as a checkout service and a database or payment service.
End-to-end tests exercise the complete user workflow across the front end, back end, database, and supporting services.
Broader tests are generally slower to run, harder to set up, and more difficult to debug.
The testing pyramid calls for many unit tests, fewer integration tests, and a smaller set of end-to-end tests for important workflows.
Choose the lowest testing layer that provides the confidence needed.
AI-generated code and tests can share the same incorrect assumption, allowing both to pass while the implementation is wrong.
Expected behavior should come from an independent source, such as a specification, API contract, or acceptance criteria.
AI can make feedback loops cheaper, but it does not define what correct means.

Chapters

Start	Chapter	Summary
00:00	Unit tests and their limits	The video introduces different testing layers and explains how unit tests check isolated logic, such as order pricing, discounts, taxes, and shipping.
01:03	Integration tests	Integration tests check whether parts of the system work together, including whether checkout saves the right data and sends the expected fields to a payment service.
02:05	End-to-end tests and the testing pyramid	End-to-end tests verify a complete customer workflow, and the testing pyramid balances these broader tests against faster, more focused tests.
03:08	Testing AI-generated code	The video warns that AI-generated code and tests can share a mistaken assumption, so expected behavior and review should come from independent checks.

Quotes

A program can pass every unit test, and the product can still be broken. — 00:00

An end-to-end test checks whether the complete user workflow works. — 02:05

The idea is to test something at the lowest layer that can give us the confidence we need. — 02:36

Don’t let the agent that build it grade its own work. — 03:38

AI can make it much cheaper to build and use these feedback loops, but it doesn’t define what correct means. — 04:08

Concepts and entities

unit test, Integration test, End-to-end test, Testing pyramid, CodeRabbit, API contract, Acceptance criteria

Full transcript (original language)

00:00 A program can pass every unit test, and the product can still be broken. Maybe the database cost is wrong. Maybe two services disagree on an API. Or maybe everything works individually, but the checkout flow still fails for the user. That’s because different tests catch different kinds of bugs. So, let’s look at the three layers of software testing and what each one is good for. Let’s start with unit tests. In this example, we’re building an online store. We have a function that calculates the final price of an order, including discounts, taxes, and shipping. A unit

00:32 test might check whether a 10% discount is calculated correctly, or whether an order over $50 gets free shipping. We’re testing one small piece of logic on its own with no database and no other services. That’s why unit tests tend to be fast, and we can run a lot of them every time the code changes. But testing pieces in isolation isn’t enough. Our checkout service, for example, needs to save the order to a database. It also needs to call the payment service. This is where integration tests come in. An

01:03 integration test checks whether multiple parts of the system actually work together. We might call the checkout API and verify that it writes the right data to the database. Or we might verify that checkout sends the payment service the fields it expects. The checkout logic could pass every unit test and still fail here because the SQL is wrong, or because the two services disagree on an API format. Today’s video is sponsored by Code Rabbit, the most installed AI app on GitHub and GitLab. These days, one AI-generated pull

01:35 request can touch a dozen unrelated things, and reviewing it means bouncing between tabs. Code Rabbit’s new review UI fixes that. It splits the diffs into small, related chunks you can take one at a time. Click any function, and its definition pops up right there. Got a question about the change? Just ask, and you get the answer without leaving the page. And it flags the critical stuff first, so you cut review time and bugs in half. Start a 14-day trial, link in the description. Now compare that with an end-to-end test. Instead of calling

02:05 the checkout API directly, we open the website like a customer would. We add something to the cart, go through checkout, and make sure the confirmation page appears. Now we are testing the whole path, the front end, back end, database, and the services behind them. That’s the key difference. An end-to-end test checks whether the complete user workflow works. The broader the test becomes, the more the real system it exercises. But it also becomes more difficult to set up, slower to run, and harder to debug. If a

02:36 unit test fail, there’s usually a small amount of code to investigate. If an end-to-end test fails, the problem could be almost anywhere along the path. This is where the idea of a testing pyramid comes from. We generally want lots of small, fast tests near the bottom, fewer integration tests above them, and a smaller set of end-to-end tests covering the most important workflows. The idea is to test something at the lowest layer that can give us the confidence we need. So what changes when AI writes the code? Mostly, the speed

03:08 and the volume of change. An AI agent can make a large change, generate tests for it, run those tests, fix the failures, and try again. But there’s one failure mode we need to watch for. If the agent writes the implementation, then writes tests based on the same implementation, it can carry the same wrong assumption into both. The code is wrong, the tests agree with the code, everything passes. The expected behavior has to come from somewhere else, a specification, an API

03:38 contract, or a set of acceptance criteria. For important changes, we can also have a separate review step, or even a separate agent to verify the result. Don’t let the agent that build it grade its own work. The same basic testing pyramids still applies. Unit tests give us fast feedback. Integration tests catch problems at the boundary between systems. End-to-end tests tell us whether the whole workflow actually works. AI can make it much cheaper to build and use these feedback loops, but

04:08 it doesn’t define what correct means. We still have to do that.