New Way Now: Captur delivers last-mile AI visual verification at the edge with Google Cloud
Read the Portuguese translation →
Author: Google Cloud · Watch on YouTube
Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.
Original language: English
Original video: New Way Now: Captur delivers last-mile AI visual verification at the edge with Google Cloud · Channel: Google Cloud · 03:17 · Watch on YouTube
Topics: Data, databases and embeddings, AI infrastructure and hardware, Multimodal models
Summary
Section titled “Summary”Captur uses real-time, mobile-friendly computer vision to verify photos and actions at the moment they happen, helping prevent disputes, refunds, and compliance issues. Its models learn from real-world scenarios and user feedback, while on-device inference provides immediate guidance to riders and drivers. Google Cloud services support Captur’s backend, data workflows, and model development, and model distillation helps adapt larger-model capabilities to smaller on-device models. The company is also exploring more specific feedback by combining visual understanding with a planning layer that recommends actions.
Key points
Section titled “Key points”- Captur applies computer vision to verify scooter parking and delivery photos in real time.
- On-device prompts can help users correct photos before problems lead to disputes or refunds.
- One delivery partner saw a 30% reduction in did-not-receive claims in the first week; 96% of drivers corrected their photos when prompted.
- Captur uses Cloud Run, BigQuery, Cloud Storage, Pub/Sub, and Gemini as part of its Google Cloud stack.
- Inference and processing happen on users’ devices, so Captur works to adapt model performance to smaller models.
- Model distillation lets a smaller model learn from a larger model’s behavior.
- Terraform and Gemini support custom model training and experimentation.
- Captur uses synthetic data and BigQuery vector search to help improve its models.
- The next step is to combine visual understanding with a planning layer that recommends actions.
Chapters
Section titled “Chapters”| Start | Chapter | Summary |
|---|---|---|
| 00:00 | Real-world data and visual verification | Captur describes how real-world scenarios and user feedback help tune its computer vision models for immediate verification. |
| 00:33 | Guiding riders and delivery drivers | The models catch problems in scooter parking and delivery photos and can prompt users to correct them on the spot. |
| 01:38 | Google Cloud and on-device models | Captur outlines its Google Cloud services and explains how model distillation helps run capable models on phones. |
| 02:44 | Model improvement and the next leap | Gemini, synthetic data, and BigQuery help improve models as Captur works toward more specific feedback and recommended actions. |
Quotes
Section titled “Quotes”Our edge is built on what we do with the data. — 00:00
We move the verification into the moment on device. — 00:33
96% of drivers corrected their photos on the spot when prompted — 01:08
We’ll have the smaller model learn from what the larger model is doing. — 02:12
Captures is creating a new way to capture the world. — 03:17
Concepts and entities
Section titled “Concepts and entities”Capture, Samana Sarma, Google Cloud, Cloud Run, BigQuery, Gemini, Terraform, distillation, Cloud Storage, Pub/Sub
Full transcript (original language)
Section titled “Full transcript (original language)”00:00 Thank you for that. >> [music] >> Our edge is built on what we do with the data. It’s this ability to find unique scenarios that only occur in the real world and we are able [music] to tune our model based off the user feedback. My name is Samana Sarma. I’m the CTO of Capture. >> Before Capture, businesses either trusted blindly or waited for slow
00:33 manual reviews. We create these real-time mobile-friendly computer vision models. [music] If a rider parks a scooter incorrectly or a driver takes a delivery photo that does not meet the rubric, we can catch that. We move the verification into the moment on device. [music] That will allow both the system and the recipient to get a good photograph. Problems get caught before they become disputes, refunds, or compliance gaps. I can give you an example. I got a delivery. All I got was a photo of the package in a bush. When the courier is taking the photograph, we can nudge them slightly. One delivery
01:08 partner saw a 30% reduction in their did not receive claims in the first week. 96% of drivers corrected their photos on the spot when prompted and invisible package deliveries dropped we have become instrumental to prove our riders are looking after the city. Capture leverages Cloud Run for backend services. Scale to zero has been one of our most powerful benefits. It’s allowed [music] us to quickly experiment with
01:38 options, BigQuery for data warehousing, Cloud Storage, Pub/Sub events, and then [music] Gemini for model training. We also use Gemini for the multimodal AI capabilities. So, this integrated [music] stack lets our very small engineering team deliver at scale. Each new phone is essentially a new platform for us [music] to support. All of the inference and processing happens on the device itself. We can’t actually ship the large model on device. That’s going to consume too much power. And so, we need to find a way to get the larger model’s performance and get it running on the smaller model. And that’s the
02:12 [music] purpose of distillation here. We’ll have the smaller model learn from what the larger model is doing. We have to [music] pick and choose the different models that will be put together to create this one final model that will actually run on device. With the use of Terraform and Gemini, we were able to train our custom models. [music] GCP lets us do that without this army of ops engineers that you’d normally need. We use Gemini to test what’s feasible before we start committing to our
02:44 on-device [music] models. And if Gemini’s struggling, we know that the task is framed in a very complex way. Previously, we’d have to iterate. And this allows us to automate our performance. And [music] we also can generate this array of synthetic data. We’ve also made heavy use of BigQuery, things like vector search. We can query semantically. And that would allow us to pull in this data to train the model to improve. The next leap, however, is making the feedback more specific. As these models become small enough and efficient enough, we can start combining this visual understanding and then a planning layer that decides what action to recommend. With Google Cloud,
03:17 Captures is creating a new way to capture the world. >> [music]