The Agent Works. Does Anyone Trust It? — Michele Grant, Leading at Scale

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: The Agent Works. Does Anyone Trust It? — Michele Grant, Leading at Scale · Channel: AI Engineer · 14:46 · Watch on YouTube

Topics: AI agents, Enterprise AI applications, AI evaluation and benchmarks, AI governance and regulation, Markets and economy

Summary

Michelle Grant argues that a working AI agent is not enough: enterprise teams must trust it before relying on it in core business workflows. She proposes answering three questions for every agent: what value it creates, who is accountable, and what evidence permits greater autonomy. Using an M&A integration scenario, she outlines value metrics, ownership and controls, and technical evidence such as decision logs, model inventories, and autonomy scorecards. Trust grows through measured, staged deployment, with independent sampling and clear human authority helping ensure the agent can operate responsibly at scale.

Key points

A successful launch proves that a system exists, not that people will use it or that it will deliver business value.
Leadership should define an agent’s value, accountable owner, and evidence requirements before granting it greater autonomy.
Set pre-deployment baselines for cycle time, analyst capacity, and detection time; account for operating costs and how saved capacity is used.
Apply the three lines model: management owns controls, risk and compliance provide oversight, and internal audit provides assurance.
Assign a daily human overseer who understands the agent’s boundaries, can interpret its output, can override it, and can safely shut it down.
Document human decision rights, required evidence, and escalation paths; do not deploy where a decision has no assigned owner.
Use an auditable decision log, a model inventory record, and an autonomy scorecard as evidence for governance and autonomy decisions.
Keep core calculations and reconciliations in deterministic, finance-approved code; treat model summaries as explanations rather than proof.
Progress through four autonomy stages: shadow mode, supervised draft, exception ensemble, and exceptions only.
At higher autonomy, monitor missed issues through independent sampling, error severity, and override rates; a falling override rate alone does not prove improvement.

Chapters

Start	Chapter	Summary
00:02	Trust as a condition for enterprise AI	Grant explains why go-live is not proof of value or adoption and introduces the three questions leaders should answer about every agent.
02:41	Measuring value in an M&A integration	An agent that prepares synergy reports provides a case study for setting value metrics and pre-deployment baselines.
06:25	Accountability, controls, and human authority	Grant assigns ownership across the three lines model and describes the human oversight, decision rights, and escalation paths an agent needs.
09:30	Evidence for autonomy and staged deployment	She presents the records and scorecard used to govern an agent, advance it through autonomy stages, and monitor risks.
13:43	The value of trusted agents at scale	Grant returns to the three leadership questions and concludes that an agent earns trust by providing verified evidence that supports timely action.

Quotes

Go-live proves that a system exists, but it doesn’t prove the people behind the work will actually use it. — 00:02

What value does it create? Who is on the hook for it? And what evidence gives it permission to run with greater autonomy? — 01:07

If the IMO cannot state the baseline today, do not deploy. — 05:54

Your real proof lives in the telemetry. — 10:32

Answering those three questions is what earns it the trust to run at scale. — 14:46

Concepts and entities

Leading at Scale, Integration management office (IMO), Three lines model, Human work standard, Autonomy scorecard, Michelle Grant, AI Engineer

Full transcript (original language)

00:02 Go-live proves that a system exists, but it doesn’t prove the people behind the work will actually use it. Or that it will ever deliver a single dollar from the deal model. I am Michelle Grant, creator of Leading at Scale, and the question every enterprise is confronting right now is the agent works, does anyone trust it? I have spent over 20 years in enterprise transformation and more than a decade leading M&A

00:33 integrations across media, tech, and consumer goods. On those deals, trust was not something we crossed our fingers and hoped for after launch. We engineered it before day one through early change management and risk management. So, when the platform went live, the teams on the ground actually relied on it. AI agents demand the exact same muscle.

01:07 An agent earns trust when it delivers the value the business budgeted for while staying strictly within the company’s approved risk appetite, policies, and regulatory guardrails. In my experience, leadership gets there by answering three straightforward questions. What value does it create? Who is on the hook for it? And what evidence gives it permission to

01:38 run with greater autonomy? The moment an agent steps out of a sandbox and into a core business workflow, the entire test changes. A pilot tests whether the software functions, but daily operations is where we test whether it creates value within the controls set by leadership. Established AI risk frameworks give you strong principles. What they do not give you is how to

02:09 apply these principles to each agent. For every agent, leadership has to decide four things. What is it permitted to do? Who owns it? What evidence it records and what triggers an escalation. That work is not an engineering task. We find it is an executive operating mandate. And today, I’ll show you how to meet it. Let us ground this in a real-world

02:41 scenario. Picture a $15 billion acquisition targeting 400 million in net synergies. The deal has already closed and the integration efforts are now underway. The integration management office, which we’ll also refer to as the IMO today, is driving the synergy roadmap across 40 workstreams. Covering everything from rationalizing vendor contracts to consolidating ERP

03:13 systems. And every single initiative has a business owner accountable for its bottom-line savings. Every week, analysts in the IMO are going to reconcile the numbers against the general ledger. As they track down status from all 40 leads, they’re also comparing actuals against the plan, then finally assembling the executive deck for the steering committee.

03:45 Which is going to include the CFO, the chief risk officer, business unit heads, and the integration lead. The engineering team builds an agent automate that first draft. The agent then ingest the weekly updates, reconciles the ledger, runs variance math against the plan, and flags the initiatives that are falling behind. Everything looks good, and they decide to run a four-week pilot.

04:18 The agent’s drafts match the analyst reports line for line. So, we can say that the tech works. Now, the question is, what value does that steering committee need before they can trust it to run the business? First, the steering committee need a written value case agreed upon before the agent ever touches production data. In our example, that comes down to three

04:50 specific metrics. First, let’s look at cycle time. If that weekly report is five business days to compile right now, you’re going to want to set a far target such as two days. Your second metric will be analyst capacity. If four analysts spend half of their week building this deck, that is 80 hours a week the agent gives back. That time counts as value only when the

05:23 integration lead assigns it to the workstreams that are behind on their synergy targets and need help getting back on track. Then, you only count that value after netting out the compute, licensing, and human review cost of running the agent. Your third metric is the most important one, detection time. That is how long it takes from the first sign of a slip in the data to the moment

05:54 the steering committee sees it. Cutting the report from five days to two days does not guarantee that the committee sees that slip any sooner. The agent class to catch it and the value case measures that directly. Every single metric requires a pre-deployment baseline. If the IMO cannot state the baseline today, do not deploy. If you can’t measure where you started,

06:25 you can never prove the agent moved the needle. Next, the steering committee needs to know who owns the risk the agent creates. We are going to apply the three lines model to the agent. In the first line, management owns the controls. The integration lead is the accountable executive. She owns the agent’s output exactly the way she owned the manual data.

06:56 In the second line, risk and compliance oversee the agent and challenge how to run it. How deep they go depends on the agent’s exposure. An internal draft scrubbed by a senior analyst warrants a very different review standard than an autonomous agent publishing figures straight to the board. In the third line, internal audit provides assurance deciding based on

07:26 material risk when and how the agent enters the annual audit plan. The agent also needs one employee assigned to oversee it every day. In our example, that is the senior analyst. He reviews the draft before it reaches the committee. Her role requires four specific capabilities. She has to first understand the operational boundaries. She has to interpret its output with

07:57 context. She must have the authority to override or reverse it. And she needs clear kill switch to shut it down safely this ticket it behaves unexpectedly. Accountability also means defining where the agent’s authority stops. I document this in a one-page policy called the human work standard. For every fork in the process, it maps

08:28 the decision, the human role authorized to make it. The required evidence and the escalation path. These decisions exist at every tier. The senior analyst decides whether a variance is an accounting timing difference or a real cash shortfall. The work stream lead decides how to remediate that shortfall. And the integration lead is going to decide whether to formally flag an

08:59 initiative as at risk to the steering committee. The agent prepares the analysis behind every one of those decisions and it stops there. The person who makes the decision owns the consequence. So keeping these decisions with your team does not slow the agent down. It can go into use while each role keeps its authority. The one reason to delight delay

09:30 deployment is a decision with no assigned owner. This means if leadership has not agreed who declares an initiative off track, the agent’s flights lead to no action. The third question is where the engineers in this room come in. The evidence for greater autonomy comes from how the agent is built.

10:00 I look for three artifacts. The first is an auditable decision log. For every workstream the agent evaluates, it records the raw data ingested, the tool calls executed, the prompt, and the mono versions. Output generated along with the human sign-off. Two engineering choices make that log airtight. First, your core math and reconciliation

10:32 rules must run in deterministic, finance-approved code, not generated as conversational prose from an LLM. Second, treat the model’s written summary as a helpful explanation, not proof. Your real proof lives in the telemetry. What records were pulled? What deterministic tools ran? And who signed the review?

11:02 If reported savings do not match the general ledger, agent does not touch the number. It flags the discrepancy and roots it up. The second artifact is the model inventory record. This captures the accountable executive, data lineage, dependencies across source systems, validation results, escalation path, and the kill switch run book.

11:35 Most importantly, it documents your regression triggers, which are any updates to our current base model or source API that require full revalidation before running in production again. Your third artifact, this is going to be the autonomy scorecard. The steering committee uses it to grant autonomy in four stages. [clears throat] Stage one is shadow mode. The analyst built the report as they do

12:08 today. The agent runs right alongside them and the team compares the two. Stage two is the supervised draft. The agent compiles the first draft and the senior analyst reviews every single line item. Stage three is the exception ensemble, where the analyst reviews lag initiatives plus a random sample of the claims. Then we have stage four,

12:39 where it’s exceptions only. The analyst reviews only high-risk anomalies supported by periodic independent sampling. Pay close attention to stage four, however, because once humans only look at exceptions, your biggest vulnerability is what the agent failed to flag. So I advise tracking mis material issues through independent sampling, monitoring error severity and watching

13:10 your override rate carefully. A falling override rate does not automatically mean the agent is getting smarter. It can also mean the depth of review has slipped. So independent sampling tells you which one of these you are seeing. The scorecard puts those control metrics side by side with your value metrics, cycle time, analyst hours deployed, and your detection time. And with that single page view, the

13:43 steering committee can make clean, efficient, data-backed decisions to either promote the agent, hold it where it is, step it back, or down to supervised review stage. So, let’s come back to the three questions. Ask them of every agent you build, fund, or govern. What value does it create? Who is accountable for it?

14:14 And what evidence permits it to operate with autonomy? In our M&A example today, the agent did not magically create the 400 million in synergies. Procurement, supply chain, and operating leaders captured that value through the gritty work of integration. But what the agent delivers is clear. Verified evidence weeks ahead of schedule, giving leadership the runway intervene while the deal’s synergies can still be

14:46 captured. Building an agent that works is just the first step. Answering those three questions is what earns it the trust to run at scale. Thank you.