Melissa Daniel

Investment Partner @ AGI House Ventures

On August 14, AGI House hosted a dinner sponsored by Forethought on the future of agentic customer experience. Guests included leading researchers like Christopher Manning and Sebastian Thrun, operators such as Nikita Shamgunov (Databricks) and Russ d&#39;Sa (Founder of LiveKit), and founders and executives from Inworld, xAI, Pika, Character.AI, Cresta, and others. Discussion revolved around how agentic commerce will emerge, the constraints of current systems, and the technical pathways toward making transactions programmable.
<img src="/imgs/blogs/agentic-customer-exp-dinner.jpg" alt="Agentic Customer Experience Discussion">
<h2>1 Execution Reliability is Core Limitation</h2>
The central bottleneck in agentic commerce is not intent recognition but reliable execution. Large language models can already parse natural language with sufficient accuracy. What breaks down is the ability to carry out multi-step transactional workflows at production-grade reliability.
<img src="/imgs/blogs/flywheel.jpg" alt="Execution Reliability Challenges">
Three technical levers for solving the execution problem:
<h3>1.1 Substrate of Execution</h3>
One source of fragility lies in the substrate of execution. Modern SaaS platforms expose APIs designed for machine consumption, but many legacy systems still force interaction through GUIs. This produces hybrid architectures where agents call APIs when available and fall back to browser automation elsewhere. Browser automation, however, is inherently brittle. A multi-step sequence with a per-step success rate of 0.9 completes only 35 percent of the time across ten steps, and UI changes can break flows without notice. 
The long-term solution is standardized machine-native protocols, such as the emerging Model Control Protocol (MCP), which would allow agents to negotiate, execute, and settle transactions with guarantees similar to TCP/IP in networking. Until such protocols consolidate, execution reliability will remain uneven.
<h3>1.2 Model Design</h3>
A second dimension of reliability comes from model design. Retrieval-augmented generation is effective for open-ended reasoning, but brittle in transactional contexts. Hallucinating a product ID or misfiring on a payment API erodes trust irreparably. In practice, fine-tuned vertical models consistently outperform generic RAG systems in domains such as travel booking, procurement, or IT service management. 
The emerging pattern is orchestration: a general-purpose model handles broad instruction parsing, then delegates execution-critical steps to fine-tuned agents with embedded guardrails. This specialization is not optional in commerce. Precision is mandatory whenever money or compliance is at stake.
<h3>1.3 Continuous Feedback Loops</h3>
Finally, execution reliability depends on continuous feedback loops. Every transaction produces structured signals: completion rates, error categories, resolution latency, and customer satisfaction. Without reintegrating this data, agent systems plateau. With feedback, they evolve asymptotically toward reliability. 
For example, a browser sequence that fails repeatedly should automatically trigger a request for API integration. A pricing agent should continuously refine its negotiation strategies based on which offers are accepted or rejected. Enterprises that close these loops build compounding performance advantages over those that treat agents as static systems.
Taken together, substrate stability, model specialization, and feedback integration form the three technical levers for solving the execution problem. Until all three mature, agentic commerce will remain constrained to narrow workflows. Once they converge, agents will be able to transact end-to-end with reliability that exceeds human operators.
That reliability imperative is already reshaping how companies are organized.
<h2>2 Organizational Restructuring to Flatter Organization Charts</h2>
Several guests described engineering capacity no longer as a bottleneck. Code generation has freed engineers from routine integration work, allowing them to be redeployed into customer experience, embedding directly in teams that curate and tune agent workflows.
<img src="/imgs/blogs/organization.jpg" alt="Organizational Changes">
This produces flatter organization charts, with ratios of PMs to engineers inverting and in some cases exceeding 1:1. Engineers move closer to the customer surface, while PMs act as system designers: defining workflow boundaries, selecting models, and integrating performance feedback. As agents blur the traditional lines between support, sales, and success, Customer Experience organizations consolidate under single leadership. CX stops being a back-office function and becomes the primary product surface.
<h2>3 Adoption Dynamics and the Build-versus-Buy Shift</h2>
<img src="/imgs/blogs/enterprise-vs-consumer.jpg" alt="Build vs Buy Dynamics">
<h3>3.1 Enterprise vs Consumer Adoption</h3>
Enterprise adoption is structurally easier because the problems are narrower, repeatable, and already well-instrumented. Support flows, procurement approvals, and IT tasks follow deterministic patterns with clear state transitions and logged outcomes. This makes them ideal for fine-tuned agents, which can be measured on metrics like resolution latency, error category, or customer satisfaction. Enterprises also operate in controlled software environments (Salesforce, ServiceNow, Workday) where agents can be restricted to a finite stack, further improving reliability. ROI is immediate: replacing human hours in a call center produces quantifiable savings, so adoption has a direct economic rationale.
Consumer adoption is harder to predict because it requires rewiring human behavior rather than just optimizing workflows. Habits like shopping, browsing, or media consumption are open-ended and less structured, so agents must succeed in noisy and heterogeneous environments. But the potential upside is larger. When agents collapse multiple state transitions into a single loop, they reduce latency and drop-off. Just as infinite scroll transformed media consumption by removing friction, &quot;infinite transactions&quot; could transform commerce.
<h3>3.2 The Economics Reversal</h3>
The economics of adoption amplify this divide. Historically, enterprises bought SaaS because building was too costly. With agents, the equation reverses. Low-cost model fine-tuning and code generation make it feasible for enterprises to replace horizontal SaaS with internal stacks tuned to their own data. 
Examples of this shift:
<ul>
<li>Coho replaced Intercom with an in-house support agent, saving $55,000 annually while gaining tighter integration</li>
<li>Databricks built R2D2, an internal AI agent managing over 140 SaaS tools</li>
</ul>
The common thread is that defensibility shifts from distribution to proprietary data. Without network effects or differentiated datasets, horizontal SaaS vendors risk being displaced by bespoke agents. For consumers, the same principle applies at platform scale: whoever owns the feedback loops on browsing, checkout, and payments will accumulate the data advantage that powers retention.
<h2>4 Outlook</h2>
The near-term architecture of agentic commerce will remain hybrid: browser automation to bridge legacy systems, APIs where available, and fine-tuned sub-agents orchestrated by general LLMs. Enterprises are likely to standardize on these patterns over the next two years. Within three, protocol layers for agent-to-agent transactions may consolidate, laying the foundation for a truly agent-native economy.
At every stage, the key variable is reliability. Gains in per-step execution, error recovery, and feedback integration will dictate how quickly workflows shift from human-mediated to fully agentic. The demand is clear. Enterprises want efficiency, and consumers want seamless experiences.
Commerce is among the first domains to to become a truly agent-native economy since it’s structured, repeatable, and directly monetizable. Each transaction generates feedback signals that agents can use to improve. This makes commerce both a proving ground and a wedge: the domain where reliability will be solved first, and from which the architecture will spread to other sectors.

Agentic Customer Experience and the Architecture of Programmable Transactions

Henry Yin

Co-founder of AGI House

Naomi Xia

Investment Partner at AGI House Ventures

<h2>1 Scaling Stalls, RL Accelerates</h2>
For the past couple of years the recipe felt obvious: stack more layers, feed more data, burn more GPU hours. Loss kept falling, and capability edged upward. However by late 2024, that cadence broke. Pre-training alone plateaued near GPT-4-level performance; almost every fresh gain now arrived after pre-training, in post-training. We’d scraped the public web to bedrock, and in his NeurIPS 2024 keynote, Ilya Sutskever observed that “pre-training as we know it will end,” signalling the need for synthetic data or wholly new paradigms.
Around that time, OpenAI released the <a href="https://openai.com/index/learning-to-reason-with-llms/">Learning to Reason with Language Models</a> blog post introducing a new scaling paradigm. It showed that performance rises smoothly along two axes:
<ol>
<li>RL compute (how much GPU you spend on reinforcement fine-tuning) </li>
<li>Inference compute (how many tokens you let the model generate at test time)</li>
</ol>
Reinforcement learning supplies that loop. Instead of brute-forcing next token prediction, the model does a task, receives feedback, and is steered:
<ul>
<li>Try a task → get feedback </li>
<li>Reward good attempts </li>
<li>Discourage bad ones</li>
</ul>
That cycle powered the year’s two headline breakthroughs: OpenAI’s o-Series reasoning models and <a href="https://arxiv.org/abs/2501.12948">DeepSeek-R1-Zero</a>, an RL-only variant that learned to chain thought, allocate compute, and rival o-Series-level reasoning using verifiable rewards alone.
<h2>2 RLVR: Data‑Light, Transfer‑Heavy</h2>
Fine-tuning with supervised data feels like hauling sandbags: every new skill demands a mountain of hand‑labels, and you still risk washing out what the model already knows. Reinforcement Learning with Verifiable Rewards (RLVR) flips the script such that one crisp reward can replace hundreds of fuzzy labels and let the model bootstrap itself.
Two recent works showcasing RL’s superior data‑efficiency and transfer capability
<ul>
<li><a href="https://arxiv.org/abs/2504.20571">Wang et al., 2025: Reinforcement Learning for Reasoning in LLMs with One Training Example</a> Conventional wisdom says RL‑for‑LLMs needs thousands of reward‑labeled traces. Wang shows the opposite: a single flawless chain‑of‑thought plus a unit‑test checker (“1‑shot RLVR”) doubles accuracy on a gnarly MATH500 item and lifts six unrelated reasoning benchmarks. Proof that data efficiency, not data volume, drives the next gains </li>
<li><a href="https://arxiv.org/abs/2507.00432">Huan et al., 2025: Does Math Reasoning Improve General LLM Capabilities?</a> Math leaderboards are the new IQ test, but do those points travel? Huan introduces a Transferability Index (TI) - &quot;for every point of math gain, how much do you gain (or lose) elsewhere?&quot;, and runs a controlled ablation on Qwen3‑14B: same math data, SFT vs. RL (GRPO). Result: half the math‑specialised SFT models show negative TI on non‑math tasks, while RL fine‑tuning keeps, or even boosts, general skills. RL &gt; SFT when you need deep specialisation without collateral damage</li>
</ul>
In practice, RLVR trades dozens of curated samples for one rocksolid reward and sculpts model behaviour around what actually matters. You avoid catastrophic “forgetting” and unlock broad generalisation in one sweep, no extra data‑wrangling required.
Economically, it rewrites the playbook. Yesterday’s fine‑tuning firms built margin on SFT APIs and endless annotation pipelines. That SFT budget is ≈ US $5 B today—and surveys show 40-60 % of it is already slated to migrate to RL-centric pipelines in the next two years, a swing worth roughly US $2-3 B. Tomorrow’s winners will package RL as a service: turnkey environments mapped to customer workflows, verifiable rewards baked in, and elastic RL loops that squeeze every drop of insight from private data. That’s the next edge.
<h2>3 The Hard Part Left: Your Environment</h2>
“Environments that capture real-world, proprietary processes—things you can’t scrape or simulate—are the most durable value. Generic web tasks aren’t compelling”
<img src="/imgs/blogs/o1-receipe.jpg" alt="The Recipe Behind Advanced RL Models">
The recipe behind DeepSeek‑R1 and OpenAI’s o‑Series distils to three ingredients:
<ol>
<li>A strong base model. Open‑source checkpoints like DeepSeek V3, Kimi K2, and Qwen 3 already sit within striking distance of frontier LMs on raw reasoning. </li>
<li>A policy‑gradient loop. Libraries such as VeRL let you fire up PPO or GRPO with two YAML edits; the Tulu 3 repo even ships the exact hyper‑params that match or beat several closed models. </li>
<li>An environment with a rock-solid, verifiable reward. Rule‑based unit tests, DOM diffs, symbolic checkers - the one piece you can’t download from Hugging Face.</li>
</ol>
The first two are commodities: checkpoints on Hugging Face and a quick `pip install verl` can get a small team training in half a day. The defensible moat is your domain’s gym - a containerised replay of real workflows (trading books, UI‑stress sandboxes, pharma‑compliance crawlers) plus the reward logic that measures true success.
Point an open recipe at that gym, spin a few thousand GRPO steps, and watch the model learn by doing. No closed‑door deals, no proprietary API keys. As these pipelines proliferate, every vertical will spawn RL‑as‑a‑Service vendors offering turnkey gyms and reward packs. In that world, the last mile of defensibility isn’t in model code. It’s in the highest‑fidelity environment libraries where your rewards and your expertise live.
<h2>4 Four Pillars of a High‑Quality RL Environment</h2>
“Now we’re in the agentic era: multi-turn, tool-using, environment-interacting agents. Training, inference, and env-execution all decouple, so the gym quality, not the base model, is the bottleneck.”
<img src="/imgs/blogs/rl-env-4-pillars.jpg" alt="rl-env-4-pillars">
<h3>4.1 Fidelity — why “sim‑to‑real” now matters for software agents</h3>
In robotics, sim‑to‑real means a controller trained in Gazebo or Isaac Gym can bolt straight onto hardware without nasty surprises. The same idea now applies to language model agents. If your coding environment fakes the Git repo or your browser-use replica skips latency, auth tokens, or race conditions, the policy will overfit to quirks you’ll never see in production.
A high-fidelity RL environment must reproduce the genuine action/observation loop:
<ul>
<li>Coding Real file‑tree diffs, authentic CI latency, flaky compilers. </li>
<li>Browser‑use Live DOM mutations, session tokens, throttled network, paginated APIs, real side‑effects.</li>
</ul>
Positive‑signal examples: SkyRL‑Gym spins SWE‑Bench repos in Docker and runs the actual <code>pytest</code> suite; fixes that pass in training nearly always pass on GitHub CI. WebArena snapshots real SaaS dashboards inside headless Chrome; agents must navigate DOM trees, obey auth scopes, and handle time‑outs just like a human.
Reward‑hacking failures: In the first release of MiniWoB++, every web page looked exactly the same in every episode. A clever agent quickly discovered that a single hard-coded click at a fixed screen coordinate could “solve” every task, pushing success to 100 %. When the benchmark’s maintainers later shuffled element positions on every reset, mirroring the variability of real websites, the agent’s performance collapsed until it learned genuine DOM navigation. The lesson is clear: low-fidelity simulations invite degenerate shortcuts.
<h3>4.2 Robust Rewards — the guardrails that keep agents honest</h3>
A learning signal only helps if it can’t be gamed. In <a href="https://arxiv.org/abs/1606.06565">CoastRunners</a> (Amodei et al., 2016), an Atari agent learned to spin in circles to farm buoys instead of finishing the race. Modern LLMs are just as opportunistic.
<ul>
<li>Positive pattern Unit‑test truth serum: SWE‑Bench rewards only when the patched repo passes all tests in a sandbox; printing expected output earns zero. </li>
<li>Failure mode Early SWE‑Bench clones let agents insert <code>@unittest.skip</code> (or delete failing tests) instead of fixing the bug; the unit‑test runner passed, reward hit 100 %, but nothing worked in real CI.</li>
</ul>
How to build them: Prefer deterministic checks (unit tests, formal proofs, SQL executors); randomise test order; use majority vote LLM judges for subjective tasks; add honeypot fields that trigger a large negative reward.
<h3>4.3 Throughput — keeping GPUs busy, decoupling slow environments</h3>
RL for LLMs touches three very different subsystems: GPU inference, CPU/IO environments, and GPU learning. If any one stalls, the others idle. Classic actor–learner splits (IMPALA, SEED‑RL) fixed this for Atari; SkyRL revives the trick for coding agents, overlapping unit‑test execution with next‑batch generation and boosting sample throughput 4 to 5×.
Good engineering Disaggregate roles: learners on A100s, inference on 4090s, envs on autoscaled CPU pods. Use asynchronous queues (Ray, Redis) so stragglers don’t block the batch. Cache deterministic tool calls; prune rollouts when the first test fails.
Bad engineering Synchronous PPO loops where one slow browser page freezes all GPUs; monolithic Docker images that rebuild when you tweak a reward.
<h3>4.4 Extensibility — environments that evolve as fast as the product</h3>
APIs ship weekly; new tools appear every sprint. A durable environment must accept plug‑in actions, observations, and reward functions without a rewrite.
<ul>
<li>Good practice Schema‑driven action spaces in JSON; plug‑in rewards registered via entry‑points; CI contract tests catch breakage. SkyRL‑Gym added a SQL executor in ~50 LOC. </li>
<li>Failure pattern Action IDs hard‑coded into model vocabularies; adding one shifts token indices and ruins checkpoints.</li>
</ul>
“Now we’re in the agentic era: multi-turn, tool-using, environment-interacting agents. Training, inference, and env-execution all decouple, so the gym quality—not the base model—is the bottleneck.”
<hr>
<h2>References</h2>
<ol>
<li>OpenAI (2024). <a href="https://openai.com/index/learning-to-reason-with-llms/">Learning to Reason with Language Models</a>. The foundational blog post that introduced the &quot;RL ≻ scale&quot; mindset and demonstrated the new scaling paradigm.
</li>
<li>DeepSeek-AI (2025). <a href="https://arxiv.org/abs/2501.12948">DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning</a>. The arXiv paper behind DeepSeek-R1-Zero, the RL-only model that rivals o-Series performance.
</li>
<li>Wang et al. (2025). <a href="https://arxiv.org/abs/2504.20571">Reinforcement Learning for Reasoning in Large Language Models with One Training Example</a>. Demonstrates how one-shot RLVR can achieve remarkable data efficiency without mountains of labels.
</li>
<li>Huan et al. (2025). <a href="https://arxiv.org/abs/2507.00432">Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning</a>. Introduces the Transferability Index and shows RL&#39;s advantages over SFT for specialized training.
</li>
<li>Amodei et al. (2016). <a href="https://arxiv.org/abs/1606.06565">Concrete Problems in AI Safety</a>. The classic paper on AI safety challenges, including the famous CoastRunners reward-hacking example.
</li>
</ol>

AGI House Blog

Agentic Customer Experience and the Architecture of Programmable Transactions

The Reinforcement Learning Renaissance (Part 1)