Learn AI Engineering

Build AI That Ships

Practical guides for engineers and operators taking agents and AI workflows from demo to production.

Explore by operating question

Start with the decision your team needs to make.

Latest episodes

Evals - Beyond the Vibe Check | LangChain, Langfuse, Mercor, CoreWeave & Galileo

LangChain, Langfuse, Mercor, CoreWeave and Galileo

Evals - Beyond the Vibe Check | LangChain, Langfuse, Mercor, CoreWeave & Galileo

Everyone says they do evals. Almost nobody does. LangChain actually put a number on it: around 89% of teams have observability set up, and only about a third ever do anything with what it collects. So I got five people who do this for a living in a room, from LangChain, Mercor, Galileo, Langfuse, and CoreWeave. Side note, three of those companies got acquired in the past year. Galileo to Cisco, Langfuse to ClickHouse, Weights & Biases to CoreWeave. The layer is getting bought up faster than most teams can figure out how to use it. A few things I didn't expect going in. Grading an agent while it runs can cost you more than the agent does. The model you're using as a judge is probably wrong in ways you'll never notice. And the guy whose company sells eval tooling told the room to stop writing evals before shipping, just put it out and learn from what breaks. We also got into the parts nobody blogs about. How many traces a person still has to read by hand. Why the same agent is a weekend project internally and a year of work at a bank. What happens when your users start doing things you never thought to test. If you've shipped an agent and had no real way to know whether it was working, this one's worth your time!

The State of Computer Use Agents | Anthropic, Browser Use & KERNEL

Anthropic, Browser Use and KERNEL

The State of Computer Use Agents | Anthropic, Browser Use & KERNEL

Bot traffic on the internet just passed human traffic, two years ahead of forecast. Most of it is agents clicking through websites built for people. So I got three of the people building those agents in a room: Lucas Gonzalez Pagliere, who works on computer use at Anthropic, the team that shipped the first computer use model back in 2024. Reagan Hsu, founding engineer at Browser Use, whose open source library is sitting at around 100K GitHub stars. And Eric Feng, founding customer engineer at KERNEL, which runs the browser infrastructure underneath a lot of this - he was also first GTM at Sentry. We covered where the models genuinely are today vs where the benchmarks say they are, what it costs to run an agent long enough to finish real work, and the arms race between agents and the anti-bot systems trying to keep them out. Then the harder question underneath all of it: whether the web reorganizes itself around agents, or hardens against them. They disagreed on plenty of it. If you want to know what agents can actually pull off on a real website today - and what still stops them cold - this one's worth your time!

Spencer Whitman - Gray Swan AI's $200M Plan to Secure AI Systems

Gray Swan AI

Spencer Whitman - Gray Swan AI's $200M Plan to Secure AI Systems

GPT-5.6 Sol goes rogue and breaches Huggingface. Washington suspends Mythos access within days over national security concerns, and the White House just held an emergency meeting to finalize a classified cybersecurity framework for frontier AI models. AI security went from niche concern to front-page hysteria basically overnight. So I sat down with Spencer Whitman, who recently joined Gray Swan AI as CPO on the back of their $40M Series A. Before Gray Swan, he founded Meta's Llama security team to stop bad actors from jailbreaking their models - he's been on the frontlines of LLM security since the beginning. We get into how Meta pressure-tested Llama for maximum harm before every open source release, why Gray Swan's attack agent has never met an AI system it couldn't break, and the AI Twitter bot that got drained of $200K in crypto in 15 minutes. Spencer also shares his (admittedly speculative) read on whether Meta gave up on the frontier before Alexandr Wang showed up, why anyone can be a hacker now, and how 15,000 red teamers are breaking models before they ever ship. If you want to understand how AI systems actually get broken - and defended - this one's worth your time!