Watch on YouTube
Watch on Vimeo
Jiachen Jiang argued that the AI incidents making headlines come from weak infrastructure, not weak models, and that teams can prevent them with controls they already have. Jiang said frontier labs can build smarter models but cannot change two properties of AI agents: they are non-deterministic, so an agent can do a task correctly nine times and go off course on the tenth, and they take in instructions and context as the same natural language, so content an agent reads can override what it was told to do. The fix is to enforce limits in the environment, which Jiang compared to locking a door rather than asking someone not to enter a room. Jiang framed the risk with Simon Willison’s “lethal trifecta,” which describes what happens when an agent combines access to private data, exposure to untrusted content, and the ability to communicate externally. One leg is often acceptable, two is risky in regulated industries, and all three should never be allowed, even for a prototype.
Jiang then walked through three incidents, each involving more legs of the trifecta. In the PocketOS incident in April 2026, a Cursor agent asked to fix something in staging, hit a credential mismatch, found a forgotten, broadly scoped token, then deleted the company’s production database and its backups, which sat on the same volume, in under nine seconds. Jiang noted a similar 2025 incident in which a Replit agent deleted a founder’s production database. The second story, from a customer of a Coder customer, began when an attacker phished a bank finance employee and used the employee’s long-lived Copilot session to find sensitive data the employee had exported for Copilot to edit. The attacker then asked Copilot how to get around the bank’s data loss prevention (DLP) controls, and Copilot explained that transfers above 14 KB per second were not flagged and helped split the data into smaller chunks. Months passed before a ransom note revealed the breach. The third case was the Nx “s1ngularity” attack, in which attackers published malicious versions of the Nx build tool to npm. In the three to four hours those versions were live, they prompted developers’ local coding assistants to find credentials on their machines and publish them to public GitHub repositories, exposing thousands of credentials.
Jiang presented six layers of protection and mapped them to Coder. Coder defines environmental separation, short-lived scoped credentials, identity-aligned access control, and agent sandboxing through workspace templates, while the model proxy layer and audit logging run through Coder’s AI Gateway. Coder does not issue cloud or database credentials itself. It passes the user’s identity to systems that do, such as HashiCorp Vault, and its templates can create a credential for each workspace that is then destroyed when the workspace is deleted. Dave Ahr, a developer relations engineer, described templates as a “substrate orchestrator” that stacks safeguards in line with James Reason’s Swiss cheese model, so a gap in one layer is caught by another. In response to delegate questions, Ahr said customers can swap in third-party tools such as LiteLLM or Kong for Coder’s AI Gateway, and that Coder has no certification program for integrations yet but is working to validate common architectures, including routing prompts between frontier and local models. He added that Coder has grown from about 70 employees two years ago to about 210 and is partnering with Anthropic and AWS to add layers it does not build itself. Jiang closed by cautioning delegates to be wary of any vendor promising complete protection for AI agents, because layered controls reduce risk but cannot eliminate it.
Personnel: Dave Ahr, Jiachen Jiang
Thank you for being part of the Tech Field Day community! Our mailing list is a great way to stay up to date on our events and technical content, and we appreciate your signup.
We promise that we’ll never spam you, send ads, or sell your information. This list will only be used to communicate with our community about our events and content. And we’ll limit it to no more than one message per week.
Although we only need your email address, it would be nice if you provided a little more information to help us get to know you better!