Posts

In which I apply basic security decomposition, least privilege, sandboxing, and memory compartmentalization to AI agents, and discover that you can, in fact, make things better in this new world. Security Principles in Play The trouble with AI agents isn’t that they are fundamentally insecure, it’s that they are insecure by default. We take decades of understanding about untrusted data cleaning, separation of concerns, zero trust, etc., and chuck AI on top of all of it at once, only then to wonder why our previously secure systems are suddenly vulnerable. ...

When thinking about making AI agents more secure, there are a some important classes into which we can place vulnerabilities. A couple of the more interesting ones are the Confused Deputy problem and Privilege Aggregation. Each of these is an issue on its own, but together they make for a very serious combination. This post series aims to tackles some of the many dimensions of this space, elucidating what’s wrong, and suggesting some ways in which we might try mitigating them. ...

The idea of an AI agent, something that does things on your behalf, but doesn’t go off the rails, is alluring. Everyone wants their own personal concierge, multiple of them if it can be helped, to manage things for them in an increasingly complex world. But the way we talk about it is super ambiguous, and sometimes that makes a difference in how we think about building particular solutions. What is an Agent? It’s worth nailing down some vocabulary, but before we do, let’s go over some examples to ground the discussion. ...

AI writes insecure code, right? We’ve certainly heard that while AI can write code, it also tends to create vulnerabilities that are preventable, to create things that shouldn’t be put out in production. It can stick secrets into code without being asked. It can generate buffer overflow vulnerabilities. It can create authentication that is easily bypassed. Honestly, a lot of that is true. But why is it true, and more importantly, when? ...