A few of us on my team used an AI coding tool to create and deploy an app. Same tool, same task.

I got a clean deployment in one shot, but some of my colleagues spent hours debugging blockers the tool kept flagging. The difference wasn't skill.

I'd happened to confirm to the tool that the deployment approach worked because I had manually deployed an app before (and frankly I thought specifying this was overkill). My teammates chose to let the tool figure it out (which I would've thought was the better, more elegant, cleaner approach), but it couldn't. It kept hedging, surfacing problems that weren't really problems, second-guessing itself.

Technically, the work was exactly the same, but the outcomes were wildly different and this really surprised me. The only variable was how we each started. That was the difference between shipping in an afternoon and spending hours troubleshooting things that the tool should've been able to handle.

We talk about AI capability like it's a fixed number on a benchmark. A benchmark tells you what a model can do. It doesn't tell you what it'll do for you. That part, it turns out, depends a lot on how you start together.