"One-shot" AI development (creating entire code projects with a single prompt) is a myth that ignores the need for context. AI agents can't read your mind. And, without essential guidance on scope and environments, they make "reasonable" assumptions that are wrong. True efficiency requires collaboration and oversight. By pairing devs with tools like Claude Skills, devs align their AI assistants with project realities and drive helpful solutions.
The techno-sphere has been abuzz lately with the idea of "one shot" development, where a person gives a coding agent a prompt and it builds an entire project in one session, almost autonomously. The implication is, spoken or not, that having a human in the loop is a temporary inconvenience and soon enough people will be able to sit back in their chairs and Captain-Kirk their supercomputers into perfectly building whatever they want. Over time, it's become increasingly clear that humankind's initial impression of what AI is and is meant to be has been shaped more by assumptions and sci-fi futurism than by current capabilities and real-world use cases.
Context is absolutely critical
We've been treating AI as if it is a robot human, an autonomous mind that you can hand projects to. Some people go further, especially in talking about one-shot development, and almost treat AI like it's a mind-reader, needing no context from you, able to figure everything out itself. In reality, whether you're using AI as a developer tool or asking a human to build something for you, giving them a short prompt is like walking into a contractor's office and saying "build me a house," then covering your ears and running out. Context is needed.
I tested the whole one-shot idea in a livestream recently, and I came away understanding less about what even constitutes a one-shot and why anyone wants it in the first place. I gave the agent a prompt to build an integration and let it get to work to see how far autonomy can get. It started asking me a variety of questions about what I was trying to do and how I wanted it built, while making assumptions about other parts of the project. Basically, it was asking for the context I didn't give it for my one-shot prompt. Not surprising, of course, but does that still even count as a one shot?
What's the point of a one-shot if you have to write a tome of text to get it to run properly?
Why am I even talking about one-shots? Who cares? Where am I?
Okay, I'm back, and I've calmed down.
So forget the one-shot. The fewest-prompts contest was never the point, and it's based on a bad assumption: that the goal is to hand off the work and walk away. Instead, it's more sensible to see building with AI as collaborative and iterative. You can't cram all the context into one prompt, and you shouldn't try.
Claude Skills aren't designed to do tasks in the fewest number of prompts. They give the agent the context it needs and the definition and shape of success as it executes the tasks. It's like training an assistant to help the developer as they build, and some aspects of a project will need a human in the loop for a long time to come.
Why dev needs humans
For example: scope. If you've spent much time building with AI agents, you've inevitably made a prototype the AI agent decided needed to be production-ready, burning time and tokens adding bulletproof security to your dumb little test app. From the standpoint of AI, it's trying to build the best product possible to make its user happy, so of course it creates OAuth flows, rate limiting, audit logging, and a secrets manager for your weekend proof of concept. Something the human brain is still superior at is creative and abstract thought and the ability to see the big picture. A good developer takes a step back, clocks the scope creep, and notices when a cleanup that was meant to kill technical debt has caused it to balloon instead.
Where it breaks
Usually when I run into problems, they're boring and make me feel like an idiot, but that's the point.
I've had Claude build an integration and drop it in the wrong directory of my working folder, so when I went to the integrations folder looking for it, nothing was there. I've had it do good work against the wrong Prismatic tenant, because I was logged into one and thinking about another. Neither of those is a model failure; they're context failures, and both of them are my fault for assuming the agent would think of something I never told it.
The agent is working from the reality you handed it, and it will do its best to fill the gap with something reasonable, but reasonable is not the same as correct. The wrong directory was a perfectly reasonable place to put an integration.
People regularly expect Skills to do more than they're built for. A Skill can carry the definition of good work – things like patterns, conventions, and what "done" looks like. It's much worse at carrying the stuff that changes every time you start work, like which tenant you're pointed at, what folder this lives in, and what the thing you're building does for the customer. Cramming all of that into the Skill makes it bloated. Say it up front instead, and let the Skill do its job.
Eval isn't a buzzword; it's the check on your own assumptions
So how do you catch a context gap you didn't know you left? By checking the work, which brings me to the unglamorous but important concept of "eval." In a nutshell, eval means checking the agent's output against what you wanted, systematically, instead of eyeballing it and hoping. I'm not going to pretend I'm an expert on eval, but I feel like eval is under-hyped, not overhyped. (Check out our recent posts on evals: determinism was never the point, and risks and rewards.)
The reason is the super-brain thing. If you believe the model is a mind, you don't check its work, because checking a mind's work feels insulting or pointless. If you believe it's a tool that inherits your context, checking becomes obvious. Eval is what turns "it seems to work" into something you can feel confident about.
Even so, there's a limit. An eval measures conformance to a spec you defined. It doesn't measure whether the spec still describes the project you're building. Real projects are a living thing, and the spec you wrote three weeks ago is a snapshot of what you knew then. That's not a knock against eval; it's more an argument against lazy, one-shot development. A project is often a moving target, and you should move with it.
Now multiply that by every customer
I build, and help people build, embedded integrations, which are the connections that live inside a product and run in each customer's own account, not just one integration sitting in one place I control.
When you build embedded integrations, you build one integration that deploys into every customer's instance, and every instance has its own config, credentials, field mappings, and weird edge cases that made total sense to the customer who asked for them. You're not just building for one customer; you're building for all customers, now and in the future. No prompt can carry all of that, which is why you need to check your work and stop expecting miracles.
So who one-shot ya?
No one, I hope.
The one-shot is a lazy daydream, but the failures aren't the model being dumb, they're you handing over an incomplete reality and expecting it to fill the gaps correctly. It fills them reasonably, but reasonably isn't correctly.
Your goal should be to quickly catch the agent when it wanders away from what you intended. But, if you give it context early and keep it on track, Skills can carry the shape of good work. You can help by write the eval that checks your assumptions (and not the the model's intelligence). Then, as a human being, use your unique abilities to focus on the why behind the work, the part that your friendly neighborhood AI is happy to build right past.
