Over the course of a few days, I had conversations with two clients who had very different opinions about the use of AI-generated prototypes. The first left me genuinely impressed by how quickly AI could turn a simple description into working software — the practice often called “vibe coding.” The second brought back memories of Rapid Application Development, code generators, regression-testing headaches, and legacy systems where the code had become the only reliable record of what the system actually did.
That contrast got me thinking. AI-generated prototypes can give stakeholders something tangible to react to almost immediately, and that’s a powerful tool for a business analyst. But once the prototype includes working code—not just a sketch or clickable mockup—the line between exploring a solution and building one starts to blur. Is this using AI to discover requirements faster, or are we skipping the work of defining and preserving how the solution is supposed to behave?
My First Conversation
My husband Bob and I had the chance to go to dinner with a client the other evening. This client believes business analysts should spend less time writing requirements and more time building rapid prototypes. Instead of defining requirements, he maintains that you should describe what you want, ask AI to build it, put it in front of stakeholders, and refine it based on their reactions.
He showed us an example that he built with GitHub CoPilot, and I could see the appeal. It was tangible, it worked, and it came together quickly. Stakeholders could react to something on the screen instead of trying to imagine a solution from words on a page. They could describe a change they wanted and see it on the screen in a matter of seconds. Honestly, it was really cool.
Steven Jobs once famously said, “People don’t know what they want until you show it to them.” As business analysts, we’ve used prototypes for decades because customers often struggle to articulate what they want. But vibe coding goes further than a sketch or a clickable mockup. AI generates the underlying code and keeps revising it as we refine our prompts.
Then Déjà Vu Kicked In
Soon afterward, another client and I were talking about AI-generated software. I related my earlier conversation to him, and we both admitted that we were a little skeptical about this. Why? Well, it boiled down to this key question: Haven’t we tried this before?
AI is new, but the concept of rapid prototyping is not. In fact, at one point in time, there was a popular application development methodology called Rapid Application Development (RAD). For years now, code generators and low-code tools have all promised to reduce the time between an idea and working software. They bring real benefits, but they also taught us a lesson worth remembering:
There is still value in defining what we are building (or have built).
AI can produce something impressive in a very short time. But here are some key questions we need to ask ourselves before we decide that a working prototype is all we need.
QUESTION #1
Is it production-worthy?
“It works” and “we can rely on it to run our business” are not the same standard. The code still needs to be secure, maintainable, and able to work within the larger environment. And from a business analysis perspective, we still have to ask whether the solution works the way the business actually needs it to work.
QUESTION #2
Have we thoroughly tested it?
Here is where things get tricky. A stakeholder asks to move a field. Someone else requests a different calculation. Then the team adds an exception, changes a rule, and revises the navigation. Each change looks fine in the area we are examining—but what happened to everything that worked three versions ago?
When developers change code, we don’t assume that everything else still works just because the latest change works. That’s why we regression test. But vibe coding moves so quickly that it’s easy to keep our eyes on the newest request and skip the broader check. We tell ourselves we’ll test later…and we all know how that goes. Later may never come.
QUESTION #3
What are we testing against?
Even if we remember to test, we still face a basic problem: How do we define our test cases?
The latest prototype shows us how the solution currently behaves. It does not necessarily tell us whether that behavior is right. To test it properly, we need to know what the solution should and should not do and what makes a feature acceptable.
Huh. Sounds kinda like requirements and acceptance criteria to me.
A prototype shows us how the solution currently behaves. Requirements tell us how it is supposed to behave. Those are not the same thing.
I’m not arguing for a lengthy requirements document before anyone writes code. Requirements can be lightweight and can evolve as we learn. Many of you have seen my very messy scribbled requirements and models on a Miro board. But if there’s no agreed, testable definition of expected behavior outside the prototype itself, we have nothing reliable to test against.
QUESTION #4
Where’s the source of truth?
There’s another reason this makes me nervous: I’ve lived through what happens when the working system becomes the only real source of truth.
Anyone who has worked with legacy systems knows this story. The original requirements disappeared. The documentation became outdated. The people who understood why certain decisions were made moved on. Eventually, if you wanted to know what the system was supposed to do, the answer was: look at the code.
Except the code didn’t tell us what the system was supposed to do. It told us what the system currently did, including years of patches, workarounds, and “fixes”.
I’ve participated in enormous reverse-engineering efforts just to understand existing systems. We had to read through the code, trying to reconstruct the logic inside the system so we could safely make a change. Those systems were fragile because changing one thing could break something somewhere else – and a lot of the time there was no way to know that until it happened.
Wouldn’t it be ironic if we used AI to create a new generation of systems with the same problem? The code works – for now! – but we have no real insight into what it’s doing or how. I don’t want to be the person who comes along in 10 years who has to figure out how to make a change. Been there, done that!
It’s Not An Either/Or
I’m not suggesting that we shouldn’t develop AI-generated prototypes. They can be a great elicitation tool. They help stakeholders react, uncover missing needs, challenge assumptions, and refine their thinking. What I’m suggesting is that we use the prototype to discover the requirements, but not to replace them.
As illustrated in our latest infographic, requirements remain at the heart of our work. As decisions are made, the important ones need to be captured somewhere outside the code. That might mean acceptance criteria, business rules, process models, data requirements, examples, or scenarios. The documentation doesn’t need to be heavy. It just needs to capture enough of what we learn to test the solution, change it safely, and understand it years from now. The prototype becomes part of the requirements, but not the sum total of them.
Requirements at the Heart Infographic Download
Requirements, analysis models, acceptance criteria, and test cases work together to define and verify a solution. Each serves a different purpose, but they are closely connected. Requirements are at the heart of that work.
If you’re thinking that a prototype will make requirements unnecessary for something you’re working on, stop and ask yourself these questions: What have we learned about the user needs? Where have we recorded it? And what will we test against? If the only answer is “the prototype,” I suggest you think again.
Ready to hone your prototyping skills? Want to dig into techniques for representing functional requirements to support those prototypes? Check out our Use Case Modeling and Solution Requirements class. You’ll gain practical, hands-on experience developing functional requirements using a number of common techniques.
Kathy Claycomb
Managing Partner, Lead Expert
Kathy Claycomb brings over 35 years of experience to the classroom. She has participated in all phases of solution development using everything from agile to waterfall methodologies (and quite a few in between). Before joining B2T, her career spanned roles from application developer to Senior Director of Services at various organizations. Kathy has broad industry background including transportation, manufacturing, insurance, energy, healthcare, and banking.
Kathy’s first love is teaching, and throughout her career she has always managed to spend a portion of her time instructing. She has an engaging, highly interactive teaching style that ensures students leave the course with a thorough grasp of the material. Her students consistently praise her teaching abilities and her talent for drawing on her personal experience to enhance their learning.
Kathy served as the Technical Editor for Business Analysis for Dummies, 2nd Edition.