I'm working on integrating a full prototyping tool into ChatPRD. This prototyping tool follows a 3 step agent pattern:

a planner designs the architecture a coder writes the files and a visual reviewer grades the result.

Every one of those three steps is a model we get to choose. So I ran a real bake-off — 82 sandbox builds across planners, coders, themes, and product types — to answer one question: is “smarter planning plus cheaper coding” actually a good trade?

TL;DR. Despite my initial love of GLM-5.2, cheaper coding is a trap. Every budget coder I tried either fell over or burned 2–6× the tokens.

GPT‑5.5 is the clear winning coder: 10/10 functional, zero hard failures, and the leanest token bill (~$0.50 a build). And as expected: the planner, not the coder, turned out to be the most interesting lever, but only when the product demanded complexity.

How we ran our evals

For each build, we ran the full pipeline in an isolated cloud sandbox: external planning → coding → static checks (typecheck/lint/build) → browser validation (it actually clicks the primary interaction and screenshots desktop + mobile) → visual review.

I defined a test set of prototypes for the experiment:

support inbox habit tracker content publishing inventory ops incident response

And execute these in two themes: a grayscale wireframe and a polished shadcn app.

Of course, to grade 82 builds, I didn't eyeball them by hand.

Claude Code ran a fan-out of inspector agents. One per build, each run reviewed the generated code and looked at every screenshot. Then, a second adversarial verifier tried to refute each verdict against the source code.

This worked pretty well: the verifier caught the visual reviewer flagging “missing” features that were actually there, just hidden behind a default state the screenshot never reached.

What “good” looks like

Before the scores, I want to show you the output! These are real, unedited builds. On the left, a polished inventory console; on the right, a wireframe editorial board — same engine, different theme and product.