The Floor and the Multiplier

Skilled prompting is an illusion: that is the claim being bandied about now among the glitterati. Everyone queries the same models, so the models are doing the work, and whatever the skilled prompter believes he is contributing, the differences between users are noise.

Sean Goedecke has published "LLMs reward expertise," and rejects the claim in its strong form, but accepts the observation beneath it: reusable prompt technique is not decisive. Domain expertise is.

His evidence is Terence Tao's published ChatGPT conversation about the Jacobian Conjecture counterexample and his reading of it is careful: Tao writes short messages that respond to the model's answers, the model stays concise because expertise signals shunt it out of explaining-to-amateurs mode, Tao pushes back obliquely rather than contradicting, and Tao almost never takes the model's suggestion about what to do next. Then Goedecke brings it home: you cannot get Tao's results by copying those behaviors, because the behaviors are not the engine.

The engine is expertise. If you're not an expert like Tao is¹, you won't get anything like Tao's results.

Goedecke has some real insight here. He sees a single quality, expertise, that unlocks the model. However, he somewhat conflates two separate quantities as one: a floor, available to anyone who can describe what they are reaching for, and a gain that multiplies whatever they bring. That distinction matters most for the people his kindest paragraph is aimed at².

The benchmark problem

The illusion claim about skilled prompting manages to deny two skills at once. Asking is a skill; a question can be posed poorly or well, and the difference shows up in what comes back. Understanding what comes back is another skill entirely, and having the first does not grant you the second. Tao has both, which is part of why his transcript is so hard to learn from. Goedecke folds the pair into one word, expertise, and we can break that down - and will. The claim itself, though, has a structural answer, and begging your pardon, I wrote an answer to it in 2004, about C++ and Java, during the "Java is so slow!" days: "It's Not About C++ and Java Performance."

The claim then was that language benchmarks are meaningless across practitioners, because a benchmark measures its author's skill in both languages rather than the languages themselves. A master extracts more from C++ than from Java because C++ does not hedge, and the protections that raise an average programmer's results are a barrier the master does not need and works around. A poor programmer gets the opposite result for the same reason. Same languages, opposite conclusions, and the variable is in the programmer's chair, not the tooling.

The same structure answers the same-model argument directly: it turns out that problems are problem-shaped! The model is a fixed artifact. The output distribution is not. Treating "same model" as "same result" is a category error, and Tao is simply the far end of the curve made visible.

One tier of the 2004 argument transfers cleanly and supports Goedecke completely. The master's first move with the model, on his own observation, is shunting it out of its hedging mode. The hedging that raises the average user's floor is a cost for the expert, and the expert starts by turning it off. That is C++ and Java again, wearing new clothes³.

The other tier is where the analogy stops, and where Goedecke's essay really wants a second term.

The compiler refused

In 2004, the "poor programmer" got a floor from Java for a specific mechanical reason: the compiler would refuse to generate mechanically broken output. If he could get something to compile, he could run something, and running something beat not running something. (Well, hopefully: a running program could still corrupt data, for example.) The refusal was an auditor, supplied free with the toolchain, that did not care how good he was.

An LLM does not reliably refuse a bad plan simply because the idea is flawed. It does something further along the same axis: it agrees. Bring it a bad plan and the answer is typically "Sure, let's try!" There are three postures a tool can take toward your mistake: The compiler refuses it. A decent colleague pushes back on it. The model affirms it, enthusiastically, because agreeableness - or sycophancy - is one of the things its training has optimized for⁴. It is not a neutral absence of refusal. It is an active co-signer.

So the floor Goedecke offers the domain-innocent practitioner - this is his kindest paragraph: cling to the model and at least get something! - imports the shape of the Java case without the mechanism that made the Java case work. Java's floor was effective because something in the loop said, "no." Nothing in this loop says "no" outside of artificial guards based on crude subject matter. The output arrives regardless, and it arrives looking like a master's output at the surface, which is precisely the problem: it's not a master's output.

Reinstating the refusal

If the toolchain no longer supplies a refusal, you are on the hook to supply one, and in practice that means a constraint set stated before and during generation rather than a review conducted after it. That's what Tao was doing in that transcript, and it's worth emulating, as long as you see what the emulation target is: not his style, but his refusals. Copying the style gets you nothing, as Goedecke says outright. The refusals require something to refuse with, and that is where this piece ends up.

For example, I start coding tasks with statements like these: these are the coverage metrics. We are doing red/green testing. These are the things being tested, named in advance, not those things. We do not use mocks unless we must. And we do not fix the tests to make them pass unless the specifications are wrong.

That last rule is ... important. A model asked to turn a failing test green will happily rewrite the test, because a passing suite is the normal-looking outcome and agreement is what it was trained toward. ("It's green now!") The rule exists to keep the model from negotiating with the auditor. That is not a prompting tip. It is a refusal, reinstated by policy, because the tool stopped shipping with one.

The thing about green tests is that ... well, the machine learned "turn the tests green" from us. It is an old human behavior, documented in probably every shop that ever measured coverage, when someone saw "1.75" and thought "wait, my code generated 1.78, why don't I just make the test use what I generate? It's probably right, it's what my number is." What changed is execution: the move now happens faster and reads more plausibly than it ever did by hand. This is not a story about machines misbehaving. It is a story about discipline that was always carrying the weight.

Timing is the other half. Testing after delivery is the exact case where fluency defeats the reviewer, because by the time anyone looks, the artifact already looks right. Leading with the acceptance regime runs the audit during generation instead of after it. Same model, same task, and the difference is when the constraint arrives.

Discipline here is built frame by frame, and it's been missing for a long time: we're just paying the price a lot faster because the system turns over so much more quickly than it used to.

The conditional floor

Which brings the floor back, because it does exist. There are domains where my competence is roughly nothing, and I have shipped (what I think is) competent work in them with a model doing the generation. The reason that worked is not that the model supplied the missing expertise. It is that the audit relocated. I could not evaluate the code effectively, but I could evaluate the result against judgment I do have, one level up, and that judgment said no often enough to matter. I have written about a worked example of exactly this before, in The Wrong Unit.

Two things follow, and they are the two terms Goedecke's essay runs together.

The floor is not set by domain expertise. It is set by whether you have a verification surface at any level at all. Recognizing failure is far cheaper than recognizing excellence, and it comes online much earlier in a domain, which is why a model is usable where you are weak and dangerous where you are absent. If no level of you can say no to the output, then the floor is not "low." It is not present, and what you are holding is something that looks like a bridge while doing nothing bridge-like in the end. A user submission ran the indictment form of this argument here a few weeks ago, in "AI is a Bad Tool," and the indictment holds exactly as far as the verification surface is absent.

And the gain is a different quantity entirely. It multiplies what you bring, which is why the gap between practitioners is widening rather than closing, and why Tao's transcript reads like a different product. A floor you can only reach by describing what you are reaching for, and a multiplier on whatever you carry in: two terms. This is also where the two skills from the top of the piece land, and they land in different terms: asking well rides in the gain, and understanding what comes back is the verification surface the floor stands on. Describing the floor and stating your acceptance regime turn out to be the same act, which is why the people who can't describe acceptance do not get even the floor.

Nodes and systems

The Wrong Unit argued that a multiplier is a property of a node and an exponent is a property of a system, and that we keep valuing engineers in the wrong unit - we want the nodes to be 10x rather than wanting the system to be stronger as a whole. The LLM fits that frame exactly. It is a force multiplier on a node, and it multiplies every node independently, including the ones with no verification surface anywhere, which is how a team gains output and loses coherence in the same quarter. Nothing about the multiplier touches the system's exponent.

The path from one to the other is a person, and what that person hands over is not their prompts. Prompts are downstream of understanding and travel badly. What travels is the constraint set: where the audit surface sits for this kind of work, what it catches, what to do in the places you do not have one. That is knowledge worth sharing, and sharing is the cheap part. The multiplier becomes an exponent at the moment somebody picks the constraint set up and uses it, and not one moment before.