Why judging what a machine produced always means judging what it was asked to do.
"You end up arguing about the tool. It was never really the tool."
If you do skilled work, you have probably had some version of this happen already. Someone hands you a result that looks finished, generated in minutes, and expects the hard part of your job to already be done. Or the opposite: someone who has watched AI fail somewhere they genuinely understand assumes it will fail in your domain too, which they probably know a lot less about. Either way, you end up arguing about the tool, when the actual problem was never really the tool at all.
I find myself increasingly caught up in this question: whether AI actually deserves a place in serious professional work, or whether the people arguing about it are really arguing about something else entirely. Some of that, I think, is simply how we talk about it. We say it thinks, it decided, it is getting smarter, the same language we would use for a colleague, not a tool. Underneath that language is a word most people would not actually say out loud: we talk about it, and treat it, as though it were sentient. I do this myself. I have felt the pull of outputs that feel lifelike enough to credit with intentions of their own.
The talk gets struck through. What was actually there does not change.
But intentions are exactly what these systems do not have. Whatever it produced, it produced because of what it was given, not because it wanted anything. Taking the sentience out of the conversation is not about being cold toward the technology. It is the engineer's move: strip away the language that treats the machine as though it has a mind of its own, get back to what the machine actually is and what it was actually asked to do, and only then can the argument people are actually having be had properly. That is where I want to put this instead.
Some of this is not really a matter of opinion anymore. A few years ago there was no AI search category to speak of, nothing to extrapolate a trend from. Now it shows up everywhere you would care to look for it: in how people search, in the briefs you are handed, in the output of the suppliers and service providers you rely on, even in the correspondence that lands on your desk in the middle of a dispute. Whether you personally use these tools or not, the people and processes around you increasingly do, and there is no selective way to exclude that. You end up somewhere between not knowing it is there and not getting what you actually wanted. The divide over adoption is real. Whether the divide changes anything is a separate question, and probably a smaller one than it looks.
The question that actually matters is not whether the tool is good enough. It is what the tool was asked to do, and by whom. A result you can point at and dismiss did not arrive from nowhere. It traces back to an instruction, and the instruction traces back to a person. You cannot fairly judge what came out without judging what went in.
That is not, on reflection, a new problem, though it takes a slightly different shape now than it once did. Put plainly, in engineering terms rather than in the language everyone reaches for when the subject is AI: the machine can extrapolate a long way past the literal words of an instruction, further than a simpler, more deterministic tool ever could, and that reach is exactly what makes it useful. Ask it something precise, demanding and well understood, and the reach becomes a genuine asset: it goes somewhere you did not explicitly point it, and gets there because you gave it enough to work from. Ask it something vague, and the same reach works against you. It does not come back flat or obviously incomplete the way an under specified instruction to a simpler tool would. It comes back confident, fully formed, and wrong, filling the gap you left with something that sounds like an answer. That is the thing people mean when they talk about a tool hallucinating, and it is the reason nearly every one of these systems now carries some version of a warning to check what it gives you. The tool did not change between the two cases. What you gave it to work with did.
That is usually read as a warning: do not trust a machine past what you gave it. I think it is closer to the opposite. If the result is bounded only by the ambition behind the instruction, the ceiling on what is achievable was never really the machine's to set. A more exact, more demanding instruction does not just avoid a poor result. It reaches somewhere the tool would not have gone by default. Responsibility and opportunity turn out to be the same fact, looked at from two directions.
You can watch this argument proved in hardware rather than in prose. NASA's Voyager 1 launched on the 5th of September 1977. Later this year, on the 18th of November, it will become the first human made object to reach one light day from Earth: a distance of just over 25.9 billion kilometres, so far that a radio signal leaving Earth now would take a full day to arrive. Voyager has no initiative of its own either. It has spent nearly fifty years doing exactly, and only, what it was built and instructed to do. What is remarkable is not that it exceeded those instructions. It is that the instructions were so precise, and the build so exact, that fifty years later the spacecraft is still faithfully carrying them out, from further away than anything humans have ever sent. The ceiling was never the machine's. It was set entirely by what its builders were willing to ask of it in 1977, and it turned out to be an extraordinarily high one.
None of this is new, in other words. It was described, in its essentials, in 1843, by Ada Lovelace, writing about Charles Babbage's Analytical Engine, a machine that never once ran a calculation in her lifetime. Her observation is usually quoted as a limitation.
"The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform."Ada Lovelace, 1843
I did not arrive at this argument by reading her. I arrived at it from the work in front of me, the tools available, and what I could see they were and were not capable of. It was only afterwards that I found she had already said it, close to two centuries before a machine existed that could prove her right.
That ordering matters, because the same principle applies to the doubt itself, not only to the tool. A person who has not genuinely spent time with these tools, who has not tried to specify something demanding and watched what comes back, is not yet in a position to say with confidence what they are or are not capable of. I do not think that makes anyone foolish. I think it means the opinion, whatever it turns out to be, is not fully informed yet. That is not a reason to feel behind. It is an invitation, the same one engineers have always had in front of them when a new set of tools arrives: find out what you can actually ask for, and then ask for more than you assumed you could.
The machine was never going to set its own ceiling. It never has, and Lovelace understood that before the first one was ever built. The only real question left is what you are willing to ask of it.
This page took two sessions rather than one: a first attempt at the concept that was set aside rather than pushed through on an uncertain read, then rebuilt from the actual reasoning behind it. None of that saved time. Writing this alone, facts half checked, would have taken a fraction of it. What went into it instead bought something a solo draft would not have: the Voyager and Lovelace details checked against primary sources, wording pushed back on until it actually sounded like me rather than a competent guess at me, an argument rebuilt rather than patched when it stopped convincing its own author. The piece above closes by asking what you are willing to ask of a tool once you know the ceiling was never really its own to set. This page is one answer to that, tested on itself rather than only argued for: not a faster version of something I could have written alone, but a version neither of us would have reached alone in the same time. If it reads as though it arrived fully formed, that was not the process, and you are welcome to ask about the parts that did not.
Start a Conversation