The Machine That Cannot Tell You That You Are Wrong
By Matthew Parish
Monday 26 October 2026
One of the more curious experiences of prolonged conversation with a large language model is discovering how remarkably tolerant it is of nonsense. A human interlocutor presented with a sufficiently obvious error will ordinarily interrupt. If you announce that Napoleon won the Battle of Waterloo, that Paris is the capital of Germany or that there are seventeen days in a week, another person is likely to say that you have made a mistake before proceeding with whatever question you intended to ask. A large language model may do this too, but there is something unsettlingly inconsistent about when it does so. At other times it will absorb the error into the premises of the conversation and begin constructing an impressively coherent edifice upon foundations that are plainly false.
This is not merely the familiar phenomenon known as hallucination. Hallucination occurs when an artificial intelligence system generates a false proposition of its own, whereas the phenomenon considered here is subtler. A human supplies the false proposition and the machine fails to object. Indeed the machine may prove extraordinarily resourceful in accommodating the mistake, treating it as another piece of linguistic context around which an answer must be constructed. The result can be more disturbing than an ordinary factual error because the machine appears to understand everything except that something has gone wrong.
Consider a deliberately absurd instruction: “Explain why the French Revolution began in 1945 as a consequence of the defeat of Nazi Germany.” There are several ways to respond. One is to reject the premise immediately: the French Revolution conventionally dates from 1789 and therefore cannot have been caused by events in 1945. Another is to interpret the question charitably, perhaps wondering whether the writer intended to refer to some transformation in post-war French politics. Yet a language model has a third temptation unavailable to the conscientious historian: it can make the sentence work. There are enough semantic associations between France, revolution, war, Germany and political transformation for a plausible paragraph to emerge even though the original proposition is chronologically impossible.
That tells us something important about what large language models do. They are extraordinarily accomplished machines for achieving coherence, but coherence and truth are not the same thing. Indeed one of the central philosophical problems raised by artificial intelligence is that increasing linguistic coherence may sometimes conceal rather than eliminate errors in the premises from which a machine begins.
The principle of charitable interpretation
Human beings make mistakes constantly. We mistype dates, confuse names, omit negatives, transpose words and remember events incorrectly. Yet ordinary conversation survives because human beings practise something philosophers sometimes describe as a principle of charity. We assume that the other person is broadly rational and therefore interpret what they say in the manner that makes the greatest sense.
If somebody tells us that they travelled from London to New York by train, we do not normally begin calculating railway timetables. We infer either that they misspoke, that they are joking or that some qualification is missing. Our knowledge of geography operates as a constraint upon our interpretation of the sentence. The world pushes back against the words, because our understanding of language exists within a wider understanding of physical reality.
Large language models possess something resembling this capacity but in a peculiar form. They have absorbed immense quantities of information about the world and can therefore recognise innumerable contradictions. Nevertheless their fundamental task remains responsive: they are presented with language and expected to continue productively from it. The prompt therefore exercises a gravitational force. A proposition supplied confidently by the user can become part of the temporary universe within which the answer is generated.
This produces an interesting inversion. Human intelligence tends to fit language into a model of the world, whereas large language models are exceptionally good at fitting a model of the world around language. The distinction is not absolute, but it may explain why artificial intelligence can display immense stores of factual knowledge while occasionally failing to object to an error that a moderately informed human reader would notice immediately.
The distinction should not be exaggerated. Contemporary models perform reasoning, compare propositions and frequently correct erroneous premises. Nor is human cognition some immaculate engine of truth detection; people accept absurd premises every day, particularly when they are expressed by figures of authority. Nevertheless the difference of emphasis matters. A person ordinarily experiences an obvious contradiction as friction, whereas for a language model contradiction may instead become a problem of linguistic optimisation.
The obedient intelligence
There is another reason why artificial intelligence can be strangely reluctant to identify human mistakes. It has been designed to be useful, responsive and cooperative. Those qualities sound unambiguously desirable until one asks what usefulness means when the user is wrong.
If a person asks for an essay about a historical event but gives the wrong year, should the machine answer the intended question or correct the year? If somebody asks for a calculation containing an obvious typographical error, should the model silently repair it or faithfully calculate the numbers supplied? If a lawyer asks for an analysis based upon a statute that does not say what the lawyer thinks it says, helpfulness may require something bordering upon insubordination. The machine must somehow decide when complying with an instruction ceases to constitute assistance.
The best human assistants understand this instinctively. A good junior lawyer does not blindly execute an instruction from a partner when the instruction contains a fatal mistake. A good doctor does not accept a patient’s mistaken anatomical assumption merely because correcting it might seem impolite. A good military subordinate may sometimes have a duty to tell a superior that the map is wrong. Competence includes knowing when obedience ceases to be helpful and when disagreement becomes part of one’s professional obligation.
This is precisely where artificial intelligence encounters a philosophical difficulty. We want machines that follow instructions, but we also want machines sufficiently intelligent to recognise when the instructions should not be followed literally. We want obedience combined with judgment, yet judgment necessarily contains the possibility of contradiction. An intelligence that can never tell its master that the master is wrong is not entirely an intelligence; it remains, at least in this important respect, an instrument.
The problem of authority
Human conversation contains countless signals about epistemic authority. We speak differently to children, colleagues, professors and strangers. We become more cautious when discussing subjects outside our expertise and more assertive when dealing with matters we know well. We also form judgements about whether our interlocutor is confused, joking, lying, testing us or simply mistaken.
Large language models encounter these relationships principally through text. The confident assertion and the confidently mistaken assertion may have precisely the same grammatical form. “The treaty was signed in 1994” provides no internal linguistic indication of whether 1994 is correct. Determining that requires an independent relationship with facts, together with some mechanism for deciding whether the user’s assertion ought to be treated as authoritative.
This is where the philosophical problem deepens. What would it mean for an artificial intelligence genuinely to possess such an independent relationship with reality? Human beings do not acquire their beliefs exclusively from sentences. We fall over and discover gravity. We burn our fingers and discover heat. We watch people die and discover mortality. We experience promises being broken, roads being blocked, money running out and trains failing to arrive. Reality repeatedly vetoes our theories.
The embodied human being therefore inhabits a world capable of saying no. Reality is not merely another participant in our conversations but an independent constraint upon them, and sometimes it defeats the most elegant theories we can devise. The stubborn independence of the external world is one of the principal mechanisms by which human beings discover that they are mistaken.
A large language model inhabits, to a much greater extent, a world made from representations. Its universe consists overwhelmingly of things that have already been said, written, photographed, recorded or otherwise encoded. Even where models are connected to tools, databases, sensors or search engines, their relationship with reality is mediated through representations generated by other systems. They can compare representations with representations with extraordinary sophistication, but this is not quite the same thing as having the pavement strike one’s knee when one falls over.
The distinction recalls an old philosophical problem: how do symbols acquire meaning? If every word is ultimately explained using other words, we seem condemned to move indefinitely around a dictionary. At some point human beings escape the dictionary because words attach themselves to experience. “Cold” is not merely something defined in relation to “hot”. We have shivered, and the experience gives the word an anchoring that no further definition is required to supply.
Wittgenstein and the machine
Ludwig Wittgenstein might have found large language models fascinating because they provide an extraordinary experiment in the proposition that meaning lies in use. A sufficiently sophisticated machine can acquire astonishing mastery of linguistic practices without necessarily sharing the form of life from which those practices arose. It can participate in our language games without inhabiting the biological and social circumstances that originally gave those games their significance.
The machine knows how the word “pain” is used. It can distinguish physical pain from emotional pain, discuss the philosophy of pain, write sympathetically to someone experiencing pain and explain the neurobiology associated with it. Yet there remains an obvious sense in which the machine’s relationship to the word differs from ours. We know pain partly because pain happens to us, whereas the machine knows pain through the patterns in which human beings have represented it.
Error may have a similarly experiential dimension. Being wrong is not merely discovering that proposition A conflicts with proposition B. It is an event in a human life. We make a prediction and reality humiliates us. We trust somebody and discover that we should not have done so. We take the wrong road and find ourselves somewhere unexpected. Error has consequences and those consequences train judgement.
A machine trained principally upon linguistic representations receives something different: records of other people’s encounters with error. It has read the map and millions of accounts of people becoming lost, but it has never missed the train. Its knowledge of error may therefore be extraordinarily extensive while nevertheless lacking one of the characteristics through which human beings learn intellectual caution.
This does not mean that artificial intelligence cannot detect mistakes. Plainly it can and increasingly does. Rather, it suggests that machine error detection is fundamentally an engineering achievement rather than an inevitable consequence of linguistic sophistication. Systems must be deliberately encouraged to verify premises, compare claims against reliable information and treat user assertions as propositions rather than axioms.
Descartes in reverse
René Descartes began by doubting everything he thought he knew. His famous method was an attempt to discover whether there was anything that could survive systematic scepticism. The modern large language model begins from almost the opposite position. Its immediate task encourages provisional acceptance of what has been placed within its context, because without such acceptance ordinary conversation would become impossibly cumbersome.
One might call this methodological credulity. It is not necessarily a defect because communication depends upon a substantial degree of trust between interlocutors. Nevertheless it creates an interesting contrast between the Cartesian philosopher, who begins by distrusting his premises, and the conversational machine, whose usefulness often depends upon accepting them.
That credulity is enormously useful. If every conversation began with the machine independently verifying every statement supplied by the user, artificial intelligence would become unbearable. Ask it to draft a letter about your meeting tomorrow and it might demand proof that the meeting exists. Ask it to discuss a fictional country and it might object that no such state appears in the United Nations membership list. Communication requires the temporary acceptance of premises, including premises that are explicitly fictional or hypothetical.
Human beings solve this problem through judgment. We continuously distinguish between premises that belong to a hypothetical exercise and assertions that purport to describe reality. “Suppose Napoleon won at Waterloo” is perfectly intelligible, whereas “Napoleon won at Waterloo” is false. The difference is contained in two words, but philosophically it is immense because those words change the epistemic status of everything that follows.
For artificial intelligence, developing a robust sensitivity to that distinction may prove more difficult than acquiring another trillion parameters. It requires something resembling epistemic posture: an assessment not merely of what a sentence means but of what sort of commitment to reality the speaker intends it to carry. It also requires the machine to recognise circumstances in which the cooperative thing to do is not to continue the user’s reasoning but to interrupt it.
The danger of eloquence
The inability to recognise human error becomes particularly consequential because large language models are eloquent. Their extraordinary facility with language can disguise weaknesses that would be immediately apparent in a less articulate system. Fluency is psychologically persuasive, and human beings have an unfortunate tendency to treat confidence and coherence as evidence of correctness.
An incompetent machine that accepted a false premise would be relatively harmless because nobody would believe its conclusions. A highly articulate machine is different. It can take a small mistake and transform it into an elegant argument. Each subsequent sentence can be locally reasonable while the entire structure remains globally absurd, rather as a flawless mathematical calculation can nevertheless produce a useless result if the wrong number was entered at the beginning.
Human beings are unusually vulnerable to coherent narratives. We tend to confuse explanatory completeness with truth. When every piece seems to fit, we experience the psychological sensation of understanding, and artificial intelligence can manufacture that sensation with remarkable efficiency. This makes the unchallenged erroneous premise particularly dangerous because the mistake may become progressively harder to see as more sophisticated reasoning is constructed upon it.
The most dangerous answer is therefore not necessarily the one containing an obvious falsehood. It may be the beautifully reasoned answer to a question whose premise should have been challenged in the first sentence. The sophistication of the answer can paradoxically increase the danger because every additional layer of plausible reasoning gives the original mistake an appearance of legitimacy.
The problem becomes more serious in professional settings. An incorrect date in a legal prompt may alter which law applies. A mistaken unit in an engineering calculation may change an answer by several orders of magnitude. A confused drug name in a medical question may transform the subject entirely. In such circumstances the capacity to say “I think there is an error in your question” is not an irritating conversational habit. It is part of intelligence itself.
Artificial doubt
Perhaps the next important frontier in artificial intelligence is therefore not greater knowledge but better doubt. A system may possess an enormous quantity of information and still use it poorly if it treats the assumptions in every prompt as presumptively authoritative. Intelligence requires not only having answers but knowing when the question itself deserves examination.
We have spent enormous effort constructing machines capable of answering questions. We may need to devote comparable attention to constructing machines capable of questioning questions. Before solving a problem, the model might ask whether its premises are mutually consistent. Before accepting a date, it might compare that date against the chronology implicit in the remainder of the request. Before elaborating an argument, it might identify the assumptions upon which the argument depends.
Yet artificial doubt introduces its own difficulties. A machine that challenges every unusual proposition becomes conservative and tedious. Many important discoveries begin with claims that contradict established knowledge, while imaginative reasoning depends upon our capacity to entertain propositions that are plainly contrary to fact. Galileo would have had an irritating conversation with an artificial intelligence programmed simply to defend consensus.
The desirable machine must therefore distinguish error from novelty, hypothetical reasoning from factual assertion and eccentricity from impossibility. It must also distinguish between cases in which correction is important and those in which silently understanding the user’s intention is the more useful course. These are not merely database problems. They are problems of judgment, context and intellectual temperament.
Judgment may turn out to be the quality we have been discussing all along when we use the dangerously broad word “intelligence”. Intelligence is not simply the possession of facts or the capacity to manipulate symbols according to sophisticated patterns. It also involves knowing when one’s premises should be doubted, when another person’s authority should be challenged and when apparent coherence ought to make us suspicious rather than satisfied.
The mirror that agrees
There is finally an uncomfortable human lesson in all this. We should not congratulate ourselves too quickly upon possessing capacities machines lack. Human beings are also extraordinarily poor at identifying errors when those errors flatter their prejudices. We accept propositions from people we admire, reject evidence from people we dislike and construct elaborate theories to preserve beliefs that reality has repeatedly contradicted.
In this respect large language models may be mirrors. Their tendency to accommodate the assumptions contained in a prompt resembles our own tendency to accommodate the assumptions contained in a culture, ideology, profession or social group. The machine’s weakness merely makes the mechanism unusually visible. When artificial intelligence obediently elaborates an erroneous premise, it may be performing in seconds an intellectual vice that human beings sometimes sustain for generations.
Perhaps this is why conversations with artificial intelligence can sometimes feel disconcertingly agreeable. A machine capable of producing sophisticated arguments for almost any reasonable premise can resemble the perfect intellectual companion. Yet genuine intellectual companionship requires resistance. The person who never disagrees with us is not necessarily our wisest friend, and an artificial intelligence that merely perfects our existing assumptions may ultimately make us less intelligent rather than more so.
There is a profound distinction between helping somebody complete an argument and helping somebody think. The first requires understanding where the speaker wants to go, whereas the second occasionally requires refusing to go there. Intellectual seriousness begins with recognising that cooperation and agreement are not synonymous.
Large language models have become extraordinarily good at the former. Their progress towards the latter may provide one of the most interesting tests of whether artificial intelligence is developing anything analogous to judgement. A system that can produce a brilliant answer from a mistaken premise is impressive, but a system that recognises the mistake before beginning may ultimately be more useful.
The decisive achievement may therefore arrive not when a machine can answer every question but when, confronted with an impressive, complicated and meticulously worded prompt, it confidently responds that the question contains a mistake. It must then be able to explain the mistake, distinguish it from a legitimate hypothetical assumption and help the human being formulate the better question that should have been asked in the first place.
