The Engine Beneath the Oracle: How Large Language Models Actually Work

Friday 14 August 2026

A large language model is an extraordinarily elaborate prediction machine. That description is entirely accurate — but, taken alone, it is also profoundly misleading. It is rather like describing a human being as a collection of chemical reactions. The statement is true, yet it conceals almost everything that makes the subject interesting.

When a large language model writes an essay, translates a document, explains a philosophical argument or diagnoses an error in computer code, it is performing the same fundamental operation repeatedly: given everything written so far, it calculates what fragment of language is most likely to come next. It chooses one such fragment, adds it to the text and performs the calculation again. From this apparently modest procedure there emerges something that can imitate reasoning, style, memory, explanation and occasionally even originality.

The first misconception to remove is that a large language model is merely a neural network connected to an enormous database. The phrase contains two related ideas that must be separated. A model is trained using an immense collection of texts, but in its basic form it does not retain that collection as a searchable library. Instead, patterns extracted from the texts are compressed into billions — sometimes hundreds of billions — of numerical settings called parameters. Some contemporary systems can additionally search the internet, consult documents or query databases. Those are compound systems in which a language model is being assisted by external sources. The model itself remains a mathematical structure rather than an encyclopaedia with pages.

From words to numbers

Computers cannot directly manipulate meaning. They manipulate numbers. Therefore, before a model can process language, the text must be converted into a numerical form.

The first stage is called tokenisation. A token may be a whole word, part of a word, a punctuation mark or occasionally a single character. A sentence such as “The government reconsidered its policy” might be divided into units resembling “The”, “ government”, “ reconsider”, “ed”, “ its” and “policy”. The precise divisions depend upon the system. Frequently used words tend to receive tokens of their own, while rare or complicated words are assembled from smaller pieces.

Each token is assigned an identifying number. Yet an identification number alone says nothing about meaning — just as a library catalogue number tells us where a book is kept but not what it contains. The model therefore converts each token into an embedding: a long list of numbers representing the token’s relationships with other tokens.

One may imagine an enormous conceptual space possessing hundreds or thousands of dimensions. In this space, words used in similar contexts occupy related regions. “King” and “queen” will have certain relationships; so will “Paris” and “France”, “contract” and “obligation” or “violin” and “orchestra”. These relationships were not entered manually by programmers. They emerged because the model repeatedly encountered the relevant expressions in related linguistic surroundings.

This is already more than a conventional database does. A database stores entries and returns them when asked. An embedding represents similarity, association and context in a continuous mathematical landscape.

The model must also know where each token occurs in a passage. “The dog bit the man” is not equivalent to “The man bit the dog”, although both sentences contain the same words. Information about position is therefore added to each token’s numerical representation.

The artificial neural network

The resulting numbers pass through an artificial neural network. The adjective neural is an analogy rather than a claim that the machine recreates a biological brain. An artificial neuron is essentially a small mathematical operation. It receives several numbers, assigns different importance to them, combines them and passes a transformed number onwards.

A network contains vast numbers of these operations arranged in layers. Information enters one layer, is transformed and passes to the next. Early layers may detect relatively simple linguistic relationships. Later ones can combine those relationships into more abstract representations — grammatical function, reference, tone, subject matter, analogy or the structure of an argument.

The numerical settings controlling all these transformations are the model’s parameters. They are commonly called weights because they determine how heavily one signal influences another. A model containing many billions of parameters consequently contains many billions of adjustable numerical relationships.

No individual parameter means “Napoleon”, “irony” or “English contract law”. Knowledge is distributed across the network. A fact or concept is generally represented by the interaction of many parameters, while any one parameter may contribute to many different concepts. This distributed character helps models generalise, but it also makes them difficult to inspect. There is no single drawer inside the machine that can be opened to discover everything it believes about France.

Attention — the decisive invention

Most modern large language models use a neural-network design called the Transformer, introduced in a celebrated 2017 paper entitled ⁠Attention Is All You Need. Its crucial mechanism is known as attention.

Attention allows every token being processed to examine other relevant tokens in the passage and determine how much weight to give them. Consider the sentence:

The ambassador put the report in the cabinet because it was secure.

To interpret “it”, the model must determine whether the expression refers to the report or the cabinet. The surrounding words provide evidence. Attention mechanisms calculate relationships between the numerical representation of “it” and those of the other tokens. Different attention patterns may examine different kinds of relationship — grammatical dependence, identity, temporal order or subject matter.

The mechanism relies upon three sets of numbers conventionally called queries, keys and values. The terminology sounds more mysterious than the process. Each token effectively asks a question — the query — of the surrounding tokens. Each surrounding token supplies a description of what it may be relevant to — the key — together with the information it can contribute — the value. The model compares queries with keys, calculates relevance scores and constructs a weighted mixture of the appropriate values.

A Transformer usually contains several attention mechanisms operating simultaneously. These are called attention heads. One head may be especially responsive to subjects and verbs, another to pronouns and their antecedents and another to connections across distant parts of an argument. It would be too simple to suppose that every head has one stable, human-readable duty — nevertheless, different heads do learn distinguishable patterns.

After attention, the information passes through another component called a feed-forward network. This performs further mathematical transformations upon each token’s representation. Attention combines information across the passage; the feed-forward network processes the combined information. These operations are repeated through many layers. Connections that carry information around individual transformations help preserve earlier signals and stabilise the calculation.

By the time the numbers reach the final layer, each token has acquired a context-sensitive representation. “Bank” in a discussion of monetary policy will be represented differently from “bank” beside a river. Meaning, in this system, is not assigned to a word once and for all. It is continually reconstructed from context.

How the model is trained

At the beginning of training, the model knows nothing. Its parameters are set to essentially arbitrary values and its predictions are correspondingly useless.

It is then shown sequences of text with part of each sequence concealed. Its task is usually to predict the next token. Given “The capital of France is”, an untrained model may assign random probabilities to thousands of possible continuations. The correct continuation in the training text provides a target against which the prediction can be measured.

The distance between the model’s prediction and the target is represented by a number called the loss. A mathematical procedure called backpropagation works backwards through the network to calculate how each parameter contributed to the error. An optimisation algorithm then adjusts the parameters by tiny amounts in directions expected to reduce future error. This process — prediction, measurement and adjustment — is repeated across immense quantities of text.

The model is not normally given an explicit list of grammatical rules. It learns grammar because models that recognise grammatical regularities predict text more accurately. Nor is it separately instructed to construct categories such as countries, legal systems, emotions or literary genres. Useful internal representations of these things arise because they assist prediction.

Training therefore compresses regularities in human language into the parameters. The model encounters that Charles Dickens wrote Bleak House in many contexts. Gradually its numerical configuration comes to favour that association. Yet it need not preserve a particular sentence from which the fact was learned. The fact becomes part of a distributed statistical structure.

This is why the database analogy is inadequate. A database stores “Charles Dickens — Bleak House” as a definite entry. A language model acquires a network of associations among Dickens, Victorian Britain, novels, characters, style and countless neighbouring ideas. This structure is flexible enough to answer questions phrased in ways that never appeared in the training material. It is also imprecise enough sometimes to connect the wrong author with the wrong book.

Research has found that performance tends to improve predictably as model size, training data and computational expenditure increase — an empirical phenomenon known as a scaling law. Yet the three resources must be balanced; merely adding parameters without adequate data or training computation is inefficient. The influential work on ⁠scaling laws for neural language models helped explain why larger systems displayed capabilities smaller ones conspicuously lacked.

Pre-training and education after training

The initial process is known as pre-training. It gives the model broad linguistic competence, factual associations and patterns resembling reasoning. It does not necessarily make the model a useful conversational partner. Raw internet text includes contradictions, hostility, falsehoods, incomplete passages and exchanges in which people do not answer questions directly.

The model therefore normally undergoes post-training. In supervised fine-tuning, it is shown examples of desirable answers to instructions. Further training may use human judgements — or judgements generated with human oversight — to teach the system which of several answers is more accurate, useful and safe.

This stage changes the model’s behaviour rather than supplying all its underlying knowledge. Pre-training is analogous to reading a vast and disorderly library; post-training is closer to learning how to conduct oneself in a seminar. It teaches the model to recognise requests, follow formats, acknowledge uncertainty and avoid certain harmful responses.

A model can also be fine-tuned for a specialised purpose — medicine, law, programming or scientific analysis. This does not necessarily require retraining the entire system. Various techniques can make targeted adjustments to a comparatively small part of its numerical structure.

What happens when the model answers

When a user submits a question, the question is tokenised and placed inside the model’s context window — the quantity of text it can consider during that particular exchange. System instructions, prior conversation and retrieved documents may also be included.

The tokens pass through the Transformer layers. At the end, the model produces a probability distribution over its vocabulary. It might calculate, for example, that after “The Treaty of Versailles was signed in” the token representing “1919” is highly probable, while “London”, “Tuesday” and “elephant” are much less probable.

A token is then selected. Always choosing the single most probable token can make prose rigid and repetitive. Systems therefore often use controlled sampling, allowing plausible alternatives some chance of selection. A setting commonly called temperature influences this variation: lower values usually make output more predictable, while higher values permit more diversity and also more risk of incoherence.

The selected token is appended to the context and the entire prediction process continues. The model does not ordinarily compose a finished paragraph in one indivisible act. It generates the passage token by token, each new choice becoming part of the evidence governing the next.

This sequential procedure explains both the fluency and fragility of model output. Once a model begins down a particular line of argument, its own words make the continuation of that argument increasingly probable. A mistaken early assumption can therefore propagate through an otherwise impressive chain of prose.

Does prediction amount to thought?

The claim that a language model “merely predicts the next word” is both true and rhetorically deceptive. The next item in a difficult sequence cannot always be predicted without representing the underlying structure.

To complete a legal analysis, the model may need to distinguish a rule from an exception and apply both to facts. To continue a mathematical proof, it may need to represent the relationship between earlier propositions. To translate a paragraph, it must preserve meaning while altering grammar. Training solely for prediction can therefore produce internal processes that perform operations recognisable as abstraction, comparison, planning and inference.

The research on GPT-3 demonstrated that sufficiently large autoregressive models could perform unfamiliar tasks from instructions or a handful of examples without further changes to their parameters — a capacity described as few-shot learning. The authors nevertheless documented important failures alongside those abilities. Their results remain a useful warning against equating wide competence with universal reliability. ⁠The original GPT-3 study established both sides of this proposition.

Whether these operations constitute “thought” is partly a philosophical question. Mechanically, one can say that the model constructs and transforms numerical representations in ways that often correspond to features of the world. One need not attribute consciousness to it in order to recognise that this is more sophisticated than retrieving stored phrases.

External databases and retrieval

Some language-model systems do possess access to an external database or search engine. This arrangement is generally called retrieval-augmented generation, or RAG.

When asked a question, such a system first searches a collection of documents for relevant passages. Those passages are inserted into the model’s context and the model writes its answer using them. The external collection is sometimes described as non-parametric memory, in contrast with the diffuse knowledge stored in the model’s parameters. The foundational ⁠RAG paper made precisely this distinction.

The advantage is substantial. A database can be updated without retraining the language model. It can supply private, specialised or recent information. It can also provide identifiable sources against which an answer may be checked.

Yet retrieval does not guarantee truth. The search mechanism may retrieve an irrelevant passage. The underlying document may be wrong. The model may misunderstand good evidence or blend it with unreliable associations acquired during training. RAG gives the model books to consult — it does not ensure that it reads them wisely.

Tool-using systems extend this principle further. A model may call a calculator, execute software, inspect a file or search the web. The language model acts as a controller: it interprets the request, decides which tool may help, constructs an instruction for that tool and incorporates the result into its answer. What appears to the user as one intelligence may consequently be an arrangement of several components.

Why models hallucinate

A language model is trained principally to produce a probable continuation, not to maintain an incorruptible distinction between truth and falsehood. Ordinarily these goals overlap — truthful statements recur systematically in serious texts. Sometimes they diverge.

If asked about a plausible but nonexistent court judgment, the model may reproduce the statistical shape of legal citation and invent a convincing name, date and holding. It has learned what an answer of that kind should look like. Unless training, retrieval or verification mechanisms intervene, the model has no internal librarian obliged to stop the sentence and announce that the requested book is absent.

Hallucination is therefore not an accidental ornament attached to an otherwise conventional database. It follows from the generative design. The same capacity that enables a model to create a novel explanation also permits it to create a novel falsehood.

Other weaknesses have the same origin. Training data contain social prejudices and uneven coverage. Rare facts are represented less reliably than common ones. Long chains of reasoning create multiple opportunities for error. A model can be strongly influenced by misleading wording in the prompt and its apparent confidence is not a calibrated measure of truth.

An immense compression of human expression

The most useful final picture is neither that of a conscious electronic person nor that of an automated filing cabinet. A large language model is a layered mathematical system trained to compress patterns in vast bodies of language and to reconstruct plausible continuations from those patterns.

Its parameters are not pages of an encyclopaedia. They are numerical dispositions — tendencies for one linguistic or conceptual feature to activate another. Attention allows the model to determine which parts of the current text matter to each other. Training adjusts billions of parameters so that accurate prediction becomes progressively more likely. Post-training turns the raw predictor into an assistant. Retrieval and external tools supplement its compressed knowledge with explicit information and exact computation.

Out of this machinery comes prose because human thought has left patterns in prose. Arguments, explanations, analogies and deductions recur in language. A machine trained with sufficient scale and precision to predict language must acquire some representation of those structures — not necessarily as human beings represent them and certainly not with all the safeguards human judgement ought to supply.

That is the achievement and the danger. The large language model is not an oracle containing the world. It is an engine constructed from the statistical remains of how humanity has described the world — capable of recombining those remains with extraordinary subtlety, yet forever requiring us to distinguish fluency from knowledge, simulation from certainty and a convincing answer from a true one.