The Sub-Word Illusion: Tokenization, Transformers, and the Mechanics of AI Hallucination

When users interact with Large Language Models (LLMs), it feels like chatting with a deeply knowledgeable, fluent entity. Yet, underneath that veneer of human-like reasoning lies a purely statistical engine. Sometimes, this engine works perfectly. Other times, it confidently invents facts—a phenomenon known as AI hallucination.

To truly understand why a Transformer hallucinates, we have to look past the smooth sentences and analyze how it actually views the world. The roots of AI hallucination are deeply entangled with the model's fundamental architecture: the Transformer and the process of tokenization.

1. Tokenization: The Broken Mirror of AI Perception

A common misconception is that text-based AIs read and write in words or letters. They do not. Instead, they process text using tokens—chunks of characters processed via algorithms like Byte-Pair Encoding (BPE).

Because the model views the world through these fragmented pieces, it introduces a semantic disconnect:

• Sub-Word Fragmentation: The word "indivisible" might be broken into ["in", "div", "is", "ible"]. The token ["in"] has its own distinct statistical weight that can conflict with the global meaning of the entire word.
• The Spelling Blindspot: Since a Transformer only sees token IDs (integers), it does not natively know that the token for "apple" contains the letter "p". This is why early models famously struggled with simple riddles like, "How many letters are in the word strawberry?"
• Loss of Granularity: When rare words, medical terminology, or complex mathematical formulas are forcefully broken into obscure, fragmented tokens, the semantic meaning degrades. The model is forced to predict the next word based on a fragmented and messy representation, significantly raising the mathematical probability of a hallucination.

2. The Transformer Architecture: Attending to the Wrong Things

Once text is tokenized, it enters the Transformer architecture. Transformers rely entirely on a mechanism called Self-Attention. This mechanism calculates mathematical relationships between every token in a sentence to determine context.

$$Attention(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$

While self-attention allows the model to understand complex context across massive paragraphs, it possesses no inherent concept of "truth." It only understands probabilistic alignment.

[Input Tokens] ──> [Self-Attention Layer] ──> [Probability Distribution] ──> [Most Likely Next Token]
                         │
                         └──> Prioritizes grammatical flow & linguistic plausibility over historical facts.

The Conflict of Two Memories

Recent machine learning research reveals that a Transformer operates using two distinct, competing memory systems:

1. Parametric Memory: The static facts and patterns baked directly into the model's weights during training.
2. Working Memory: The active context provided by the user in the prompt.

When a prompt is ambiguous or queries a rare fact, the Transformer's hidden mathematical states drift into spaces where no concrete data exists. Instead of stopping, the self-attention mechanism forces a convergence anyway. It outputs the most syntactically pleasing token sequence, effectively choosing linguistic plausibility over factual reality. The model isn't "lying"; it is simply doing math correctly on flawed associations.

3. Visual Hallucinations: Expanding the Paradigm

The term "hallucination" has taken on a more constructive meaning in multimodal AI architectures. In frameworks like VALHALLA (Visual Hallucination for Machine Translation), researchers intentionally use an autoregressive transformer to hallucinate discrete visual representations (image tokens) from text inputs.

If a model is translating an under-resourced language or a sentence with an ambiguous homonym (e.g., the animal "bat" vs. a baseball "bat"), the hallucination transformer generates an internal mental image token. This visual context grounds the translation transformer, preventing textual errors by using visual imagination to stabilize the output.

4. Closing the Factuality Gap

Mitigating hallucinations requires addressing both the inputs (tokens) and the underlying cognition (the Transformer). The industry is attacking this through several engineering shifts:

• Advanced Tokenization: Building semantic-aware tokenizers that avoid breaking critical keywords, numbers, or entities into unprompted fragments.
• Retrieval-Augmented Generation (RAG): Constraining the Transformer's working memory by injecting verified, external text documents directly into the prompt, reducing its reliance on spotty parametric memory.
• Attractor Geometry Detection: Utilizing real-time geometry monitoring of the model's hidden layers to detect when an AI's internal state is drifting into an unmemorized "hallucination basin," allowing the system to flag an error before it finishes generating the response.

Ultimately, hallucinations are not software bugs that can be easily patched out. They are an inescapable characteristic of probabilistic modeling. By refining how data is tokenized and guarding how Transformers query their own memory, developers are steadily turning fluent fabricators into reliable intellectual partners.