LLM Prototype

This folder holds a tiny, complete language model. It exists so you can see how text generation works from the inside.

The model has only ~150k parameters. Real models have billions. The goal is not power. The goal is clarity.

The One Big Idea

Every large language model does the same core job:

Look at the text so far. Predict the next token. Append it. Repeat.

That loop, scaled up, produces fluent writing, code, and reasoning.

This prototype makes that loop visible and runnable.

For a broader view of how all the modules (Predictor, ReAct, Memory, Reliability Lab, Trajectory Evaluator) fit together, see architecture.md.

The project also maintains a hosted Reflect and Attempt Quizz (reflect-and-attempt-quizz.html, spaced-repetition active recall) and a Knowledge Reference Site (rendered explainer docs with Mermaid) under the Cloudflare Pages project named prototype-it-to-explain-itself. Both live in the monorepo and are designed to be deployed together.

Rule: After implementing any new prototype, the author must update the root README, this file (new section + sequencing), and architecture.md.

How Text Flows During Generation

flowchart LR
    P[Prompt text] --> E[Encode<br/>to token IDs]
    E --> V[Embedding layer<br/>learned vectors]
    V --> L[LSTM layers<br/>carry memory]
    L --> H[Linear head<br/>raw scores]
    H --> S[Softmax<br/>probabilities]
    S --> Sample[Sample one token]
    Sample --> Append[Append to context]
    Append -->|repeat| L

The model never plans the whole reply at once. At every step it only chooses what comes next.

Code Structure & Big-Picture Flow

The entire prototype lives in one file, deliberately. The numbered sections below map directly to the code.

flowchart TD
    subgraph SRC["simple_llm_prototype.py — 7 Sections"]
        direction TB
        C1["1. CORPUS<br/>STORY repeated 15×"]
        C2["2. TOKENIZER<br/>char ↔ id + encode/decode"]
        C3["3. DATASET<br/>CharDataset — sliding windows"]
        C4["4. MODEL<br/>TinyLLM — Embed + LSTM + Head"]
        C5["5. TRAIN<br/>train_model — next-char loss"]
        C6["6. GENERATE<br/>generate_text + show_top_predictions"]
        C7["7. main<br/>CLI + full pipeline orchestration"]
    end

    C1 --> C2
    C2 --> C3
    C3 --> C5
    C4 --> C5 & C6
    C7 -->|orchestrates| C3 & C4 & C5 & C6

Runtime Execution Flow (what happens when you run the script)

flowchart TD
    Start["python llm/simple_llm_prototype.py<br/>--prompt ... --tokens ..."] --> D1["Encode CORPUS → token tensor"]
    D1 --> D2["CharDataset + DataLoader<br/>creates (context_window, next_char) pairs"]
    D2 --> M["Instantiate TinyLLM<br/>~150k parameters"]
    M --> T["train_model<br/>25 epochs, AdamW, loss only on last position"]
    T --> G["generate_text<br/>autoregressive loop: predict → sample → append → repeat"]
    G --> Opt{"--show-probs ?"}
    Opt -->|yes| P["show_top_predictions<br/>prints the model's actual probability distribution"]
    Opt -->|no| Out["Print generated text + educational summary"]
    P --> Out

Model Architecture (with shapes)

flowchart LR
    subgraph TinyLLM
        direction TB
        In["input_ids<br/>(B, T)"] --> Emb["Embedding<br/>(B, T, 64)"]
        Emb --> Lstm["2-layer LSTM<br/>(B, T, 128)"]
        Lstm --> Head["Linear head<br/>(B, T, vocab_size)"]
        Head --> Logits["logits<br/>(B, T, V)"]
    end

    Logits --> Train["Training: use only [:, -1, :]<br/>CrossEntropy with true next char"]
    Logits --> Gen["Generation: softmax last position<br/>sample 1 token, append, repeat"]

    style In fill:#e3f2fd
    style Logits fill:#fff3e0

Training vs Generation (the same model, two very different loops)

flowchart LR
    subgraph Train["TRAINING (batched, offline)"]
        direction TB
        TW["Many fixed windows<br/>(30 chars)"] --> TF["Forward pass<br/>(teacher forced)"]
        TF --> TL["Loss only on position -1<br/>(predict the 31st char)"]
        TL --> TBK["Backprop + AdamW step"]
    end

    subgraph Gen["GENERATION (online, one token at a time)"]
        direction TB
        GS["Current context + hidden state"] --> GF["Forward pass (small window)"]
        GF --> GSamp["Softmax + temperature<br/>sample ONE token"]
        GSamp --> GApp["Append sampled token<br/>to context + hidden"]
        GApp -->|repeat N times| GS
    end

    Train ~~~ Gen

These diagrams show the structure of the code and how data moves, not just the abstract idea. The source is intentionally small so you can hold the whole picture in your head while reading any one section.

Training Teaches the Guesses

We turn the story into many short examples by sliding a window across it.

flowchart LR
    Corpus["...the machine whispered secrets..."] --> Window1["window: 'the machine w'"]
    Corpus --> Window2["window: 'machine whis'"]
    Window1 --> Target1["target: 'h'"]
    Window2 --> Target2["target: 'p'"]
    Target1 --> Learn["model learns:<br/>after 'the machine w'<br/>'h' is likely"]

Each training step asks only one question: given these characters, what comes next? The model sees thousands of such questions from the repeated story.

We repeat one short story 15 times. The model overfits on purpose. It learns the names, the rhythm, and the world of that story.

The Main Parts

1. Data — One story, repeated. See STORY and CORPUS.
2. Tokenizer — Characters only. About 50 symbols. Real systems use subword tokens (30k–100k).
3. Dataset — Sliding windows that create (context, next-char) pairs.
4. Model — Embedding → 2-layer LSTM → Linear prediction head.
5. Training — AdamW optimizer. Loss only on the single next character.
6. Generation — The autoregressive loop with temperature + (future) top-p / top-k sampling control. See sampling-strategies.md.
7. Inspection — --show-probs shows the model’s top guesses for the next character.

The script runs all of these steps in order when you execute it.

Run It

From the project root:

python llm/simple_llm_prototype.py

Useful variants:

python llm/simple_llm_prototype.py --prompt "Elara dreamed of" --tokens 180 --temp 0.6
python llm/simple_llm_prototype.py --show-probs

See the module docstring in simple_llm_prototype.py for every flag and example.

What We Left Out (on purpose)

Real LLMs	This Version	Reason for the cut
Subword tokenization (BPE)	Character level	Characters are simple to watch and debug
Transformer blocks + attention	2-layer LSTM	LSTM state is easier to follow step by step
Billions to trillions of params	~150k parameters	Small enough that one person can read it all
Internet-scale training data	One story repeated 15×	You can hold the whole set in your head
Long training runs	25 short epochs	Fast edit-run-inspect cycle

The central act stays identical: predict, append, repeat.

Experiments That Teach

• Change the STORY text. Retrain. Notice how the generated voice changes.
• Set --temp 0.3 (safe) vs --temp 1.3 (wild).
• Use --show-probs and watch how the ending context shifts the probabilities.
• Increase hidden_dim or num_layers in the model and measure the effect.

For a deeper look at why temperature alone is not enough and how production systems (including Grok) actually control generation, read sampling-strategies.md. It covers Top-k and Top-p (nucleus) sampling with examples that map directly to the code in generate_text.

From Next-Token to Agent: The Mini ReAct Loop

The original prototype shows the heart of every LLM: predict one token, append it, repeat.

The next natural question is: “How do we turn that predictor into something that can use tools and solve tasks?”

ReAct (Reason + Act) is one of the simplest and most effective patterns.

It is not a bigger model. It is a small control loop:

1. The predictor receives a prompt that contains the goal + tool descriptions + what has happened so far.
2. It generates more text (“Thought: …”).
3. The loop looks for a structured request (“Action: calc[2+3]”).
4. If found, the real Python tool runs and the result is appended as “Observation: …”.
5. The growing history goes back into the next prompt.
6. Repeat until the model writes “Final: …”.

The LLM is still only doing next-token prediction. The agent is the Python code that orchestrates the loop, calls tools, and manages the trajectory.

Visual: ReAct Loop over the Predictor

flowchart TD
    Q[Question for Elara] --> P["Build prompt:<br/>tools + history + 'Thought:'"]
    P --> Pred["Predictor<br/>(tiny_predictor.py)"]
    Pred --> Gen["TinyLLM generate_text<br/>(the same brain)"]
    Gen --> Parse{Parse?}
    Parse -->|"Action: name[args]"| Exec["Execute real Tool<br/>(calc, lookup...)"]
    Exec --> Obs["Observation: result"]
    Obs --> Hist[Append to trajectory]
    Parse -->|Final: answer| Done[Return answer]
    Hist --> P
    Parse -->|no clear action| Hist

The Predictor is the narrow reusable seam (see tiny_predictor.py). Everything above it (the loop, the tools, later memory and evaluators) only sees “text in → text out”.

Run the New Prototypes

# The predictor abstraction itself (tiny)
python llm/tiny_predictor.py   # (mostly docs + example factory)

# The actual ReAct agent
python llm/mini_react.py
python llm/mini_react.py \
  --question "How can Elara measure the power of the glowing crystals?" \
  --max-steps 6 \
  --temp 0.65

Both files live next to simple_llm_prototype.py and import from it. The training story, the Elara universe, and the generation primitive are all reused — no duplication of concepts.

What the Trace Looks Like (example)

=== STEP 2 ===
Model thought / decided:
The crystals glow when the machine is near. I should calculate how much
energy they might hold if we assume each one gives a small spark.
Action: calc[3 * 4 + 2]

Action parsed: calc[3 * 4 + 2]
Observation: 14.0

The loop is visible. The model’s creativity (or lack of perfect formatting) is also visible. This is intentional.

Tool-Use Reliability Lab

See tool_reliability_lab.py.

This prototype turns the ReAct agent into a measurable system. It runs a suite of test cases (direct tool use, ambiguous goals, cases that need no tool, error-injection style), records which tools were actually called, and produces a report with success rates and breakdown by category.

It reuses the exact same Predictor, tools, and (silent) ReAct loop from the previous prototype. Run it after you have a saved model:

python tool_reliability_lab.py

With the current tiny model you will see very low reliability numbers. That is the pedagogical point — it makes the real-world difficulty of reliable tool use visible and quantifiable. Later prototypes can explore better prompting or different models against the same test suite.

Memory

See memory.py and memory_explainer.py.

A tiny, dependency-free memory module designed to be queried by the ReAct agent (or any Predictor-based loop). memory_explainer.py shows the full loop with memory injected into the prompt.

It provides two layers:

• ShortTermMemory: a fixed-size sliding window of recent turns.
• LongTermMemory: a simple list of facts with cheap keyword-overlap retrieval (no vector DB yet — keeps the prototype tiny and easy to read).

Retrieved memories are turned into a plain-text block via format_memories(...) and can be injected into the prompt using the new extra_context parameter on build_prompt.

The module is deliberately small (one file, ~150 lines including a self-test) so you can understand the whole thing quickly and immediately see the effect on the prompt the tiny model receives.

Example from the built-in demo:

stm = ShortTermMemory(window=4)
ltm = LongTermMemory()
ltm.add_facts([... Elara story facts ...])
...
relevant = ltm.retrieve(recent_turns, k=2)
block = format_memories(stm.get(), relevant)
prompt = build_prompt(question, tools, trajectory, extra_context=block)

Trajectory Evaluator

See trajectory_evaluator.py.

This is the first “meta” prototype: it treats the ReAct agent (and future agents) as a system you can run many times and score.

It defines a tiny benchmark of goals in the Elara world, runs run_react in batch mode (using the new return_trajectory=True option), and scores every trajectory on three axes:

• Outcome — simple heuristic checks on the final answer (did it contain an expected fact or number?).
• Process — steps taken and tool calls made (efficiency + whether the agent even tried to use tools).
• Soundness — optional weak “LLM-as-judge” that asks the same tiny model to rate the reasoning 1-5 and give a one-sentence reason. (Expect this to be noisy or unparsable — that’s data.)

The script prints a summary report, per-category breakdown, and one or two example trajectories so the concrete failure modes are visible.

Run:

python llm/trajectory_evaluator.py
python llm/trajectory_evaluator.py --max-steps 6 --episodes 8 --judge

Because the underlying model is deliberately tiny and story-overfit, success rates will be low. The value is turning “the agent sometimes works” into numbers and concrete traces you can improve against.

Local Inference Playground

See local_inference_playground.py.

This prototype exists to prove that the Predictor abstraction (prompt: str → text) really is a stable seam.

It provides several backends that all satisfy the exact same Predictor callable:

• tiny-lstm — the real character-level model we have been using all along.
• stub-fast and stub-smart — pure-Python simulations of “local inference” with different latency/quality trade-offs (no external models required).

You can:

• Run a full ReAct agent (or the evaluator) against any backend.
• See live metrics (wall time, simulated tokens/s).
• Run a small benchmark comparing backends on the same questions.

Crucially, mini_react.py, memory.py, trajectory_evaluator.py, and the tools are never modified when you change the brain.

Run:

python llm/local_inference_playground.py --backend stub-smart
python llm/local_inference_playground.py --backend tiny-lstm --benchmark --runs 4
python llm/local_inference_playground.py --backend stub-fast --question "..." 

In a real environment you would replace the stubs with from_ollama(...), from_llama_cpp(...), from_mlx(...) etc. The agent code would stay identical.

Synthetic Data Factory

See synthetic_data_factory.py.

This is the self-improvement prototype. It uses everything built so far:

• Runs the agent (via the evaluator or direct run_react) many times.
• Adds a cheap self-critique step (another Predictor call that scores “was this a high-quality, correctly formatted run?”).
• Filters to keep only the good trajectories.
• Turns the good traces into training text that teaches the model the ReAct format + useful Elara facts.
• Optionally mixes the synthetic data with the original STORY and actually continues training the tiny model.

The result is a new checkpoint that has seen its own (filtered) successful behavior.

Run:

python llm/synthetic_data_factory.py --episodes 20 --filter
python llm/synthetic_data_factory.py --episodes 15 --train --epochs 5

After training you can load the new _synthetic.pt checkpoint in the playground or evaluator and measure the improvement.

This is the smallest visible version of the “use your agent’s own outputs to make the next version better” loop that powers modern agent and model improvement pipelines.

Typed Agent Workflow

See typed_agent_workflow.py.

This prototype attacks reliability at the type level instead of (or in addition to) prompting and evaluation.

It defines a small set of frozen dataclasses and an Enum for every legal step in a ReAct trajectory (Thought, Action, Observation, FinalAnswer, Error). A simple validator enforces the only allowed transitions:

• Thought → Action or Final
• Action → Observation or Error
• Observation → Thought
• Final / Error → end

The existing untyped run_react is still used to let the tiny model do its thing, but its output is immediately lifted into the typed world. Anything that doesn’t fit becomes an explicit ErrorStep.

In Rust (the production mirror for this prototype) you would get compile-time guarantees via exhaustive match and typestates. The Python version here is the educational stand-in that still runs in our repository and reuses the Predictor.

Run:

python llm/typed_agent_workflow.py
python llm/typed_agent_workflow.py --backend stub-smart --question "calc[9*9]"

Human-in-the-Loop Agent Desktop

See human_in_loop.py.

This is the oversight prototype. It turns the agent from an autonomous black box into a supervised process.

The terminal “desktop” lets you:

• Run with different autonomy modes (manual, ask-on-low-confidence, run-then-review)
• See low-confidence signals (long runs, poor judge scores, errors)
• Approve, edit the final answer, inject observations, or abort
• Get a full audit log of every human intervention with reasons

It still only ever talks to a Predictor, so the same supervision UI works whether the brain is our tiny model or a future strong one.

Run:

python llm/human_in_loop.py
python llm/human_in_loop.py --mode run-then-review --question "..."

In real life this would be a proper desktop app (Tauri, Textual, NiceGUI, etc.) with buttons, live previews of “what the agent is about to do”, and persistent session storage. The patterns (low-confidence surfacing, explicit intervention points, audit logs, autonomy modes) are the same.

Multi-Agent Debate / Collaboration

See multi_agent_debate.py.

This is the final prototype in the original sequence — the capstone.

A tiny orchestrator spawns 2–3 specialist agents for the same hard goal. The specialists are just differently configured versions of everything we built before (different backends via the Predictor, optional memory, typed lifting, etc.). They each produce a proposal. A critic (another Predictor call) scores them. A short debate/synthesis round produces a final answer that is usually better than any single specialist could have managed.

Because every specialist and the critic only ever talk to a Predictor, you can later give some of them strong local models while the orchestrator stays on the tiny teaching brain — or any other combination.

Run:

python llm/multi_agent_debate.py
python llm/multi_agent_debate.py --goal "What is the true relationship between the crystals and the machine?" --agents 3

This pattern (multiple reasoners + critic + synthesis) is the same one used in production mixture-of-agents, self-consistency, and debate systems. Here it is completely visible, tiny, and built on top of the previous eight prototypes.

Sequencing — What Comes Next

We keep every new prototype small by adding one clear idea on top of what already exists:

1. Mini ReAct — the control loop + the Predictor abstraction.
2. Tool-Use Reliability Lab — run the same ReAct + tools many times and measure how often parsing and tool use succeed.
3. Memory — a tiny memory.py (short-term window + simple long-term facts with retrieval) that can be injected into the prompt the ReAct sends to the Predictor. memory_explainer.py demonstrates the full loop.
4. Trajectory Evaluator (trajectory_evaluator.py) — run many ReAct episodes, score outcome + process + weak self-judge, produce reports. (Implemented — this is how we turn demos into an improvement loop.)
5. Local Inference Playground (local_inference_playground.py) — demonstrate that the entire agent stack only ever talks to a Predictor. Swap in stub “local” backends (or the real tiny one) and measure latency/quality while keeping ReAct, memory, and the evaluator 100% unchanged. (Implemented)
6. Synthetic Data Factory (synthetic_data_factory.py) — generate many trajectories, self-critique + filter the good ones, turn them into training data, and (optionally) actually improve the model. Closes the “agent improves itself” loop. (Implemented)
7. Typed Agent Workflow (typed_agent_workflow.py) — model the ReAct loop with strict types and an explicit state machine so many classes of invalid execution become impossible or loudly rejected. Python illustration of the “reliability by construction” idea (the real version belongs in Rust). (Implemented)
8. Human-in-the-Loop Agent Desktop (human_in_loop.py) — terminal “desktop” that surfaces low-confidence steps, lets a human approve/edit/inject observations, logs every intervention, and supports different autonomy modes. Teaches the oversight patterns that real production agents need. (Implemented)
9. Multi-Agent Debate / Collaboration (multi_agent_debate.py) — orchestrator spawns specialist agents (different backends/memory), lets them propose, runs a critic/judge round, and synthesizes a better answer. The capstone that composes everything previous through the Predictor seam. (Implemented)

Later steps deliberately choose different languages/stacks when they teach the concept better and match real production usage (Rust for typed/verifiable workflows, GUI stacks for human oversight, mixed backends for local inference, etc.). The Predictor is the place where language boundaries become possible.

See PROTOTYPE_ROADMAP.md (full curated 9-idea list + concepts + table + current status) and the live todo list for the prioritized sequence.

This Prototype Fits a Larger Pattern

This is one working example of the “prototype it to explain itself” method.

See the main README for the intent behind the whole collection and the plan for more prototypes.

---

Run it. Read every line. Change one thing. Run it again. The mechanism stops being abstract.

Test the ReAct responsibility split