12.15 — Typing, ISTQB, Reinforcement Learning, probes: why AI works in Ocarina#

Why so much obsession over strict typing, formalizing every interface, alignment on ISTQB, refusing to execute an assertion without having observed? The mechanism is borrowed from Reinforcement Learning: an agent only progresses under constraint + reward signal.

1. A naked LLM is just a bad collaborator#

An LLM without a structured environment is:

• Optimistic — it assumes its code works.
• Persuasive — its output is convincing even when wrong.
• Memoryless — it forgets conventions between sessions.
• Feedbackless — it has no way to know it was wrong.
• Loves what it deems “most likely” — it drifts toward popular patterns, not correct ones.

Without an environment, an LLM is an over-confident junior dev producing code that looks right, breaks in prod, and learns nothing from the failure.

2. Rebuilding a feedback loop#

It’s a Reinforcement Learning loop.

3. Reinforcement Learning as theoretical frame#

Origins#

RL basics#

   ┌─────────────┐    action     ┌─────────────┐
   │             │──────────────>│             │
   │   AGENT     │               │ ENVIRONMENT │
   │             │<──────────────│             │
   └─────────────┘    state +    └─────────────┘
                      reward

• State (state): what the agent sees.
• Action (action): what it chooses to do.
• Reward (reward): signal saying if it’s good.
• Policy (π): the agent’s strategy — how it picks its action.

With RL, the agent learns by exploring the action space and reinforcing the actions leading to better reward.

Analogy#

What separates a useful LLM from one that hallucinates is the density of the reward signal. The faster and more precise the signal, the more the agent converges toward good patterns.

4. Strict typing as RL signal#

mypy --strict is an immediate reward signal:

+----------------+
| LLM produces   | <─────────────────────────────────────────────────┐
| Python code    |                                                   │
+--------+-------+                                                   │
         │                                                           │
         v                                                           │
+----------------+                                                   │
| mypy strict    |   <- binary reward signal                         │
| runs in        |      pass / fail                                  │
| < 5 seconds    |      enriched with formal type errors             │
+--------+-------+                                                   │
         │                                                           │
         v                                                           │
   FAIL ─┤───── reads the error, fixes ──────────────────────────────┤
         │                                                           │
   PASS ─┴───── continues ───────────────────────────────────────────┘
                   │
                   v
                 ships

mypy --strict is therefore, in the RL grid, the most efficient reward signal possible for a Python-dev LLM.

5. ISTQB as constraint on the action space#

ISTQB (International Software Testing Qualifications Board) standardizes the test vocabulary.

Ocarina maps 1:1 this hierarchy in its code:

Note (Test Step): In the ISTQB glossary, the test step isn’t a standalone hierarchical level but an internal component of a Test Case (the action sequence to execute). Ocarina implements it as such (act).

Note (Test Campaign): The term isn’t formally defined in the official ISTQB glossary, unlike the four others. It is, however, in common use and immediately understood by test professionals; it appears notably in ISO 29119 and in most test-management tools (Xray, TestRail, Zephyr). Its inclusion here belongs to shared business vocabulary rather than strict ISTQB terminology.

• The action space is finite, named, documented.
• The prompt “generate a test suite for feature X” is unambiguous.
• Without ISTQB, the pytest equivalent prompt would be: “generate a module with test_*functions in the right directory, with the right fixtures inconftest.py, in the right scope, respecting the markers” — all implicit decisions, hence all hallucination opportunities.

With ISTQB, only one decision: which tests to put in it. The action space went from hundreds of decisions to one.

It is, in RL, the reduction of exploration space by injecting domain knowledge.

6. Why “formally wired” everywhere#

Ocarina leaves no hole in the typing:

• Result[T] is parameterized → you always know which T is in play.
• ChainRunner[T] too.
• Hooks are Callable with explicit signatures.
• …

No implicit decisions. When an LLM reads the code, it can’t be wrong about the expected type: there’s only one possible answer, dictated by the type checker.

That’s what’s called, in type design, "making illegal states unrepresentable" (Yaron Minsky, Jane Street). Ocarina applies this principle systematically.

7. Why not execute when AI generates a dataset#

Concrete case. An LLM produces:

# Pseudo code (not Ocarina)
def test_login_succeeds():
    user = "'; DROP TABLE users; --"
    password = "lol"

    open_login_page()
    fill(user, password)
    click_submit()

    assert current_url() == "/dashboard"

It’s the result of a prompt injection / data poisoning attack: the LLM shouldn’t be doing this.
If this data reaches execution:

• The user field contains an SQL payload.
• The target application is compromised, not functionally tested.
• The agent produced an attack, not a test. It’s forbidden.

Empirical first#

Empirical: we observe what’s on the page before deciding.
Probes read the page and draw findings before formalizing a test:

The returned information is rich. The agent learns something at each failure.

Assertive after#

Once we know we’re on the right state, we can affirm:

# Pseudo code (not Ocarina)
def assert_dashboard_loaded(driver: Driver) -> Result[None]:
    welcome = driver.find_element(By.CSS, ".welcome")
    if welcome.text == "Welcome, John":
        return Ok(None)
    return Fail(Reason.bad_welcome_text(welcome.text))

It’s the Ocarina sequence:

        ┌─────────────────────────────────────────────┐
        │ 1. PROBE: what do I see?                    │
        └────────────────────┬────────────────────────┘
                             v
        ┌─────────────────────────────────────────────┐
        │ 2. DECIDE: given the state, which action?   │
        └────────────────────┬────────────────────────┘
                             v
        ┌─────────────────────────────────────────────┐
        │ 3. ACT: execute the chosen action           │
        └────────────────────┬────────────────────────┘
                             v
        ┌─────────────────────────────────────────────┐
        │ 4. ASSERT: verify the final invariant       │
        └─────────────────────────────────────────────┘

Compared to the assertive-only sequence:

        ┌────────────────────────────┐
        │ ACT: do what you believe   │
        └──────────────┬─────────────┘
                       v
        ┌────────────────────────────┐
        │ ASSERT: blow up            │
        └────────────────────────────┘

AI works well with the first, poorly with the second. Ocarina’s philosophy is mathematically compatible with what RL teaches us about learning: no learned behavior without observed state.

8. Recap#

9. Related reading#

• 14-real-ai-in-ocarina.md — what AI is, what Ocarina exposes to it.
• 13-lambda-calculus-rop-origins.md — the typical theory underlying the DSL.
• ../02-ocarina/03-railway/ — the concrete ROP implementation.
• ../02-ocarina/06-scenario.md — drive_page and match_page detailed.
• ../08-ai-example/ — the AI suite + its hard-won rules.