12.14 — AI, and how Ocarina applies it for real#
“AI-powered” became a totally idiotic marketing slogan in 2023-2026: 99% of products claiming it bolt a
chat.completionsendpoint onto an existing product. Ocarina proposes, on the contrary, a software infrastructure designed so an AI can contribute like a senior developer.ocarina-with-ai-exampleis the proof by example.
1. What AI is today#
| Layer | Example | Appearance date |
|---|---|---|
| Statistical learning | Regression, k-means, decision trees | 1960s-1990s |
| Deep learning | CNN (vision), RNN/LSTM (text) | 2012 (AlexNet — mainstream adoption) |
| Transformer-based foundation models | GPT, Claude, Gemini, Llama | 2017 (paper), 2018 (BERT), 2022 (ChatGPT) |
| Agentic workflows | AutoGPT, LangChain, Claude Code, Cursor agents | 2023+ |
Note: CNNs were invented by Yann LeCun as early as 1989 (MNIST). AlexNet (Krizhevsky, Sutskever, Hinton, Toronto, 2012) was the industrial breakthrough. LeCun, Hinton, and Bengio share the 2018 Turing Award for their foundational contributions to deep learning.
When Ocarina’s Holy Book talks about “AI”, it talks about layers 3 and 4: pre-trained transformer-based LLMs, and the agentic loops that invoke them.
Attention is All You Need#
| Field | Value |
|---|---|
| Paper | "Attention Is All You Need" |
| Authors | Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin |
| Org | Google Brain, Google Research, University of Toronto |
| Date | June 2017 (arXiv), presented at NeurIPS December 2017 |
| Contribution | Transformer architecture: self-attention instead of recurrence (LSTM) |
The Transformer architecture lets you parallelize training on GPU. What took weeks in LSTM does hours. Training scales to Internet-size. GPT-1 (2018) → GPT-2 (2019) → GPT-3 (2020, 175B params) → GPT-4 (2023) → Claude 3 (2024) → Claude 4 (2025).
Actors#
| Org | Models | Posture |
|---|---|---|
| OpenAI (Sam Altman, San Francisco) | GPT-4o, o1, GPT-5, GPT-5.5 | Closed weights, subscription |
| Anthropic (Dario & Daniela Amodei, San Francisco) | Claude 3, 3.5, 4 | Closed weights, focus on safety and alignment (Constitutional AI) |
| Google DeepMind | Gemini 1.5, 2.0, 2.5 | Closed weights, Google integration |
| Meta AI | Llama 2, 3, 4 | Open weights (almost open source) |
| xAI (Musk) | Grok 1, 2, 3 | Partial open weights |
| Mistral AI (Paris) | Mistral, Mixtral | Open weights, European |
2. Operational capabilities of a 2025-2026 LLM#
To understand how Ocarina uses them, you need to list honestly what a modern LLM can and cannot do.
What an LLM does well#
| Capability | Level |
|---|---|
| Reformulation, translation | ★★★★★ |
| Local code generation (one file, one function) | ★★★★★ |
| Understanding docs that fit in context | ★★★★★ |
| Detecting patterns in code | ★★★★ |
| Generating tests from documented code | ★★★★ |
| Mechanical refactor (rename, extract) | ★★★★ |
| Following a convention if explicitly documented | ★★★★ |
| Applying a fixed grammar (bounded, typed DSL) | ★★★★★ |
What an LLM does poorly, and how Ocarina responds#
| Limit | Consequence | Ocarina response |
|---|---|---|
| Hallucinations | Invents nonexistent functions / classes | Mypy strict |
| Long-context degradation | Loses precision on the middle of a long file (lost in the middle) | Short modules, split SKILL.md files |
| No runtime feedback without tooling | If nothing tells it that it crashed, the LLM continues | Ocarina probes (see 15-typing-ai-rl-probes.md) |
| Prefers popular patterns | Drifts toward pytest, async/await, mock | Explicit documentation of Ocarina’s refusals |
| Doesn’t fill in a poorly specified domain | Unsatisfying with ill-defined technologies | Well defined, algebraic conventions, ISTQB-first |
| No memory between sessions | Loses project conventions | CLAUDE.md, skills |
| No intuition about invariants | Generates code that passes tests but doesn’t respect the spirit | Structural ROP that enforces the grammar |
3. What “AI-first” means operationally#
| Level | Characteristics | Example |
|---|---|---|
| 0 — Slogan | “Powered by AI” without a real surface | Majority of 2024 SaaS |
| 1 — External chatbot | OpenAI endpoint in a UI wrapper | Majority of AI-enhanced |
| 2 — RAG over the docs | Retrieves passages, glues them into the response | Algolia DocSearch + OpenAI plugin, Mintlify |
| 3 — Native AI interface | The product is designed for an agent to act on it | MCP servers, llms.txt, AI SDK, Cursor rules |
| 4 — Co-design | The product creates deliverables co-written by an AI agent | Ocarina + ocarina-with-ai-example |
Ocarina is one of very few projects to reach level 4 publicly and with auditable evidence.
4. AI-first artifacts in Ocarina#
llms.txt, llms-full.txt#
De-facto standard proposed by llmstxt.org (Jeremy Howard, fast.ai), 2024.
llms.txt: a short Markdown index, designed for an agent to find what to read.llms-full.txt: the full concatenated doc, designed for a single prompt that ingests everything.
Ocarina exposes both. See 09-holy-book/06-public-resources.md
CLAUDE.md / CLAUDE.slim.md#
Anthropic convention: CLAUDE.md at the project root, automatically read by Claude Code (and compatibles):
- Project identity, philosophy.
- Conventions to follow.
- Main commands.
- Known gotchas.
Ocarina maintains:
- Long
CLAUDE.md(large context) for deep onboarding. - Short
CLAUDE.slim.md(token thrift) for routine interactions.
Skills (SKILL.md)#
40+ skills exposed by the Holy Book. One skill = one SKILL.md file:
- YAML metadata (
name,description). - Step-by-step procedure.
- Invocation conditions ("when the user asks for X, do Y").
Lets Claude Code, Cursor, etc. plug in. See 09-holy-book/03-skills/
ocarina-with-ai-example#
A full e2e project, written to test a real app (CURA), co-written by Claude. The repo’s CLAUDE.md documents the hard-won rules learned during co-writing.
This suite is a proof by example that Ocarina’s AI-first approach isn’t an invention: it has been tested against a SUT in documented collaboration with an LLM. See 08-ai-example/
5. “AI-marketing” vs “AI-for-real”#
| Criterion | AI-marketing | AI-for-real (Ocarina) |
|---|---|---|
| Documented AI surface | “Powered by GPT-4” | llms.txt, llms-full.txt, CLAUDE.md, 40+ skills |
| Proof of AI use | None or wobbly | ocarina-with-ai-example |
| Doc format | PDF / SaaS dashboard only | Raw Markdown |
| Agent compatibility | Nonexistent | Built for Claude Code, Cursor, Continue, Cline |
| Grammar convention | Ad-hoc | Strict ROP + ISTQB, static typing |
| LLM context management | Neglected (huge monolithic files) | Modularized to the end |
| Reaction to hallucinations | None | Well-defined technology, type errors, max strictness |
| Reproducibility | Non-deterministic by construction | Probes + caching |
6. Why this preparation gives Ocarina an edge for the next 5 years#
- The barrier to entry of e2e testing will collapse because an LLM can write most of the boilerplate.
- Only LLM-compatible frameworks will stay relevant; the others will become legacy.
- LLM compatibility depends on three things:
- A small well-defined DSL (fits in context).
- A formal grammar (mypy / ROP / ISTQB).
- Structured documentation (Markdown + skills).
- Ocarina ticks every box.
- Pytest, Cypress, Playwright, Robot Framework don’t. Their surface is too large (hundreds of plugins), their grammar too fuzzy (async, fixtures), their documentation too fragmented (SaaS sites, videos, scattered plugins).
Ocarina bets that AI will devour large-surface frameworks, exactly as Tugan Bara was ejected by AI in copywriting (see 10-infopreneurs-tugan-bara-ai.md).
7. How AI actually works on Ocarina#
1. Claude reads /CLAUDE.md (project conventions).
2. Claude reads /docs/what-is-it.md (philosophy).
3. Claude reads /SKILL_<feature>.md (matching skill).
4. Claude reads /tests/scenarios/<family>/*.py (sample).
5. Claude writes a new `.py` file.
6. Mypy strict runs automatically.
7. If mypy fails → Claude reads the diagnostic, fixes, reruns.
8. If mypy passes → Claude runs the test against the SUT.
9. If the test fails → Claude reads the output, adjusts probes, re-reads.
10. If the test passes → Claude commits, pushes, opens a PR.Steps 2, 3, 4, 6, 9 are constrained by Ocarina’s infrastructure. Without CLAUDE.md, step 1 is random. Without mypy strict, step 6 doesn’t exist. Without transient_errors, match_page, and watchers, step 9 produces flaky tests.
That’s what makes Ocarina designed for AI, not bolted to AI.
8. Related reading#
13-lambda-calculus-rop-origins.md— Ocarina’s DSL’s theoretical base, which makes it LLM-compatible.15-typing-ai-rl-probes.md— why typing + RL + probes are the triad that makes AI productive.../08-ai-example/— the proof by example: the AI suite.../09-holy-book/03-skills/— the exposed skills.