09.03.01 — Review skills#

Static reads, surface findings. A large family. Lets the AI review its project systematically.

Listing (potentially non-exhaustive)#

review-dead-code#

input  : test base
output : list of unused elements (connectors / POMs / scenarios / fragments / constants)
         + per-element recommendation:
            - delete
            - move to incubator (<root-source>/incubator/, dependency tree preserved)
            - keep (justify)

review-report#

input  : a recent execution (logs + reports)
output : per-test classification:
            - PASS                  (nothing to do)
            - SKIP                  (why?)
            - intentional gap FAIL  (G-DATA-*, G-SEC-*, ...)
            - cross-browser FAIL    (B-BROWSER-*)
            - transient FAIL        (A-ENV-*)
            - regression            ⚠️ ALERT

review-watcher-misuse#

input  : all watcher.report(...) calls
output : list of reports that look positive ("success", "completed", "ok", ...)
         → recommendation: remove or rephrase as negative

review-comment-drift#

input  : every code comment
output : list of comments that no longer match the adjacent code
         (typical: comment mentions foo, code mentions bar)

Helps remove stale comments.
“Teach the pattern, not the symptom.”

09.03.02 — Analyse skills#

Dynamic analyses: use the logs / reports of a recent execution to diagnose a failure. Two root-cause skills sit in front of the family — diagnose-root-cause for a deterministic red, diagnose-flake-root-cause for an intermittent one — and route to the controlled analyse-* experiments.

Listing (potentially non-exhaustive)#

diagnose-root-cause#

input  : a deterministic red (fails every run)
output : structured root-cause analysis:
            - re-derive the failure, don't inherit prior assumptions
            - climb the synthetic-to-real ladder (fake driver → real browser)
            - read the source, confirm with a probe
            - root cause sorted into five buckets
         hands off to diagnose-flake-root-cause if the failure proves intermittent

diagnose-flake-root-cause#

input  : an intermittent failure (a flake)
output : distribution-based root-cause analysis:
            - establish a failure rate (the distribution is the evidence)
            - pin the signature, correlate
            - route to the right analyse-* experiment, raise the rate to confirm
            - classify into five flake buckets
         the orchestrator of the analyse-* family

analyse-flakiness#

input  : logs of the last N runs
output : list of tests that retry often (by % of retries)
         + recommendation:
            - add an exception to the transient_errors tuple
            - investigate a particular test (the issue might come from the test itself)

analyse-fixture-flakiness#

input  : tests with setup/teardown
output : tests whose setup fails often
         + recommendation:
            - verify setup idempotency
            - check teardown is clean (otherwise cross-test contamination)
            - move the seed out of setup into a manual init if too fragile

analyse-watcher-flakiness#

input  : watcher.report(...) logs over N runs
output : watchers with abnormal reporting:
            - too many emissions (dedup issue)
            - no emissions (the watcher is useless)
            - false positives (expected element reported as intruder)

analyse-screenshot-flakiness#

input  : screenshots from N runs, grouped by (test, step, browser)
output : visual flakiness:
            - tests whose screenshots vary abnormally between runs
            - tests that pass but whose look changes
            - identify whether the variation is semantic (real diff) or cosmetic

Uses heuristics (hash, dimensions, dominant palette). Not a real image diff.

09.03.03 — Black-hat skills (6)#

Ideations of business logic vulnerabilities — no execution: “security testing is functional and static, never active”.

Listing (potentially non-exhaustive)#

Perimeter#

All these skills imagine attacks the AI could phrase.

09.03.04 — Comprehend skills#

Skills that help the AI understand a project or ecosystem before acting.

Listing (potentially non-exhaustive)#

understand-ocarina#

input  : user question — routed by class:
            - "how do I do a match_page?"        (reference: what a primitive is) → Holy Book
            - "why is the suite built on Railway?" (intent / cartography / why)    → the from-ocarina-to-igor book
output : Claude loads, by question class:
            - reference  → relevant Holy Book pages (https://mojo-molotov.github.io/ocarina-holy-book)
            - intent     → relevant chapters of the from-ocarina-to-igor book (https://mojo-molotov.github.io/from-ocarina-to-igor/)
            - behaviour  → Ocarina source clone, if the docs don't cover it
            - shape      → the worked example matching the project's driver adapter:
                              Selenium   → ocarina-example, ocarina-with-ai-example
                              Playwright → ocarina-with-playwright-example
         then answers with citations (tier + page/chapter URL or file:line)

SKILL.md:

09.03.05 — Pick skills#

Picking artifacts (screenshots, logs, reports). Always by mtime, never by filename.

Listing (potentially non-exhaustive)#

“mtime”#

Holy Book quote:

Mtime, not filename. UUID suffixes are random; pick-* sorts by mtime.

09.03.06 — Author skills#

Skills that produce a deliverable. Tests, probes, docs, reports.

Listing (potentially non-exhaustive)#

empiricism#

input  : user intent ("write a test that asserts X")
output : before encoding X:
            1. empirically verify that X is true
                - probe
                - gh api on the source code
                - curl -v to inspect HTTP responses
                - ...
            2. if X confirmed → encode
            3. if X false → correct intent
            4. if X ambiguous → ask the human

“Fair point, I’m assuming. Let me verify empirically.”

09.03.07 — Refactor skills#

Skills that refactor the existing automated-test base.

Listing (potentially non-exhaustive)#

refactor-fragmentation#

input  : the test base
output : DRY refactor suggestions:
            - blocks repeated in 3+ scenarios → extract into a fragment
            - near-identical connectors → extract a parameterized connector
            - repeated log+screenshot sequences → extract a helper
         + per-case recommendation

CLAUDE.md:

09.03.08 — State skill#

A single skill: question-state. Interrogates the environment before believing a result.

The skill#

input  : a surprising result ("the test fails")
output : a series of questions about state:
            - which browser are we running on?
            - which commit?
            - is the SUT warm or cold?
            - is Redis available?
            - what's the configured wait-timeout?
            - what's the driver version?
            - does ChromeDriver have its password-manager-off fix?
            - is this an isolated run or a parallel run?
            - are other tests touching the same SUT resources?

Approach#

CLAUDE.md:

09.03.09 — Setup skills#

Two skills: setup-environment (onboarding a new contributor, human or AI) and profile-environment (governing how much latitude the engagement grants the LLM). Both feed the suite’s assembled CLAUDE.md.

The skill#

input  : a fresh repo
output : onboarding steps:
            1. venv
            2. pip install (dev deps + ocarina)
            3. ruff / mypy / pre-commit installed
            4. CLAUDE.local.md created (paths requested from user)
            5. runner smoke-check (minimal test that validates everything works)
            6. pre-commit loop tested

Step 1: venv#

python -m venv .venv
source .venv/bin/activate

Step 2: pip install#

pip install . ruff mypy mypy-extensions typing-extensions pre-commit

Step 3: dev tooling#

pre-commit install --config .pre-commit-config.yaml

Step 4: CLAUDE.local.md#

CLAUDE.local.md is gitignored and contains machine-specific paths.

09.03.10 — Run skills#

Surface the choices to make before a local dispatch, compose the command, and hand it back. The skill never launches the run itself.

Listing (potentially non-exhaustive)#

propose-visual-review#

input  : an upcoming local run
output : the headed vs headless choice:
            - --not-headless: watch the browser, useful when debugging a flaky flow
            - headless (CI-shaped): faster, matches what CI does
         + the trade-off and what to watch for during a headed run
         + the composed command, handed back for the user to launch

Role#

• Surface the pre-run choice; explain the trade-off.
• Compose the command.
• Don’t launch it — the user runs it.