Skills on Ocarina — manual

09.03.01 — Review skills

Wed, 20 May 2026 00:00:00 +0000

09.03.01 — Review skills#

Static reads, surface findings. A large family. Lets the AI review its project systematically.

Listing (potentially non-exhaustive)#

Skill	Target
`review-spec-gaps`	Clarification questions on the FRDs
`review-watcher-misuse`	Verifies the “negative-only” principle of `watcher.report(...)`
`review-compartmentalisation-leaks`	Detects URLs, selectors, magic numbers in wrong places
`review-dead-code`	Detects unused connectors / POMs / scenarios / suites / fragments / constants
`review-report`	Classifies each FAIL / SKIP of an execution
`review-type-ignore`	Audits `# type: ignore`s (are they justified?)
`review-match-candidates`	Spots places where a `match` (Python, pattern matching) could be used instead of `if` `elif` `elif` `if` `if` `elif`
`review-unverified-transitions`	Verifies that every page transition has a `verify`
`review-submit-dispatchers`	Audits input-confirmation methods (click vs enter key…)
`review-comment-drift`	Detects comments that have drifted from the code
`review-suite-stability`	Evaluates a suite’s stability (proportion of retries, transient_errors hits)
`review-intent-collisions`	Detects tests that overwrite each other (conflicting intents) and asks/proposes clarifications
`review-watcher-emissions`	Audits watcher emissions (volume, dedup, relevance)
`review-hierarchy-naming`	Audits the test-hierarchy naming (`TestCycle` / `TestCampaign` / `TestSuite` / `Test`) for the lazy-naming antipattern where a child carries the parent’s name

`review-dead-code`#

input  : test base
output : list of unused elements (connectors / POMs / scenarios / fragments / constants)
         + per-element recommendation:
            - delete
            - move to incubator (<root-source>/incubator/, dependency tree preserved)
            - keep (justify)

`review-report`#

input  : a recent execution (logs + reports)
output : per-test classification:
            - PASS                  (nothing to do)
            - SKIP                  (why?)
            - intentional gap FAIL  (G-DATA-*, G-SEC-*, ...)
            - cross-browser FAIL    (B-BROWSER-*)
            - transient FAIL        (A-ENV-*)
            - regression            ⚠️ ALERT

`review-watcher-misuse`#

input  : all watcher.report(...) calls
output : list of reports that look positive ("success", "completed", "ok", ...)
         → recommendation: remove or rephrase as negative

`review-comment-drift`#

input  : every code comment
output : list of comments that no longer match the adjacent code
         (typical: comment mentions foo, code mentions bar)

Helps remove stale comments.
“Teach the pattern, not the symptom.”

09.03.02 — Analyse skills

Wed, 20 May 2026 00:00:00 +0000

09.03.02 — Analyse skills#

Dynamic analyses: use the logs / reports of a recent execution to diagnose a failure. Two root-cause skills sit in front of the family — diagnose-root-cause for a deterministic red, diagnose-flake-root-cause for an intermittent one — and route to the controlled analyse-* experiments.

Listing (potentially non-exhaustive)#

Skill	Target
`diagnose-root-cause`	Structured RCA for a deterministic red: re-derive, the synthetic-to-real ladder, root cause sorted into five buckets
`diagnose-flake-root-cause`	Structured RCA for an intermittent failure (a flake): failure rate, signature, correlation; orchestrates the `analyse-*` experiments
`analyse-flakiness`	Widens the `transient_errors` net; chronic deaths are real flakes
`analyse-fixture-flakiness`	Instruments setup/teardown; surfaces cross-test contaminations
`analyse-watcher-flakiness`	Analyzes watcher reliability (volume, dedup, false positives)
`analyse-screenshot-flakiness`	Groups screenshots by `(test, step, browser)`, detects differences

`diagnose-root-cause`#

input  : a deterministic red (fails every run)
output : structured root-cause analysis:
            - re-derive the failure, don't inherit prior assumptions
            - climb the synthetic-to-real ladder (fake driver → real browser)
            - read the source, confirm with a probe
            - root cause sorted into five buckets
         hands off to diagnose-flake-root-cause if the failure proves intermittent

`diagnose-flake-root-cause`#

input  : an intermittent failure (a flake)
output : distribution-based root-cause analysis:
            - establish a failure rate (the distribution is the evidence)
            - pin the signature, correlate
            - route to the right analyse-* experiment, raise the rate to confirm
            - classify into five flake buckets
         the orchestrator of the analyse-* family

`analyse-flakiness`#

input  : logs of the last N runs
output : list of tests that retry often (by % of retries)
         + recommendation:
            - add an exception to the transient_errors tuple
            - investigate a particular test (the issue might come from the test itself)

`analyse-fixture-flakiness`#

input  : tests with setup/teardown
output : tests whose setup fails often
         + recommendation:
            - verify setup idempotency
            - check teardown is clean (otherwise cross-test contamination)
            - move the seed out of setup into a manual init if too fragile

`analyse-watcher-flakiness`#

input  : watcher.report(...) logs over N runs
output : watchers with abnormal reporting:
            - too many emissions (dedup issue)
            - no emissions (the watcher is useless)
            - false positives (expected element reported as intruder)

`analyse-screenshot-flakiness`#

input  : screenshots from N runs, grouped by (test, step, browser)
output : visual flakiness:
            - tests whose screenshots vary abnormally between runs
            - tests that pass but whose look changes
            - identify whether the variation is semantic (real diff) or cosmetic

Uses heuristics (hash, dimensions, dominant palette). Not a real image diff.

09.03.03 — Black-hat skills

Wed, 20 May 2026 00:00:00 +0000

09.03.03 — Black-hat skills (6)#

Ideations of business logic vulnerabilities — no execution: “security testing is functional and static, never active”.

Listing (potentially non-exhaustive)#

Skill	Target
`business-logic-vulnerability-ideation`	Break the product through legitimate-looking but malicious usage paths
`incoherence-attack-ideation`	Each step taken alone looks innocent; but combinations of these steps can cause an inconsistency in the system
`persistence-attack-ideation`	Repeated attempts at a blocked action
`permission-appropriateness-audit`	Is the access model itself appropriate?
`bfcache-exposure-ideation`	BFCache attacks
`lateral-resource-ideation`	IDOR via the address bar only

Perimeter#

All these skills imagine attacks the AI could phrase.

09.03.04 — Comprehend skills

Wed, 20 May 2026 00:00:00 +0000

09.03.04 — Comprehend skills#

Skills that help the AI understand a project or ecosystem before acting.

Listing (potentially non-exhaustive)#

Skill	Target
`assess-test-base`	Catalogs the existing test base
`assess-ecosystem`	Bounded public research, capped by a token budget
`assess-impact`	Forward impact analysis: a change’s blast radius through the dependency graph
`understand-sut-constraints`	Understands the SUT’s “boundaries” so as not to exceed them
`understand-ocarina`	Routes by question class: Holy Book for reference, the from-ocarina-to-igor book for intent; then Ocarina source + the adapter-matched worked example

`understand-ocarina`#

input  : user question — routed by class:
            - "how do I do a match_page?"        (reference: what a primitive is) → Holy Book
            - "why is the suite built on Railway?" (intent / cartography / why)    → the from-ocarina-to-igor book
output : Claude loads, by question class:
            - reference  → relevant Holy Book pages (https://mojo-molotov.github.io/ocarina-holy-book)
            - intent     → relevant chapters of the from-ocarina-to-igor book (https://mojo-molotov.github.io/from-ocarina-to-igor/)
            - behaviour  → Ocarina source clone, if the docs don't cover it
            - shape      → the worked example matching the project's driver adapter:
                              Selenium   → ocarina-example, ocarina-with-ai-example
                              Playwright → ocarina-with-playwright-example
         then answers with citations (tier + page/chapter URL or file:line)

SKILL.md:

09.03.05 — Pick skills

Wed, 20 May 2026 00:00:00 +0000

09.03.05 — Pick skills#

Picking artifacts (screenshots, logs, reports). Always by mtime, never by filename.

Listing (potentially non-exhaustive)#

Skill	Target
`pick-screenshots`	Picks the most recent screenshots (by mtime), contextualizing them with logs where possible
`pick-logs`	Picks the most recent logs (by mtime)
`pick-reports`	Picks the most recent DOCX/JSON reports (by mtime)

“mtime”#

Holy Book quote:

Mtime, not filename. UUID suffixes are random; pick-* sorts by mtime.

09.03.06 — Author skills

Wed, 20 May 2026 00:00:00 +0000

09.03.06 — Author skills#

Skills that produce a deliverable. Tests, probes, docs, reports.

Listing (potentially non-exhaustive)#

Skill	Target
`empiricism`	Verify before encoding; don’t crush an intentionally failing gap test
`write-a-probe`	Throwaway “probe” script, gitignored
`write-test-strategy`	Generates the test strategy document from the suite
`extend-coverage`	Extends coverage from existing heritage
`update-frd-and-tests`	Propagates a spec update
`manual-reproduction-guide`	Writes a step-by-step reproduction scenario executable by a human
`manage-backlog`	`BACKLOG.md`
`pr-report`	PR report adapted to context
`plan-test-effort`	First-pass test-effort plan: graded requirements, lightweight risk register, relative effort weights (S / M / L), open questions

`empiricism`#

input  : user intent ("write a test that asserts X")
output : before encoding X:
            1. empirically verify that X is true
                - probe
                - gh api on the source code
                - curl -v to inspect HTTP responses
                - ...
            2. if X confirmed → encode
            3. if X false → correct intent
            4. if X ambiguous → ask the human

“Fair point, I’m assuming. Let me verify empirically.”

09.03.07 — Refactor skills

Wed, 20 May 2026 00:00:00 +0000

09.03.07 — Refactor skills#

Skills that refactor the existing automated-test base.

Listing (potentially non-exhaustive)#

Skill	Target
`refactor-fragmentation`	DRY based on user preference
`introduce-pom-retries`	POM-internal retries, with split (first-try + with-retries)

`refactor-fragmentation`#

input  : the test base
output : DRY refactor suggestions:
            - blocks repeated in 3+ scenarios → extract into a fragment
            - near-identical connectors → extract a parameterized connector
            - repeated log+screenshot sequences → extract a helper
         + per-case recommendation

CLAUDE.md:

09.03.08 — State skill

Wed, 20 May 2026 00:00:00 +0000

09.03.08 — State skill#

A single skill: question-state. Interrogates the environment before believing a result.

The skill#

input  : a surprising result ("the test fails")
output : a series of questions about state:
            - which browser are we running on?
            - which commit?
            - is the SUT warm or cold?
            - is Redis available?
            - what's the configured wait-timeout?
            - what's the driver version?
            - does ChromeDriver have its password-manager-off fix?
            - is this an isolated run or a parallel run?
            - are other tests touching the same SUT resources?

Approach#

CLAUDE.md:

09.03.09 — Setup skills

Wed, 20 May 2026 00:00:00 +0000

09.03.09 — Setup skills#

Two skills: setup-environment (onboarding a new contributor, human or AI) and profile-environment (governing how much latitude the engagement grants the LLM). Both feed the suite’s assembled CLAUDE.md.

The skill#

input  : a fresh repo
output : onboarding steps:
            1. venv
            2. pip install (dev deps + ocarina)
            3. ruff / mypy / pre-commit installed
            4. CLAUDE.local.md created (paths requested from user)
            5. runner smoke-check (minimal test that validates everything works)
            6. pre-commit loop tested

Step 1: venv#

python -m venv .venv
source .venv/bin/activate

Step 2: pip install#

pip install . ruff mypy mypy-extensions typing-extensions pre-commit

Step 3: dev tooling#

pre-commit install --config .pre-commit-config.yaml

Step 4: `CLAUDE.local.md`#

CLAUDE.local.md is gitignored and contains machine-specific paths.

09.03.10 — Run skills

Wed, 20 May 2026 00:00:00 +0000

09.03.10 — Run skills#

Surface the choices to make before a local dispatch, compose the command, and hand it back. The skill never launches the run itself.

Listing (potentially non-exhaustive)#

Skill	Target
`propose-visual-review`	Offer headed (`--not-headless`) vs headless (CI-shaped) before a run, then compose the command

`propose-visual-review`#

input  : an upcoming local run
output : the headed vs headless choice:
            - --not-headless: watch the browser, useful when debugging a flaky flow
            - headless (CI-shaped): faster, matches what CI does
         + the trade-off and what to watch for during a headed run
         + the composed command, handed back for the user to launch

Role#

Surface the pre-run choice; explain the trade-off.
Compose the command.
Don’t launch it — the user runs it.

Skills on Ocarina — manual

09.03.01 — Review skills

09.03.01 — Review skills#

Listing (potentially non-exhaustive)#

review-dead-code#

review-report#

review-watcher-misuse#

review-comment-drift#

09.03.02 — Analyse skills

09.03.02 — Analyse skills#

Listing (potentially non-exhaustive)#

diagnose-root-cause#

diagnose-flake-root-cause#

analyse-flakiness#

analyse-fixture-flakiness#

analyse-watcher-flakiness#

analyse-screenshot-flakiness#

09.03.03 — Black-hat skills

09.03.03 — Black-hat skills (6)#

Listing (potentially non-exhaustive)#

Perimeter#

09.03.04 — Comprehend skills

09.03.04 — Comprehend skills#

Listing (potentially non-exhaustive)#

understand-ocarina#

09.03.05 — Pick skills

09.03.05 — Pick skills#

Listing (potentially non-exhaustive)#

“mtime”#

09.03.06 — Author skills

09.03.06 — Author skills#

Listing (potentially non-exhaustive)#

empiricism#

09.03.07 — Refactor skills

09.03.07 — Refactor skills#

Listing (potentially non-exhaustive)#

refactor-fragmentation#

09.03.08 — State skill

09.03.08 — State skill#

The skill#

Approach#

09.03.09 — Setup skills

09.03.09 — Setup skills#

The skill#

Step 1: venv#

Step 2: pip install#

Step 3: dev tooling#

Step 4: CLAUDE.local.md#

09.03.10 — Run skills

09.03.10 — Run skills#

Listing (potentially non-exhaustive)#

propose-visual-review#

Role#

`review-dead-code`#

`review-report`#

`review-watcher-misuse`#

`review-comment-drift`#

`diagnose-root-cause`#

`diagnose-flake-root-cause`#

`analyse-flakiness`#

`analyse-fixture-flakiness`#

`analyse-watcher-flakiness`#

`analyse-screenshot-flakiness`#

`understand-ocarina`#

`empiricism`#

`refactor-fragmentation`#

Step 4: `CLAUDE.local.md`#

`propose-visual-review`#