Invariants on Ocarina — manual

Chapter 02.04 — Invariants

Wed, 20 May 2026 00:00:00 +0000

Chapter 02.04 — Invariants#

Ocarina’s sub-DSL for typed, composable invariants. Used everywhere: CLI, POMs, the custom_invariants/testing/ that check suite consistency before any test runs.

Plan#

#	File	Subject
01	`01-validate-flow.md`	The full flow `validate(value) → assert_that → execute → raise_if_invalid` + the `_ValidationChain` class.
02	`02-assertions.md`	Catalog of builtin assertions (`is_str`, `is_email`, `is_positive`, `is_in`, `has_unique_elements`, `each`, `is_valid_filename`, …).
03	`03-otherwise-any-of.md`	`.otherwise(...)` and the OR combination via `_any_of`.
04	`04-then-chain-of-validations.md`	`.then(new_value)` and `chain_validations(...)` for composition.
05	`05-business-vs-framework-validator.md`	`BusinessInvariantValidator` vs `FrameworkInvariantValidator`
06	`06-invariant-errors.md`	`InvariantViolationError`, `DuplicatesError`, `AggregateInvariantViolationError`.

Why the invariants DSL#

Pre-execution guard. TestSuite.__init__ checks — before launching anything — that test names / IDs are unique, max_workers >= 1, etc.
CLI validation. Every flag carries its validate=lambda chain: chain.assert_that(...) inside the CliStore. Errors aggregate into one message.
POM-side validation. POM actions validate their parameters ("retries must be positive") via validate(retries, name="retries").assert_that(is_positive).execute().raise_if_invalid().

The Holy Book sums it up:

Chapter 02.05 — Orchestration

Wed, 20 May 2026 00:00:00 +0000

Chapter 02.05 — Orchestration#

Test → TestSuite → TestCampaign → TestCycle: how each level slots into the next, who owns parallelization, who owns retries, who decides skipping, and where the pre-execution invariants live.

Plan#

#	File	Subject
01	`01-test.md`	The `Test[Driver]` class — `spawn`, pre/post fragments, skip.
02	`02-test-executor.md`	`TestExecutor` — execution of one attempt, order: setup → watchers.start → chain → watchers.stop → teardown.
03	`03-test-flow-retries.md`	`TestFlow` — retry loop (1 + max_retries), linear backoff, setup-failure handling.
04	`04-test-suite.md`	`TestSuite` — parallelization with `ThreadPoolExecutor`, ID filtering, guardrails.
05	`05-saturation.md`	Worker saturation: random cloning `[COPY N]`. Why and how.
06	`06-test-campaign.md`	`TestCampaign` — sequence of suites, `campaign_has_failed`.
07	`07-test-cycle-modes.md`	`TestCycle` — smoke + main, `fail-fast` vs `wait-for-all` modes, `has_test_cycle_failed`.
08	`08-filter-tests-by-ids.md`	`filter_tests_by_ids` — `--only`/`--exclude`, mutex, ignores unknown IDs.

Diagram#

            ┌────────────────────────────────────────────────┐
            │                  TestCycle                     │
            │                                                │
            │  ┌──────────────────────────────────────────┐  │
            │  │ smoke_tests_campaigns                    │  │ ◄── first, gate
            │  │  └─ TestCampaign                         │  │     mode: fail-fast |
            │  │      └─ TestSuite                        │  │           wait-for-all
            │  │          └─ Test                         │  │
            │  └──────────────────────────────────────────┘  │
            │                                                │
            │  ┌──────────────────────────────────────────┐  │
            │  │ campaigns (main)                         │  │ ◄── skipped if a
            │  │  └─ TestCampaign                         │  │     smoke fails
            │  │      └─ TestSuite                        │  │
            │  │          └─ Test                         │  │
            │  └──────────────────────────────────────────┘  │
            └────────────────────────────────────────────────┘

Who does what?#

Level	Single responsibility	Concurrency	Out of scope
`Test`	Metadata + `spawn(driver, logger)`	none	execution
`TestExecutor`	One attempt	none	retries, driver acquisition
`TestFlow`	Retry loop	none	parallelization, aggregation
`TestSuite`	Parallelization + saturation + ID filtering	N threads	inter-suite sequence
`TestCampaign`	Sequence of suites	suite-level	smoke vs main
`TestCycle`	Smoke + main + mode	campaign-level	bootstrap / post-exec plugins

The strict separation is deliberate. TestExecutor knows nothing about retries. TestFlow knows nothing about concurrency. TestSuite knows nothing about campaign-level aggregation. One axis of responsibility per class.

04.06 — Property-based testing (hypothesis)

Wed, 20 May 2026 00:00:00 +0000

04.06 — Property-based testing (`hypothesis`)#

A single file (test_invariants_properties.py), but the pattern is interesting: we generate inputs and verify universal properties of the invariant predicates.

The concept#

hypothesis auto-generates test cases that satisfy given constraints, then checks a property holds for every generated case. On failure, hypothesis does shrinking to find the smallest counter-example.

from hypothesis import given, strategies as st

@given(st.integers())
def test_is_positive_passes_on_positive(value):
    if value >= 0:
        is_positive(value)             # must not raise
    else:
        with pytest.raises(InvariantViolationError):
            is_positive(value)         # must raise

→ hypothesis generates integers (typically 100 by default), checks the property on each. Unexpected exception → fail. The shrinker finds the smallest failing case (0? -1? MIN_INT?).

Invariants on Ocarina — manual

Chapter 02.04 — Invariants

Chapter 02.04 — Invariants#

Plan#

Why the invariants DSL#

Chapter 02.05 — Orchestration

Chapter 02.05 — Orchestration#

Plan#

Diagram#

Who does what?#

04.06 — Property-based testing (hypothesis)

04.06 — Property-based testing (hypothesis)#

The concept#

04.06 — Property-based testing (`hypothesis`)#