Chapter 02.04 — Invariants#

Ocarina’s sub-DSL for typed, composable invariants. Used everywhere: CLI, POMs, the custom_invariants/testing/ that check suite consistency before any test runs.

Plan#

Why the invariants DSL#

1. Pre-execution guard. TestSuite.__init__ checks — before launching anything — that test names / IDs are unique, max_workers >= 1, etc.
2. CLI validation. Every flag carries its validate=lambda chain: chain.assert_that(...) inside the CliStore. Errors aggregate into one message.
3. POM-side validation. POM actions validate their parameters ("retries must be positive") via validate(retries, name="retries").assert_that(is_positive).execute().raise_if_invalid().

The Holy Book sums it up:

Chapter 02.05 — Orchestration#

Test → TestSuite → TestCampaign → TestCycle: how each level slots into the next, who owns parallelization, who owns retries, who decides skipping, and where the pre-execution invariants live.

Plan#

Diagram#

            ┌────────────────────────────────────────────────┐
            │                  TestCycle                     │
            │                                                │
            │  ┌──────────────────────────────────────────┐  │
            │  │ smoke_tests_campaigns                    │  │ ◄── first, gate
            │  │  └─ TestCampaign                         │  │     mode: fail-fast |
            │  │      └─ TestSuite                        │  │           wait-for-all
            │  │          └─ Test                         │  │
            │  └──────────────────────────────────────────┘  │
            │                                                │
            │  ┌──────────────────────────────────────────┐  │
            │  │ campaigns (main)                         │  │ ◄── skipped if a
            │  │  └─ TestCampaign                         │  │     smoke fails
            │  │      └─ TestSuite                        │  │
            │  │          └─ Test                         │  │
            │  └──────────────────────────────────────────┘  │
            └────────────────────────────────────────────────┘

Who does what?#

The strict separation is deliberate. TestExecutor knows nothing about retries. TestFlow knows nothing about concurrency. TestSuite knows nothing about campaign-level aggregation. One axis of responsibility per class.

04.06 — Property-based testing (hypothesis)#

A single file (test_invariants_properties.py), but the pattern is interesting: we generate inputs and verify universal properties of the invariant predicates.

The concept#

hypothesis auto-generates test cases that satisfy given constraints, then checks a property holds for every generated case. On failure, hypothesis does shrinking to find the smallest counter-example.

from hypothesis import given, strategies as st

@given(st.integers())
def test_is_positive_passes_on_positive(value):
    if value >= 0:
        is_positive(value)             # must not raise
    else:
        with pytest.raises(InvariantViolationError):
            is_positive(value)         # must raise

→ hypothesis generates integers (typically 100 by default), checks the property on each. Unexpected exception → fail. The shrinker finds the smallest failing case (0? -1? MIN_INT?).