02.01 — Ocarina’s technical identity#

pyproject.toml#

[project]
name = "ocarina"
version = "1.1.10"
description = "Websites test framework for Igor"
requires-python = ">=3.14"
authors = [{ name="Igor Casanova", email="[REDACTED]" }]
license = "MIT"
license-files = ["LICEN[CS]E*"]
readme = "README.md"

dependencies = ["python-docx>=1.2.0"]

1. version = "1.1.10": stable 1.x, not a pre-release.
2. requires-python = ">=3.14": PEP 695 generic typing everywhere.
3. dependencies = ["python-docx>=1.2.0"]: one runtime dependency. The rest lives in dev.
4. license = "MIT".
5. description = "Websites test framework for Igor": not “for everyone,” not “for humans,” explicitly “for Igor.” Lines up with ../01-philosophy/05-political-stance.md: “It’s my car.”

dev dependencies#

[dependency-groups]
dev = [
    "ruff>=0.15.0",
    "mypy>=1.17.0",
    "mypy-extensions>=1.1.0",
    "typing-extensions>=4.14.0",
    "pytest>=9.0.0",
    "pytest-cov>=7.0.0",
    "hypothesis>=6.151.0",
    "pytest-mypy-plugins>=3.1.0",
    "allure-pytest>=2.15.3",
    "allure-python-commons>=2.15.3",
    "pre-commit>=4.5.1",
    "syrupy>=5.1.0",
    "selenium>=4.40.0",
    "playwright>=1.60.0",
    "twine>=6.2.0",
    "build>=1.4.2",
    "prysk>=0.20.0",
]

mypy.ini#

[mypy]
python_version = 3.14
strict = true

# * ... Allow missing annotations (type inference is cool)
disallow_incomplete_defs = false

# * ... Allow missing annotations (type inference is cool)
disallow_untyped_defs = false

• strict = true: enables --warn-redundant-casts, --warn-unused-ignores, --no-implicit-optional, --check-untyped-defs, etc.
• Two exceptions: disallow_incomplete_defs and disallow_untyped_defs are off. mypy’s type inference is good enough when explicit annotations don’t add value. Annotations live where they actually matter.

pyproject.toml#tool.ruff#

[tool.ruff.lint]
select = ["ALL"]
ignore = [
    "ANN002", "ANN003", "ANN201",
    "TRY003",
    "C901",
    "D203",    # conflict with D211
    "D213",    # conflict with D212
    "COM812",  # conflict with the ruff formatter (auto-fix)
]

[tool.ruff]
exclude = ["**/.venv/**", "**/bin/**", "**/__init__.py", "**/__bypass_linter__"]

• select = ["ALL"]: every ruff rule on (~800).
• Short ignore list: six rules, each one justified.
• B008 must NEVER be ignored ("# "B008", # * ... NEVER ignore this rule without knowing very well what you are doing: https://docs.astral.sh/ruff/rules/function-call-in-default-argument/"). Note left so the past mistake doesn’t repeat.
• ****/**bypass_linter****: directory convention for explicitly un-linted code (handy for internal experimental modules).

pyproject.toml#tool.pytest.ini_options#

testpaths = ["tests"]
python_files = ["test_*.py"]
norecursedirs = [".*", "__pycache__"]
log_cli = true
log_cli_level = "DEBUG"
log_cli_format = "%(asctime)s [%(levelname)s] %(name)s: %(message)s"
log_cli_date_format = "%Y-%m-%d %H:%M:%S"
addopts = """
--cov=src
--cov-branch
--cov-report=term-missing
--cov-report=html
--cov-report=xml:coverage.xml
"""

pyproject.toml#tool.coverage#

Full detail at ../04-internal-tests/07-coverage-policy.md. Short version: anything that’s pure shape (custom_types, ports, errors), anything that needs a real browser (Selenium adapters), and anything covered by cram tests (cli/store, cli/builder) is explicitly omitted. Otherwise the metrics lie.

02.02 — The ocarina module tree#

The Python source module ships 134 .py files across four conceptual layers.

src/ocarina/
├── __init__.py                              # empty — no global "facade", everything is imported precisely
├── py.typed                                 # PEP 561: declares the package as "typed"
│
├── railway/                                 # Layer 1 — FP primitives
│   ├── __init__.py
│   └── result.py                            # Ok[T] / Fail / Result[T] / is_ok / is_fail
│
├── custom_types/                            # Layer 2 — types and aliases
│   ├── __init__.py
│   ├── effect.py                            # type Effect = Callable[[], None]
│   ├── thunk.py                             # type Thunk[T] = Callable[[], T]
│   ├── tpom.py                              # TypeVar TPOM bound POMBase
│   ├── supports_write.py                    # Protocol SupportsWrite[T], mirrors a type not exposed by Python stdlib
│   ├── built_web_driver.py                  # type BuiltWebDriver[Driver] = tuple[Driver, Effect]
│   ├── scenario.py                          # Scenario[Driver] (frozen dataclass)
│   ├── test_components.py                   # TestChain / TestSetup / TestTeardown / TestWatchers
│   ├── test_runner.py                       # TestRunner[Driver]
│   ├── oc_test.py                           # TestName / TestScenario / TestScenarioFragment
│   ├── oc_test_layers.py                    # TestResult / TestSuiteResult / TestCampaignResults / TestCycleResults
│   ├── selenium/
│   │   ├── built_web_driver.py              # BuiltSeleniumWebDriver
│   │   ├── oc_test_scenario.py
│   │   ├── supported_browsers.py            # type SupportedSeleniumBrowser = Literal["chrome", "firefox", "edge", "safari"]
│   │   └── web_drivers_pool.py              # type SeleniumWebDriversPool = WebDriversPool[WebDriver]
│   └── playwright/
│       ├── built_web_driver.py              # type BuiltPlaywrightDriver = BuiltWebDriver[PlaywrightDriver]
│       ├── oc_test_scenario.py              # PlaywrightTestScenario / PlaywrightTestScenarioFragment
│       ├── supported_browsers.py            # type SupportedPlaywrightBrowser = Literal["chromium", "firefox", "webkit"]
│       └── web_drivers_pool.py              # type PlaywrightDriversPool = WebDriversPool[PlaywrightDriver]
│
├── custom_errors/                           # Layer 2: errors
│   ├── __init__.py
│   └── test_framework/
│       ├── __init__.py
│       ├── no_matching_branch.py            # NoMatchingBranchError
│       ├── pages.py                         # PageVerificationError
│       ├── driver_died.py                   # DriverDiedError
│       └── campaigns.py
│
├── custom_invariants/                       # Layer 2: ready-to-use invariants
│   ├── __init__.py
│   └── testing/
│       ├── __init__.py
│       ├── workers.py                       # validate_workers_amount
│       ├── oc_test_runners_ids.py           # validate_test_runners_ids
│       ├── oc_test_runners_names.py         # validate_test_runners_names (unique + valid filename)
│       ├── oc_test_suites_names.py
│       ├── oc_test_campaigns_names.py
│       └── oc_test_cycles_names.py
│
├── ports/                                   # Layer 3: ports (interfaces)
│   ├── __init__.py
│   ├── ilogger.py                           # ILogger (ABC)
│   └── itake_screenshot.py                  # ITakeScreenshot[Driver] (Protocol)
│
├── pom/                                     # Layer 3: Page Object Model
│   ├── __init__.py
│   ├── base.py                              # POMBase (ABC: verify + get_current_title)
│   ├── selenium/
│   │   ├── __init__.py
│   │   └── muted.py                         # MutedPOM utility
│   └── playwright/
│       ├── __init__.py
│       └── muted.py                         # MutedPlaywrightPOM utility
│
├── aggregates/                              # Layer 3: result aggregates
│   ├── __init__.py
│   └── tests_layers.py                      # is_test_result_ok / _fail / _skipped (TypeGuard)
│
├── dsl/                                     # Layer 4: Ocarina DSL
│   ├── __init__.py
│   ├── invariants/
│   │   ├── __init__.py
│   │   ├── validate.py                      # entry-point validate(), BusinessInvariantValidator, FrameworkInvariantValidator
│   │   ├── assertions.py                    # is_str / is_email / is_positive / is_in / has_unique_elements / each / ...
│   │   ├── errors.py                        # InvariantViolationError / DuplicatesError / AggregateInvariantViolationError
│   │   └── internals/
│   │       ├── __init__.py
│   │       └── validation_chain.py          # ValidationStartBlock / ValidationAssertBlock / _ValidationChain / _any_of / chain_validations
│   ├── testing/
│   │   ├── __init__.py
│   │   ├── oc_test.py                       # class Test[Driver]
│   │   ├── oc_test_suite.py                 # class TestSuite[Driver]
│   │   ├── oc_test_campaign.py              # class TestCampaign[Driver] + campaign_has_failed
│   │   ├── oc_test_cycle.py                 # class TestCycle[Driver] + type Mode + has_test_cycle_failed
│   │   ├── filter_tests_by_ids.py
│   │   ├── watcher.py                       # class Watcher[Driver]
│   │   ├── internals/
│   │   │   ├── __init__.py
│   │   │   ├── test_executor.py             # class TestExecutor[Driver] + ExecutionOutcome (frozen, slots)
│   │   │   └── test_flow.py                 # class TestFlow[Driver]
│   │   ├── selenium/
│   │   │   ├── __init__.py
│   │   │   ├── create_test.py               # create_selenium_test(...)
│   │   │   └── create_watcher.py            # create_selenium_watcher(...) + type SeleniumWatcher
│   │   └── playwright/
│   │       ├── __init__.py
│   │       ├── create_test.py               # create_playwright_test(...)
│   │       └── create_watcher.py            # create_playwright_watcher(...) + type PlaywrightWatcher
│   └── testing_with_railway/
│       ├── __init__.py
│       ├── chain_actions.py                 # class ChainRunner[T] + chain_actions
│       ├── match_page.py                    # When + _match_page_builder + create_match_page
│       ├── constructors/
│       │   ├── __init__.py
│       │   └── create_act.py                # low-level primitive (to be wrapped on the project side)
│       └── internals/
│           ├── __init__.py
│           └── action_chain.py              # ActionStart → ActionFailure → ActionSuccess → ActionChain
│                                            # + Neutral{Start,Failure,Success}  (failure rail)
│
├── infra/                                   # Layer 5: infrastructure
│   ├── __init__.py
│   ├── drivers_pool.py                      # class WebDriversPool[Driver] + WarmupTimeoutError
│   ├── driver_builder.py                    # class DriverBuilder[Driver]
│   ├── screenshotter.py                     # class Screenshotter[TDriver] + ScreenshotterConfig
│   ├── act_counter.py                       # class ActCounter (interface)
│   ├── selenium/
│   │   ├── __init__.py
│   │   ├── create_driver.py                 # _build_firefox / _build_chrome / _build_edge / _build_safari
│   │   ├── create_drivers_pool.py           # create_selenium_drivers_pool
│   │   ├── create_screenshotter.py          # create_selenium_screenshotter
│   │   ├── driver_healthcheck.py            # driver_healthcheck (pings driver.title)
│   │   └── mixins.py                        # SeleniumTitleMixin (typing keyway)
│   └── playwright/
│       ├── __init__.py
│       ├── create_driver.py                 # create_playwright_driver (chromium / firefox / webkit)
│       ├── create_drivers_pool.py           # create_playwright_drivers_pool
│       ├── create_screenshotter.py          # create_playwright_screenshotter + _playwright_save_full_page
│       ├── driver.py                        # class PlaywrightDriver (sync wrapper)
│       ├── driver_healthcheck.py            # playwright_driver_healthcheck (pings driver.title)
│       └── mixins.py                        # PlaywrightTitleMixin (typing keyway)
│
└── opinionated/                             # Layer 6: opt-in, everything that is "nice but replaceable"
    ├── __init__.py
    ├── consts/loggers_choices.py            # LOGGERS_CHOICES = ("terminal", "file", "terminal+file", "muted")
    ├── infra/
    │   ├── __init__.py
    │   └── act_counter.py                   # ThreadsBasedActCounter (thread-local counter)
    ├── cli/
    │   ├── __init__.py
    │   ├── builder.py                       # CliBuilder + CliArg + _SilentArgumentParser
    │   ├── store.py                         # CliStore[TKeys] + _CliField[T] + field(...)
    │   ├── phantoms.py                      # phantom_validate (no-op predicate)
    │   ├── selenium/
    │   │   ├── __init__.py
    │   │   ├── cli_store_singleton.py       # SeleniumCliStoreSingleton (push / get)
    │   │   └── create_cli_store.py          # create_selenium_{auto,win,macos,linux}_cli_store
    │   └── playwright/
    │       ├── __init__.py
    │       ├── cli_store_singleton.py       # PlaywrightCliStoreSingleton (push / get)
    │       └── create_cli_store.py          # create_playwright_{,auto_}cli_store
    ├── dsl/
    │   ├── __init__.py
    │   └── drive_page.py                    # drive_page = chain_actions, a semantic alias
    ├── launcher/
    │   ├── __init__.py
    │   └── bootstrap.py                     # bootstrap(...) + run_plugins(*plugins, exceptions_logger)
    ├── loggers/
    │   ├── __init__.py
    │   ├── create_matching_logger.py        # create_matching_logger("terminal"|"file"|...) + get_default_log_dir
    │   ├── print_logger.py                  # PrintLogger (ANSI)
    │   ├── file_logger.py                   # FileLogger (taxonomy -> file tree)
    │   ├── print_and_file_logger.py
    │   ├── muted_logger.py
    │   ├── custom_types/supported_loggers.py
    │   └── utils/
    │       ├── __init__.py
    │       └── format_metadata_str.py       # format_utc_date_metadata_str / format_current_thread_metadata_str / concat_metadata
    └── plugins/
        ├── __init__.py
        └── reports/
            ├── __init__.py
            ├── pretty_print_results.py      # hierarchical ANSI report
            ├── results_to_json.py           # JSON generator
            ├── docx_tests_proofs.py         # DOCX generator (consumes the log tree)
            └── timing.py                    # context manager `with timing(prefix="Duration:")`

Layered diagram#

┌──────────────────────────────────────────────────────────────────────────┐
│ USER PROJECT (e.g. ocarina-example, ocarina-with-ai-example)             │
│  ┌────────────────────┐  ┌────────────────────┐  ┌─────────────────────┐ │
│  │ pages/ (POMBase)   │  │ scenarios/ (Test)  │  │ adapters/ (act, ...)│ │
│  └────────────────────┘  └────────────────────┘  └─────────────────────┘ │
└─────────────────────────────────┬────────────────────────────────────────┘
                                  │ imports
┌─────────────────────────────────▼────────────────────────────────────────┐
│ LAYER 6 — opinionated/  (CLI, loggers, plugins, bootstrap)               │
│   Opt-in. Can be replaced wholesale.                                     │
└─────────────────────────────────┬────────────────────────────────────────┘
                                  │
┌─────────────────────────────────▼────────────────────────────────────────┐
│ LAYER 5 — infra/  (pool, builder, screenshotter, Selenium adapters)      │
│   I/O. No DSL logic.                                                     │
└─────────────────────────────────┬────────────────────────────────────────┘
                                  │
┌─────────────────────────────────▼────────────────────────────────────────┐
│ LAYER 4 — dsl/  (testing, invariants, testing_with_railway)              │
│   Pure DSL, no I/O. All the grammar.                                     │
└─────────────────────────────────┬────────────────────────────────────────┘
                                  │
┌─────────────────────────────────▼────────────────────────────────────────┐
│ LAYER 3 — ports/  +  pom/  +  aggregates/                                │
│   Interfaces and abstractions.                                           │
└─────────────────────────────────┬────────────────────────────────────────┘
                                  │
┌─────────────────────────────────▼────────────────────────────────────────┐
│ LAYER 2 — custom_types/  +  custom_errors/  +  custom_invariants/        │
│   Shapes, aliases, types, exceptions, ready-to-use invariants.           │
└─────────────────────────────────┬────────────────────────────────────────┘
                                  │
┌─────────────────────────────────▼────────────────────────────────────────┐
│ LAYER 1 — railway/  (Result, Ok, Fail, is_ok, is_fail)                   │
│   The functional primitive. No dependencies.                             │
└──────────────────────────────────────────────────────────────────────────┘

This strict stratification rules out import cycles and lets you swap any upper layer without touching the ones below:

02.06 — Scenario[Driver]#

Source file: src/ocarina/custom_types/scenario.py

Dataclass#

@final
@dataclass(frozen=True)
class Scenario[Driver]:
    test_chain: TestChain
    setup: TestSetup = field(default=None)
    teardown: TestTeardown = field(default=None)
    watchers: TestWatchers[Driver] | None = field(default=None)

# src/ocarina/custom_types/test_components.py
type TestChain = Sequence[ChainRunner[Any]]
type TestSetup = Effect | None
type TestTeardown = Effect | None
type TestWatchers[Driver] = Sequence[Watcher[Driver]] | None

Lifecycle (per attempt)#

1. setup()           — optional, free-form Effect (DB, API, …)
       → raises    :  test_chain skipped, jump to teardown,
                      return Outcome(setup_failed=True, should_retry=True)
       → ok        :  continue to test_chain

2. test_chain        — the actual chain (Sequence[ChainRunner])
       (watchers run during this time)

3. teardown()        — optional, always executed
       → raises    :  log warning + ignore (does not affect the verdict)

If EVERY attempt raises during setup:
    → test marked SKIPPED (not FAILED)
    → warning log « setup keeps failing »

setup and teardown#

setup and teardown are driver-free and injection-free by design.

02.07 — Watcher[Driver]#

Source file: src/ocarina/dsl/testing/watcher.py

Parallelized observer running as a daemon thread alongside the test_chain. Built to catch unpredictable frictions with no direct impact on the scenario that pop up while the scenario runs: random error toasts, parasitic validations, unexpected popups.

The problem#

Holy Book quote (First real-world hurdles):

Even more surprisingly: apps displaying error toasts for no apparent reason, or forms flagging validation errors on inputs that are actually correct, without blocking the journey.

02.08 — POMBase#

Source file: src/ocarina/pom/base.py

Framework-agnostic abstract base for the Page Object Model. Two required methods. Zero mention of Selenium.

Code#

class POMBase(ABC):
    @abstractmethod
    def verify(self, *, timeout: float | None = None) -> Self:
        ...

    @abstractmethod
    def get_current_title(self) -> str:
        ...

Six bytes of contract (the two signatures). That’s all.

Why two methods#

verify(timeout=None) -> Self#

Purpose: prove we’re on the right page. Otherwise, raise.

• timeout lets you wait up to a given time for the page to load (handy for Selenium WebDriverWait).
• None default lets the implementation decide (typically reading get_timeout() which reads the CLI).
• Returning Self enables fluent method chaining (MyPage(...).open().verify().click()).

The only method test-side that’s truly needed to guarantee “I’m on the page I want.” Everything else (click_xxx, enter_xxx) is on the subclass.

02.09 — Ports: ILogger, ITakeScreenshot#

Source folder: src/ocarina/ports/

Two ports. That’s it. Everything else (WebDriversPool, Screenshotter, etc.) lives under infra/, not ports/. The line is sharp: a port is an abstraction the DSL sits on; an infra is the implementation of adapters.

ILogger#

class ILogger(ABC):
    @abstractmethod
    def set_prefix(self, prefix_thunk: Thunk[str]) -> Self: ...

    @abstractmethod
    def set_domain_taxonomy(self, taxonomy: tuple[str, ...]) -> Self: ...

    @abstractmethod
    def raw(self, *args: object, stream: SupportsWrite[str] | None = None, **kwargs: object) -> None: ...

    @abstractmethod
    def critical(self, msg: str, *args, exc: Exception | None = None, **kwargs) -> None: ...

    @abstractmethod
    def error(self, msg: str, *args, exc: Exception | None = None, **kwargs) -> None: ...

    @abstractmethod
    def warning(self, msg: str, *args, exc: Exception | None = None, **kwargs) -> None: ...

    @abstractmethod
    def info(self, msg: str, *args, exc: Exception | None = None, **kwargs) -> None: ...

    @abstractmethod
    def debug(self, msg: str, *args, exc: Exception | None = None, **kwargs) -> None: ...

    @abstractmethod
    def test_name(self, msg: str, *args, exc: Exception | None = None, **kwargs) -> None: ...

    @abstractmethod
    def success(self, msg: str, *args, exc: Exception | None = None, **kwargs) -> None: ...

    @abstractmethod
    def exception(self, msg: str, *args, exc: Exception | None = None, **kwargs) -> None: ...

    @abstractmethod
    def cleanup(self) -> None: ...

Three things that are not in ILogger#

• No set_level(): level handling is the implementation’s call (e.g., MutedLogger filters everything).
• No add_handler(): no logging.Logger-style stdlib API. Composition happens by instantiating different loggers (PrintAndFileLogger wraps both).
• No child(name): taxonomy is passed as a block via set_domain_taxonomy.

set_prefix(thunk)#

logger.set_prefix(lambda: f"[{datetime.now().isoformat()}]")

A str would be computed at set_prefix time and frozen. A Thunk recomputes the prefix on every log call. Perfect for timestamps.

02.12 — Custom types & custom errors#

Ocarina’s shape layer. No logic — types, aliases, exceptions, protocols. What makes the DSL typed without adding runtime logic.

custom_types/#

All these files are excluded from coverage (pyproject.toml#tool.coverage):