02.05.01 — Test[Driver]#

Source file: src/ocarina/dsl/testing/oc_test.py

Signature#

@final
class Test[Driver]:
    def __init__(
        self,
        *,
        name: TestName,
        test_id: str | None = None,
        test_scenario: TestScenario[Driver],
        pre_test_scenarios_fragments: Sequence[TestScenarioFragment[Driver]] | None = None,
        post_test_scenarios_fragments: Sequence[TestScenarioFragment[Driver]] | None = None,
        skipped: bool = False,
    ) -> None:
        if test_id is None:
            test_id = name
        self.name = name
        self.test_id = test_id
        self._test_scenario = test_scenario
        self._pre_test_scenarios_fragments = pre_test_scenarios_fragments or []
        self._post_test_scenarios_fragments = post_test_scenarios_fragments or []
        self._skipped = skipped

Six parameters:

1. @final#

No user inheritance. Want a “special” test? Compose via fragments or a scenario — don’t subclass.

04.01 — "From the outside like a user" strategy#

The posture lives in the scenario tests’ conftest.py. No test cheats by peeking at internals.

tests/scenarios/conftest.py#

FakeDriver#

@dataclass
class FakeDriver:
    title: str = "fake"
    disposed: bool = False

That’s all. Ocarina’s core calls no Selenium API.

02.10.02 — DriverBuilder[Driver]#

Source file: src/ocarina/infra/driver_builder.py

Encapsulates profile handling (often a tmp copy of a user directory) and produces the (driver, dispose) pair the pool expects.

Why a builder?#

Building a Selenium driver with a profile requires:

1. Copy the profile to a temp folder (otherwise Firefox/Chrome locks the original).
2. Start the driver pointing at the temp folder.
3. At the end: driver.quit(), then delete the temp folder.

DriverBuilder factors out those three steps:

00.03 — End-to-end execution flow of an e2e campaign#

Run python -u src/main.py … on either suite (ocarina-example or ocarina-with-ai-example) and this chain kicks off:

USER  ────────────────────────────────────────────────────────────────────
       python -u src/main.py --browser firefox --workers 3 [...]
              │
              ▼
       (1) parse CLI
           CliStoreSingleton.push(create_selenium_auto_cli_store())
              │
              ▼
       (2) build pool
           create_selenium_drivers_pool(max_size=N)
              │
              ▼
       (3) warm up external dependencies
           - Redis (ocarina-example)
           - Heroku dyno (ai-example, via curl --retry 6)
              │
              ▼
       (4) bootstrap(
              test_cycle  = create_e2e_test_cycle(drivers_pool),
              run_plugins = lambda results: run_plugins(
                                generate_docx_proof, generate_json_results,
                                exceptions_logger=...),
              post_exec   = pretty_print_results + sys.exit(1) on fail
           )
              │
OCARINA  ─────┼──────────────────────────────────────────────────────────
              │
              ▼
       (5) TestCycle.run_all (saturate_workers=True)
              ├─ smoke_tests_campaigns      [mode: fail-fast | wait-for-all]
              │     └─ TestCampaign.run_all
              │           └─ TestSuite.run (max_workers, saturate_workers)
              │                 └─ ThreadPoolExecutor → TestFlow.run
              │                       └─ pool.acquire() → TestExecutor.execute
              │                             ├─ setup()                  (optional)
              │                             ├─ watchers.start()         (daemon threads)
              │                             ├─ chain_runners            (Railway DSL)
              │                             ├─ watchers.stop()
              │                             └─ teardown()               (always)
              ├─ campaigns (main)
              │  (skipped if a smoke failed)
              │
SUT   ────────┼────────────────────────────────────────────────────────────
              │
              ▼
       Selenium WebDriver ⇄ real browser
       sometimes: OTP HTTP GET + Redis (on the ocarina-example side)
              │
              ▼
       (6) run_plugins(results)
              ├─ generate_docx_proof  (reads the log tree, builds .docx files)
              ├─ generate_json_results (serializes results to .json)
              ├─ others if declared (parallelized via ThreadPoolExecutor)
              │
              ▼
       (6') post_exec(results)
              ├─ pretty_print_results (ANSI, hierarchical)
              └─ has_test_cycle_failed → sys.exit(1) if applicable

Step-by-step#

(1) Parse CLI#

create_selenium_auto_cli_store() detects the OS via platform.system():

02.10.03 — Screenshotter[TDriver]#

Source file: src/ocarina/infra/screenshotter.py

Generic, thread-safe screenshot utility. Driver-agnostic via a Protocol, configurable via a dataclass. Supports burst.

Protocol#

class ScreenshotDriver(Protocol):
    def save_screenshot(self, path: str) -> bool:
        """Save standard viewport screenshot."""

That’s it. Any object with save_screenshot(path: str) -> bool qualifies. Selenium WebDriver has it natively (driver.save_screenshot), Playwright can be wrapped, a fake driver in tests exposes it trivially.

ScreenshotterConfig[TDriver]#

@dataclass(frozen=True)
class ScreenshotterConfig[TDriver: ScreenshotDriver]:
    output_dir: Path
    file_ext: str = ".png"
    health_check: HealthCheck[TDriver] | None = None
    save_full_page: SaveFullPageScreenshot[TDriver] | None = None
    default_burst_delay: float = 0.5
    max_filename_retries: int = 500
    uuid_length: int = 8

Immutable. Configure once, ship.

02.11.03 — Auto CLI store + flags + validation#

Source file: src/ocarina/opinionated/cli/selenium/create_cli_store.py

Flags#

_DEFAULT_WORKERS_AMOUNT = 5
_DEFAULT_LOGGER: SupportedLogger = "terminal+file"
_DEFAULT_BROWSER_AUTOMATION_TIMEOUT = 10
_MAX_BROWSER_AUTOMATION_TIMEOUT = 60

Literal[...]#

type SeleniumCliStoreKeys = Literal[
    "driver_path", "profile_path", "browser", "headless", "workers",
    "logger", "wait_timeout", "force_delete_tmp_dirs", "only", "exclude",
]

These keys are the store’s whole vocabulary. Call-sites use them: store.get("workers"), store.get("browser"), etc.

10.04 — ocarina-with-ai-example workflows#

See also ../08-ai-example/09-ci-matrix.md

Overview#

Differences vs ocarina-example#

Heroku warm-up#

curl -sf --retry 6 --retry-delay 5 --retry-all-errors \
  --max-time 30 https://katalon-demo-cura.herokuapp.com/ > /dev/null
echo "Dyno is warm"

CURA runs on a Heroku dyno that sleeps after inactivity. Tests hitting a cold dyno produce timeouts (false fails) and slow confirmations (false passes on gap tests). Wake it explicitly before any browser opens.

01.05 — The project’s political stance#

The Holy Book doesn’t separate philosophy from the politics of the repo itself. Five operating commitments come out of it.

1. Incorruptibility#

Sharing Ocarina means sharing a car I have maintained entirely myself, for myself, therefore with great care. Still, it is shared as-is, and will remain so. It’s my car.

I channeled all my anger into it, to pour all my love into it.

02.10.05 — Selenium adapters#

Source folder: src/ocarina/infra/selenium/

Everything that materializes the abstractions (POMBase, WebDriversPool, Screenshotter) in Selenium flavor. Lives outside the pure DSL — and since 1.1.3 it has a twin: src/ocarina/infra/playwright/ ships the exact same contracts in Playwright flavor. Swapping backend is now picking a folder, not writing one.

Files in the folder#

create_driver.py#

def _build_firefox(*, profile_path, driver_path, headless, wait_timeout) -> WebDriver:
    service = FirefoxService(executable_path=driver_path)
    options = FirefoxOptions()
    if headless:
        options.add_argument("-headless")
    if profile_path:
        options.profile = FirefoxProfile(profile_path)
    driver = Firefox(service=service, options=options)
    driver.implicitly_wait(wait_timeout)
    return driver


def _build_chrome(*, profile_path, driver_path, headless, wait_timeout) -> WebDriver:
    service = ChromeService(executable_path=driver_path)
    options = ChromeOptions()
    if headless:
        options.add_argument("--headless=new")
    if profile_path:
        options.add_argument(f"--user-data-dir={profile_path}")
    driver = Chrome(service=service, options=options)
    driver.implicitly_wait(wait_timeout)
    return driver


def _build_edge(*, profile_path, driver_path, headless, wait_timeout) -> WebDriver:
    # … same as Chrome with EdgeService/EdgeOptions/Edge …


def _build_safari(*, profile_path, driver_path, headless, wait_timeout) -> WebDriver:
    # … Safari supports neither driver_path (uses macOS' native safaridriver),
    #   nor profile_path (no custom profile), nor headless …

1. implicitly_wait set only once#

Each builder calls driver.implicitly_wait(wait_timeout) once. Any find_element that doesn’t find the element immediately waits up to wait_timeout seconds before raising NoSuchElementException.

02.07 — Watcher[Driver]#

Source file: src/ocarina/dsl/testing/watcher.py

Parallelized observer running as a daemon thread alongside the test_chain. Built to catch unpredictable frictions with no direct impact on the scenario that pop up while the scenario runs: random error toasts, parasitic validations, unexpected popups.

The problem#

Holy Book quote (First real-world hurdles):

Even more surprisingly: apps displaying error toasts for no apparent reason, or forms flagging validation errors on inputs that are actually correct, without blocking the journey.