Skip to main content
Version: MVP

Development Checks

4 min readFor contributorsUpdated 2026-05-19

What you'll do

Run the same quality gates locally that Craik runs in CI on every pull request and push to main. By the end you'll have a tight inner loop: tests, linting, type-checking, and the policy contract — all reproducible.

Prerequisites​

  • Python 3.12 or newer (python3 --version).
  • A local checkout of the Craik repo with the dev extras installed.
  • uv recommended (but not required) for reproducible, hermetic command runs.
One-time setup
git clone https://github.com/eidetic-labs/craik.git
cd craik
python3 -m pip install -e ".[dev]"

The four core gates​

pytest

Unit, contract, and integration tests. Covers runtime contracts, case-file assembly, policy gates, receipt redaction, handoff structure, and provider adapters.

ruff check

Lints style, imports, and a curated set of correctness rules. Runs in under a second on the full repo.

mypy

Strict type-checking. Craik runtime contracts are Pydantic models — keep them honest.

craik policy test

Exercises the policy contract: strict defaults, immutable path behavior, memory proposal defaults, fail-open receipts, automation fail-closed behavior, and redaction.

uv run resolves a per-invocation Python environment, so the command runs the same way locally as in CI.

The full pre-PR sweep
uv run --python 3.12 --extra dev pytest
uv run --python 3.12 --extra dev ruff check .
uv run --python 3.12 --extra dev mypy
uv run --python 3.12 --extra dev craik policy test

Run them with the venv you already have​

If you installed the dev extras into a virtual environment, just call the tools directly:

From an activated venv
pytest
ruff check .
mypy
craik policy test

Useful narrower runs​

pytest -k case_file

Run only the case-file-related tests.

pytest tests/test_docs.py -x

Run just the docs tests (link resolution, required coverage). Stops on first failure.

ruff check src/craik/runtime

Lint just one subtree — useful when iterating on a module.

mypy src/craik/policy

Type-check one package at a time.

What CI runs​

The CI workflow (.github/workflows/ci.yml) runs the same four gates plus a quickstart smoke test (python scripts/quickstart_smoke.py) and the Docusaurus build (docs/). If your local sweep is green, the chances of CI surprising you on a vanilla change are low.

What to do when a gate fails​

Tests failing

  • Re-run with pytest -x -vvv to stop at first failure with full output.
  • Use pytest --lf after a fix to re-run only the last failures.
  • Snapshot mismatches in contract tests usually mean a schema changed — update the matching fixture deliberately.

Lint / type failing

  • ruff check . --fix handles the autofixable cases.
  • mypy errors that look mysterious are almost always a missing or wrong type hint on a runtime contract — start there.
  • Don't disable a rule to make CI green without a comment justifying why.

What's next​

DiagnoseRun doctorHealth-check identity, credentials, and provider posture without firing a real run.OperateRelease managementHow releases are cut, signed, and gated for v0.x.ReferencePolicy testsWhat craik policy test actually checks and how to extend it.