Getting started

Install ghostrun, set up the local judge, and get a passing test in under a minute.

Install

pip install ghostrun

For the default (local, free, private) judge, install Ollama and pull a small model:

ollama pull llama3.2:3b

Tip

If anything doesn't work, ghostrun doctor diagnoses the setup — see Configuration.

Fastest start: ghostrun init

ghostrun init scaffolds a working first test against whatever LLM SDK it finds in your project (OpenAI, Anthropic, or a generic HTTP fallback) plus a .ghostrun.yaml — no config to author by hand:

ghostrun init
pytest test_ghostrun_example.py

Quickstart

# test_customer_support.py
import ghostrun
from my_app import generate_reply

@ghostrun.record(model="gpt-4o-mini")
def test_reply_generation():
    reply = generate_reply("Where is my refund?")

    ghostrun.expect(reply).contains_intent("apology")
    ghostrun.expect(reply).contains_intent("refund policy")
    ghostrun.expect(reply).does_not_contain_intent("arguing")
    ghostrun.expect(reply).tone_is("empathetic")

$ pytest test_customer_support.py
================================ test session starts ================================
collected 1 item

test_customer_support.py .                                                     [100%]
================================ 1 passed in 0.04s =================================

The 0.04s is the whole point — after the first record, calls replay from disk.

Note on the API

The assertion entry point is ghostrun.expect(...), not ghostrun.assert(...) — assert is a reserved Python keyword and cannot be a function name.

Record/replay alone needs no Ollama at all — the judge is only touched when you call a judge-backed assertion (contains_intent, tone_is, matches). Deterministic assertions (contains, is_valid_json) and tool-call assertions never invoke it.

Is this for you?

Use ghostrun if you're writing pytest tests around code that calls an LLM (directly or via the OpenAI/Anthropic SDKs) and want that suite to run offline, free, and deterministically after the first recording.

Skip it if you need a hosted dashboard/observability platform for production traffic (see Langfuse/LangSmith/Braintrust instead), you're building a red-team/adversarial test suite (see Giskard), or you want 50+ pre-built judge metrics out of the box today (see DeepEval — more mature, more metrics, but doesn't intercept your app's own HTTP calls the way ghostrun does). See the comparison research for the full breakdown.

NextRecording and replay