Getting Started

Prerequisites

  • Python 3.12+
  • uv — the recommended package runner (handles all dependency installation automatically)
  • An LLM API key — Gemini is the default, but OpenAI and local models are also supported (see below)

Quick Start

# 1. Install pytifex
pip install pytifex

# 2. Set your API key (choose one provider)
export GEMINI_API_KEY=your_key      # Google Gemini (default)
# export OPENAI_API_KEY=your_key    # OpenAI
# (no key needed for local models)

# 3. Run the full pipeline
uv run pytifex

That’s it. uv automatically installs all dependencies (including all four type checkers) on first run.

Choosing an LLM Provider

Google Gemini (default)

export GEMINI_API_KEY=your_key
uv run pytifex                             # uses gemini-2.5-flash by default
uv run pytifex --model gemini-2.5-pro      # more capable model

Get a Gemini API key at aistudio.google.com/apikey.

OpenAI

export OPENAI_API_KEY=your_key
uv run pytifex --model gpt-4o --openai-base-url https://api.openai.com/v1

Local Model (llama-server, Ollama, vLLM)

Any server that exposes an OpenAI-compatible /v1/chat/completions endpoint works. No API key is needed for local servers.

# Start your local server first (example: llama.cpp llama-server)
# llama-server -m qwen3-6b.gguf --port 8080

uv run pytifex --model qwen3-6b --openai-base-url http://localhost:8080/v1

Cohere

export COHERE_API_KEY=your_key
uv run pytifex --model command-a-03-2025

What Happens When You Run It

  1. Seed mining — Pytifex fetches closed bug reports from mypy, pyrefly, and ty GitHub repositories
  2. Code generation — Seeds are sent to the LLM, which generates new code variations targeting similar edge cases
  3. Type checking — Each generated example is run through mypy, pyrefly, zuban, and ty
  4. Filtering — Only examples where checkers disagree are kept
  5. Evaluation — A two-tier objective system determines which checker is correct using runtime evidence only

Output is saved to generated_examples/<timestamp>/.

Optional: GitHub Token

Without a token, GitHub API requests are limited to 60/hour. For heavier usage:

export GITHUB_TOKEN=ghp_your_token    # raises limit to 5,000/hour

Or skip GitHub entirely:

uv run pytifex --no-github

Next Steps

  • Installation — Detailed setup including manual installation
  • Usage — Full CLI reference, commands, and options
  • Evaluation — Understand how the two-tier objective evaluation works
  • Architecture — Understand how the pipeline works under the hood