AGSelected work
Case study 05

Developer tooling · 2026

Mock AI API

Test the failure paths before your users find them.

BUILT BYAGPublished on PyPI
Product surfaceIllustrated from the working application
zsh — mock-ai-api

$ pipx install mock-ai-api && mock-ai-api

OpenAI-compatible server · http://127.0.0.1:8011

Anthropic-compatible server · http://127.0.0.1:8012

POST /v1/chat/completions gpt-4.1:mock-tool-call 200

POST /v1/messages claude-sonnet-4:mock-anthropic-stream 200 · SSE

POST /v1/chat/completions gpt-4.1:mock-bad-tool-json 200

POST /v1/chat/completions gpt-4.1:mock-429 429

POST /v1/messages claude-sonnet-4:mock-stream-error error event

POST /v1/chat/completions gpt-4.1:mock-503 503

POST /v1/messages claude-sonnet-4:mock-tool-use 200

POST /v1/chat/completions gpt-4.1:mock-401 401

GET /v1/models 200

test_retries.py
client = OpenAI(
    base_url="http://127.0.0.1:8011/v1",
    api_key="mock",
)

def test_backs_off_on_rate_limit():
    with pytest.raises(RateLimitError):
        client.chat.completions.create(
            model="gpt-4.1:mock-429",
            messages=[{"role": "user", "content": "hi"}],
        )

✓ 48 passed in 0.62s · 0 tokens spent

YEAR

2026

ROLE

Creator · Product & engineering

STATUS

Published on PyPI

SOURCEView repository ↗

01 / Context

The challenge

AI integrations are expensive and unpredictable to test. Real providers introduce network latency, changing responses, rate limits, and usage costs into test suites that need to be fast and repeatable.

The response

I built a deterministic local server that speaks OpenAI and Anthropic-compatible APIs. Teams can exercise streaming, tools, malformed JSON, and provider errors through the official SDKs without calling a real model.

02 / What it does

Built around the
real work.

  1. 01

    Deterministic OpenAI and Anthropic responses

  2. 02

    Streaming, tool calls, and malformed payload scenarios

  3. 03

    Provider-shaped 400, 401, 429, 500, and 503 failures

  4. 04

    Installable CLI with official SDK compatibility tests

03 / Built with

PythonFastAPIOpenAI SDKAnthropic SDK

04 / Read more

The happy path is the least interesting thing to testReal AI providers make it hard to test rate limits, malformed tool calls, and broken streams. So I built Mock AI API, a local stand-in that fails on purpose.
Next case study · 06CharGen