Cannonade

Local-First LLM Testing

Run one test suite across many models

Create your test cases once, run them against every AI model you're evaluating, and compare their pass rate, score, cost, and speed.

DownloadDocs
  • Windows, macOS, Linux
  • Free and open source
  • Runs on your machine
Cannonade test run details: three models scored against an LLM rubric suite

Use cases

Decisions it helps you make

A set of test cases costs you an afternoon to write, then pays it back every time you run it again: against the next model, the next prompt, the next provider.

Prompt engineering

Prompts version themselves on every save, so the next run says whether the new wording actually scored better.

Comparing local models

A suite built around the work you need done, with every candidate running it together on your own hardware.

Model discovery

A new model inherits the suite you already trust, so you get its numbers on your own cases the day it ships.

Regression checks

Nothing tells you when a model or a prompt starts behaving differently. Re-running the suite does.

Sampling settings

Temperature, top_p, and the rest are usually set once and never looked at again. Run the same cases at different values and compare the scores.

Cost calculation

Your smallest, largest, and typical inputs as cases. The run reports what they cost, so there is nothing to estimate.

Cost optimization

Does a cheaper model clear the same bar? Run the suite and check the pass rate and the cost.

Provider comparison

The same model on two providers can differ in speed, price, and output. Run the same cases on both and see which one to pick.

Local model tuning

Quantization and runtime settings change how good the answers are. A test run measures the difference instead of leaving it to guesswork.

How it works

What a run looks like

  1. Add a provider

    Point Cannonade at a local server or a cloud gateway.

  2. Write the cases

    A case is an input, an optional system prompt, and the evaluators that score the answer.

  3. Start a run

    Pick as many models as you want.

  4. Check the result

    See what each model answered, what it scored, and how fast it was.

Features

What is in the app

Run results

Results arrive while the run is still going, and stay on disk once it finishes.

  • Pass rate and score per model
  • Tokens, speed, and cost
A finished test case in a run: system prompt, model output, evaluator score, and token stats

Playground

A chat view for trying a model before writing cases for it.

  • Any configured model
  • Sampling parameters per message
  • Stats on every response
The playground chat view with a local model selected and response stats above the reply

Prompt library

System prompts live in one place, so one edit reaches every case using them.

  • Shared by suites and the playground
  • Versioned
The prompt library listing shared system prompts

Model management

Handle local models without switching to another app.

  • Load, unload, download or delete them
The models view listing downloaded local models

Providers

Bring the models you already run

A provider is whatever serves the models: a local server on your machine or network, or a cloud gateway behind an API key. Add one and its models appear in the app. A run targets a single provider and any number of its models.

  • LM Studio
  • Ollama
  • llama.cpp
  • OpenRouter
  • Vercel AI
  • and more

Download

Get Cannonade

Free and open source.

Windows.exe installerWindows 10 and later, x64DownloadmacOS.dmgApple siliconDownloadLinux.AppImagex64, no install requiredDownload

All builds are published on GitHub Releases.