Documentation

Cannonade is a cross-platform desktop app for building test suites, running them against several models, and comparing the results. There is no eval harness to write and nothing is sent to a third party unless you configure a cloud provider yourself.

Getting started

Start here if you are new.

• Quick start: install the app, add a provider, run the bundled suite.
• Connect a provider: Ollama, LM Studio, llama.cpp, OpenRouter, Vercel AI Gateway, and any OpenAI-compatible endpoint.

Guides

• Build a test suite: write cases, attach evaluators, and iterate on a prompt.
• Understand your results: how scoring works, and how to read the numbers a run reports.
• Prompts and versioning: reuse a system prompt across cases, and pin one to a version.
• Playground: chat with one model, tune parameters, and try a prompt before it becomes a case.
• Model downloads: pull models from a registry or Hugging Face, and pick a quantization.

Reference

Look things up here.

• Evaluators: every scoring method and when to use it.
• Providers: supported providers, their defaults, and what each one can do.
• Files and folders: where Cannonade keeps your suites, runs, prompts, and settings on disk.
• Settings: what each pane controls, and the choices with consequences.
• Experiments: opt-in features that are not settled yet.
• Test suite format: how a suite file is laid out, for generating and reviewing them outside the app.

Something missing?

These docs live in the same repository as the app, and every page has an “Edit this page” link at the bottom. Corrections and gaps are welcome as pull requests or issues.