PromptFoo
Open-source CLI for testing, evaluating, and red-teaming LLM applications
PromptFoo is an open-source tool for testing and evaluating LLM outputs. It lets you compare prompts and models side-by-side across 50+ providers, define test cases in declarative YAML with assertions and scoring, and red-team LLM apps against 50+ vulnerability types including jailbreaks and prompt injection. Results are viewable in a terminal matrix or built-in web UI. Integrates into CI/CD pipelines to gate deployments on eval results. Used by 200,000+ developers and 80+ Fortune 500 companies.
Pricing: Free / enterprise
Choosing an LLM evaluation tool right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
Choosing an LLM evaluation tool right now?
One click, anonymous. Helps us know our visitors.
Thanks, that helps.
PromptFoo Alternatives
Explore 12 products in the Prompt engineering category. View all PromptFoo alternatives.
FinetuneDB
Capture production data, evaluate outputs collaboratively, and fine-tune your LLM's performance
Giskard
Eliminate risks of biases, performance issues & security holes in AI models. In <10 lines of code.
Klu
Collaborate on prompts, evaluate, and optimize LLM-powered Apps with Klu.
Portkey
AI gateway for routing to 1,600+ LLMs with observability, guardrails, and prompt management
Orq.ai
Sovereign AI gateway and LLMOps platform: model routing, prompt management, evals, and governance
Prompty
Structured prompt builder that treats prompts like code, with versioning and a reusable building-block library
Work on PromptFoo? Feature it at the top of Prompt engineering.
Is your product missing?