promptfoo
Promptfoo is a command line tool and library developed to evaluate applications based on large language models (LLM) and perform security tests. It allows you to compare the performance of different models using declarational configurations and scan for vulnerabilities. This tool, which can run locally, can be integrated into the CI/CD processes.
promptfoo
100promptfoo · DevOps / Build / Infra
Common commercial products in this category: GitHub Actions, CircleCI, Atlassian Bamboo
- With automated evaluation processes, you can test your requests and models.
- You can secure LLM applications by performing red teaming and security scans.
- You can compare the performance of different models side by side.
- • High number of open issues (889)
- • NeutralHacker News
Promptfoo Is Joining OpenAI
“Hey HN - Michael here, co-founder of Promptfoo. Happy to answer questions. The one I'd ask if I were reading this: what happens to Promptfoo open source? We're going to keep maintaining it. The repo will stay public under the same license, we will continue to support multiple…”
- • NeutralHacker News
Promptfoo Is Joining OpenAI
“You guys absolutely rock. You've built an industry standard with Promptfoo. Keeping it open source was the right choice.”
Verified against README · Oct 26, 2026 · source