Advanced vs basic reviews
BattleTest runs one of two review modes depending on your plan and current usage. Advanced reviews run the full pipeline: dependency CVE scanning, secret detection across the full branch history, configuration hardening checks, regression detection, and multi-turn AI code analysis. Advanced reviews produce proof-of-concept scripts for confirmed findings and post a detailed GitHub PR review with inline comments. Basic reviews run a fast single-pass scan covering secret detection and CVE scanning only. No AI analysis, no regression detection, no PoC generation. Basic reviews run automatically when your advanced limit is reached so every PR gets at least partial coverage. The PR review will include a note indicating that the basic scan ran. Which type runs depends on your plan:- Free, public repo: 5 advanced reviews/hour, then basic fallback (5/hr)
- Free, private repo: 10 advanced reviews/month, then basic fallback (5/hr)
- Paid plans: Advanced on every PR, no monthly cap
Dependency CVE scanning
Every dependency added or updated in the PR diff is checked against the Open Source Vulnerability (OSV) database and NVD. The result is a list of CVEs affecting the specific versions in the PR, with CVSS scores and fixed-version guidance. This catches the most automatable and most commonly exploited vulnerability class: known-vulnerable third-party libraries that enter a codebase through routine upgrades.Secret scanning
Secret scanning runs against the full commit history of the PR branch — not just the staged diff. This matters because a developer who commits a secret and then “removes” it in a subsequent commit has still exposed the secret in the branch history. Anyone who has cloned the repository can read that commit. An extensive rule set covers cloud credentials (AWS, GCP, Azure), SaaS API keys (Stripe, SendGrid, Twilio, and hundreds of others), database connection strings, and cryptographic key formats. Findings include the commit hash, file path, and line number. If you receive a secret finding, the credential needs to be rotated immediately — removing it from the current branch is not sufficient. See how to respond to an exposed credential.Configuration hardening
A set of structural checks against the repository itself: is there aSECURITY.md? Is Dependabot configured? Is there a CODEOWNERS file? Are lockfiles committed alongside package manifests? Does a CI configuration include security gates? These are not code vulnerabilities — they’re the organisational prerequisites for a healthy security posture.
Regression detection
New findings are compared against your repository’s finding history. If the same vulnerability class was previously resolved and has returned, it’s flagged as a regression and weighted more heavily in the risk score. This comparison is what makes consecutive reviews useful — each PR is assessed not just in isolation but against what your team has already fixed.AI security analysis
An AI agent analyses the complete diff in context — the changed lines, the functions they interact with, and the data flow from request input to storage and output. The agent produces a prioritised finding list with severity classification, code location, a plain-English explanation of why the finding is a problem, and a remediation example in the language and framework used by the PR. This layer runs after the others because it’s richer for it: a CVE in a new dependency informs the analysis of what risk surface has changed; a detected secret is cross-referenced against the rest of the codebase. The AI analysis is not a replacement for the earlier layers — it adds semantic understanding on top of them. This layer only runs in advanced reviews.Why findings are advisory by default
BattleTest posts findings as a PR comment and sets a commit status check. It does not, by default, block merges. This is deliberate: a security tool that blocks deploys on every false positive quickly gets bypassed. The goal is that every finding is visible and tracked — not that every finding requires resolution before a PR can merge. The check conclusion mirrors the review verdict: a risk score of 40 or higher reportsaction_required (“request changes”), anything lower reports success. On its own this only surfaces a status — it blocks nothing. Teams that want a hard gate add the check as a required status check in GitHub branch protection, which then blocks the PRs whose risk warrants changes while letting low-risk PRs through. See how to require security review before merging.