How PR review works

When a pull request event arrives from GitHub, BattleTest runs several layers of analysis in sequence. Knowing what each layer checks, and what it can and cannot catch, helps you interpret findings accurately.

Advanced vs basic reviews

BattleTest runs one of two review modes depending on your current usage.

Advanced reviews run the full pipeline: dependency CVE scanning, secret detection on the PR diff, configuration hardening checks, regression detection, and multi-turn AI code analysis. They produce proof-of-concept scripts for confirmed findings and post a detailed GitHub PR review with inline comments.

Basic reviews run a fast single-pass scan covering secret detection and CVE scanning only. No AI analysis, no regression detection, no PoC generation. Basic reviews run automatically when your advanced limit is reached, so every PR gets at least partial coverage. The PR review includes a note indicating that the basic scan ran.

Which type runs depends on repository visibility and how much you have already used:

• Public repo: 20 advanced reviews/hour per repository, then basic fallback (20/hour per repository)
• Private repo: 10 advanced reviews/month, then basic fallback (5/hour)

See Limits & Quotas for the full figures.

Dependency CVE scanning

Every dependency added or updated in the PR diff is checked against the Open Source Vulnerability (OSV) database and NVD. The result is a list of CVEs affecting the specific versions in the PR, with CVSS scores and fixed-version guidance.

This catches the most automatable and most commonly exploited vulnerability class: known-vulnerable third-party libraries that enter a codebase through routine upgrades.

Secret scanning

Secret scanning runs against the PR diff (the lines the pull request adds or changes) using gitleaks with 150+ rules. Any changed file that looks like a credential store (.env, key files, and similar) is additionally fetched and scanned in full, not just the diffed lines. PR review scans the change. Full-history secret sweeps run as part of the repo-posture battletest, not on every PR.

The rule set covers cloud credentials (AWS, GCP, Azure), SaaS API keys (Stripe, SendGrid, Twilio, and hundreds of others), database connection strings, and cryptographic key formats. Findings include the file path and line number.

If you receive a secret finding, rotate the credential immediately. Removing it from the current branch is not sufficient: it remains readable in the commit history of anyone who has already cloned the branch. See how to respond to an exposed credential.

Configuration hardening

A set of structural checks against the repository itself: is there a SECURITY.md? Is Dependabot configured? Is there a CODEOWNERS file? Are lockfiles committed alongside package manifests? Does a CI configuration include security gates? These are organisational prerequisites for a healthy security posture, not code vulnerabilities.

Regression detection

New findings are compared against your repository’s finding history. If the same vulnerability class was previously resolved and has returned, it’s flagged as a regression and weighted more heavily in the risk score. This comparison makes consecutive reviews useful: each PR is assessed against what your team has already fixed.

AI security analysis

An AI agent analyses the complete diff in context: the changed lines, the functions they interact with, and the data flow from request input to storage and output. The agent produces a prioritised finding list with severity classification, code location, a plain-English explanation of why the finding is a problem, and a remediation example in the language and framework used by the PR.

This layer runs after the others because it’s richer for it: a CVE in a new dependency informs the analysis of what risk surface has changed, and a detected secret is cross-referenced against the rest of the codebase. The AI analysis adds semantic understanding on top of the earlier layers.

This layer only runs in advanced reviews.

Why findings are advisory by default

BattleTest posts findings as a PR comment and creates a check run. By default it does not block merges. A security tool that blocks deploys on every false positive quickly gets bypassed, so the goal is that every finding is visible and tracked, not that every finding requires resolution before a PR can merge.

The check conclusion mirrors the review verdict: a risk score of 40 or higher reports action_required (“request changes”), anything lower reports success. On its own this surfaces a status and blocks nothing. Teams that want a hard gate add the check as a required status check in GitHub branch protection, which then blocks the PRs whose risk warrants changes while letting low-risk PRs through. See how to require security review before merging.