The cheapest security fix is the one that happens before merge, while the author still has the change in their head. The most expensive one is the same fix six weeks later, in a backlog, assigned to whoever has capacity.
That is the entire case for security review inside the pull request. It is also, in practice, the fastest way to build a bot every engineer dismisses, because most implementations comment on everything and block on nothing useful. This is how to get the first outcome instead of the second.
Almost every noisy PR security bot fails the same way — it reports the repository's problems on a change's review.
Draw the line here:
Get this wrong and a two-line change collects fourteen comments about things the author did not do. The team learns to scroll past the bot within a week, and after that the tool is worse than nothing: it is a control you will report to an auditor that nobody reads.
A failing check that cannot be overridden is a strong statement. Spend it on findings that are (a) almost never false positives and (b) genuinely worse after merge than before.
Block on:
NEXT_PUBLIC_, VITE_, REACT_APP_ — anything so prefixed is compiled into the browser bundle. A token-shaped value behind one of those prefixes is a shipped secret.Comment, do not block:
dangerouslySetInnerHTML, eval, or an unsanitised redirect target.Neither — put it in the queue:
The difference between a comment that gets fixed and one that gets resolved-without-reading is almost always specificity.
A weak comment:
⚠️ Potential security issue detected. Possible hardcoded secret. Severity: HIGH. Confidence: MEDIUM. See documentation for remediation guidance.
A comment that gets fixed:
src/lib/mail.ts:14— this is a live SendGrid key (SG.prefix, 69 chars). It is in the diff, so it will be in git history after merge; rotate it as well as removing it. Suggested change below moves it to an environment variable.const apiKey = process.env.SENDGRID_API_KEY;
Four things make the second one work:
SG. prefix and the length are the evidence. An engineer can verify the claim in two seconds instead of taking it on faith.suggestion block as a one-click commit. The gap between "understands the fix" and "has applied the fix" is where most findings die.That last point is worth more than any detection improvement. A finding with a one-click fix gets fixed at a completely different rate from one with a paragraph of advice.
There is a strong pull toward putting an LLM in the review loop. In a pull request specifically, it is usually the wrong trade.
A PR check runs on every push from every engineer, several dozen times a day. It needs to be:
Pattern-based detection — secret formats with their real prefixes and entropy, unsafe API usage, dependency versions against an advisory database — is fast, deterministic, and free. Save the model for the parts where judgement genuinely helps: writing the explanation, or drafting a fix for a finding that is already confirmed.
The most useful number on a PR comment is not "23 issues". It is:
2 new · 1 fixed since
main
That framing does three things. It tells the author what they introduced. It gives credit for what the branch cleaned up, which is the only mechanism that makes anyone volunteer to fix an old finding. And it makes the comment shrink to nothing on the majority of PRs that introduce no new issues — which is the honest outcome, and the thing that keeps the team reading the comment when it does say something.
Week 1 — observe only. Run the checks, post the comment, block nothing. Read what it produces yourself before anyone else has to.
Week 2 — cut the noise. Turn off every rule that produced a false positive. Do this ruthlessly and before the team's opinion sets. One wrong block costs more trust than ten correct comments earn.
Week 3 — turn on the hard block, for secrets only. One rule, the one nobody argues with. Let the team experience the bot being right.
Week 4 — widen carefully. Add blocking rules one at a time, each with a visible override path. An override that requires a comment saying why is a control; an override that requires a Slack message to whoever owns the config is a bottleneck people route around.
Specifically, so this is checkable:
```suggestion blocks where the fix is mechanical.git apply locally when you would rather not let a tool commit to your branch at all.PR review is on both team tiers, from $25 per seat per month with a five-seat minimum. The pricing page has the rest.
For a small number of near-certain findings, yes — a live credential in the diff is the clearest case, because after merge the fix is rotation plus history rewriting rather than deletion. For judgement calls like a route with no visible auth check, comment instead. Every blocking rule you add that produces one false positive costs more team trust than several correct comments earn back.
Scope it to the diff. Report only what this branch introduced or changed, and route everything the repository already had into a tracked queue with an owner and a deadline. A bot that comments on pre-existing findings during an unrelated review teaches the team to scroll past it, and a control nobody reads is worse than no control, because you will still report it as one.
They find different things. Static analysis reads code paths and catches injection patterns, hardcoded secrets and unsafe API usage before merge. Scanning the deployed app catches what only exists at runtime: missing security headers, live TLS configuration, CORS as the server actually answers it, publicly readable database tables, and secrets that reached the shipped JavaScript bundle regardless of how they got there. Teams that run only one of the two have a predictable blind spot.
Not as the detection layer. A PR check runs dozens of times a day and needs to be fast, cheap, and identical on identical input — a bot whose findings vary between runs cannot be trusted by engineers or evidenced as a control that operated consistently. Pattern-based detection gives you that. A model is better spent on explaining a confirmed finding or drafting a fix than on deciding what counts as one.
File and line, the evidence for the claim, the consequence the author might not know, and a committable suggestion where the fix is mechanical. The evidence matters most: "this is a live SendGrid key, SG. prefix, 69 characters" can be verified in seconds, while "possible hardcoded secret, confidence medium" asks the reader to take it on trust — and they will not.
Only if you want it to comment. Read access is enough to clone, scan and report findings in your own dashboard. Posting review comments or opening fix branches needs an explicitly granted write scope, which is worth treating as a separate decision — and a patch you apply locally with git apply is a reasonable middle ground for teams that would rather no external tool committed to their branches.
CheckVibe posts inline, committable security suggestions on pull requests, scoped to the diff, on both team tiers. See what teams get, or scan your deployed app free to see what runtime checks add on top.
Paste your URL and get a security report in 30 seconds — 100+ automated checks with AI-ready fix prompts.
Related articles
A short, enforceable policy for Copilot, Cursor and Claude Code on a real codebase: what to allow, what to gate in review, and which rules to skip.
How a small team turns a scanner's output into owned, deadlined work: who triages, what gets closed on sight, and the queue design that stops backlogs.
What a SOC 2 auditor actually asks a five-person engineering team for, which controls you can evidence automatically, and which ones need a human.