Heal · comparison · October 2026
Heal vs Greptile: TREX runs a snippet, Heal runs the product
Greptile with TREX is the one review bot that runs code. It runs a snippet to check a claim; Heal runs the whole product to find one.
GreptileCode review, with code snippet runtime validation (TREX)TL;DR
- Greptile is an AI code reviewer, and TREX adds code snippet runtime validation to its review. Heal builds a production-like sandbox of the whole product and drives it as its users do, then proves every defect with a failing test.2
- Greptile posted the fewest comments of the three bots: 207 on the open-source products, 45 of them raising a defect.4
- On top of cubic it named 6 more Heal defects, all on Product A. On the three open-source products it named none of the 74.5
- 70 % of the user-facing defects Heal proved were found by no review bot. Keep Greptile for the diff; add Heal for what users hit.6
1. Heal and Greptile, side by side
Both aim to catch defects before they reach users. Greptile reads what a developer changed; Heal runs what users will meet.1
| Greptile | Heal | |
|---|---|---|
| What it is | A PR review bot: it reads the diff and comments on it | A runtime bug hunter: it runs the product and proves the defects it finds |
| Input | A diff, with the codebase as context | The whole codebase, and a diff |
| The question it answers | What would a developer reading this change fix? | What will a user hit when they use the product? |
| When | Minutes after the push: a median of 11 minutes in the benchmark | Minutes to hours: a full run over the product |
| Does it run code? | Code snippets, through TREX: runtime validation inside the review | The whole product, in the Heal application sandbox, through every entry point: UI, API, CLI, webhooks, jobs |
| Where candidates come from | The diff and the codebase as context | The product: many candidates from the code, scanners and QA priors, each driven on the running product |
| Its proof | An argument in a comment, with a proposed fix | A test that fails on the running product |
| Best at | Style, consistency, internal guidelines, and files no user reaches: CI, migrations, scripts, tests | Behaviour, security and performance defects a user reaches, across files, PRs and repositories |
2. What the benchmark measured
Greptile with TREX is the quietest of the three bots: 207 comments on the open-source products, against 329 for CodeRabbit and 510 for cubic, and 45 of them raising a defect. It says less; the benchmark does not tell how much of that is TREX.2,4
On top of cubic, it named 6 more Heal defects, all on Product A, the biggest gain any second bot brought. On the open-source products it named none of the 74 Heal defects on the PRs all three bots reviewed.5
The defects it missed are the ones a snippet cannot hold: a bug that takes two repositories, three PRs, or the frontend and the backend disagreeing about a route. Running a piece of the diff checks the piece; it does not show what the whole product does with it.6
3. What Greptile catches that Heal doesn't
Like any review bot, Greptile flags files Heal does not look at, by design: CI configuration, migration history, analytics, upgrade scripts, the product's own tests. No user reaches those files, so Heal never maps them.7
Some flags sit on code Heal tests but did not report, mostly race conditions, which one pass through a feature does not always trigger. The benchmark did not check whether those are real.7,9
4. Use both
TREX and Heal both answer the same worry, that a claim read off the code may be wrong, at two scales. TREX validates code snippets around the diff, minutes after the push. Heal starts from the product, lets its candidates be wrong, and keeps only what it reproduces on the running product, with the test that proves it.2,8
Running snippets did not close the gap: most user-facing defects still went unnamed by every bot. Keep Greptile on the diff and add the run of the product.6,5
Read the full benchmark, or run Heal on your repository.
5. Notes
Every number on this page comes from our benchmark of Heal against CodeRabbit, cubic and Greptile on 236 pull requests of five products (October 2026), or from its raw data.
- 236 pull requests of five products (two private, anonymised as Product A and Product B, and Formbricks, Sim and Infisical) replayed PR by PR into fresh private repositories, with CodeRabbit, cubic and Greptile installed on their default settings, Greptile with TREX on, and Heal's full run before and after the sequence of PRs tested. A language-model judge, calibrated on a manually curated sample, decides whether a bot's claim and a Heal defect on the same lines name the same problem. Benchmark, section 3: The benchmark.
- Greptile was evaluated as "Greptile with TREX": code review, with code snippet runtime validation, on for every review. The benchmark measures what Greptile posted with TREX on; it does not separate TREX's share of it. Benchmark, section 3: Tools evaluated.
- Greptile reviewed 50 of Product A's 60 PRs and every PR of the four other products, on its default settings with TREX on; on Infisical the repository's own Greptile config file applied. Benchmark, section 3: Benchmark stats.
- The per-bot counts on this page (comments posted, comments raising a defect, median delay before a comment) are computed from the appendix's raw data: every comment the three bots posted on the 146 replayed PRs of Formbricks (56), Sim (44) and Infisical (46), as CSV. A comment raising a defect is one the appendix tags functional, security, data, performance or concurrency; summaries, docs, tests and style are left out. Benchmark appendix: the raw data.
- Figure 4, every Heal defect on the PRs all three bots reviewed (247 across the five products): cubic alone named 46 (29 of 120 on Product A, 13 of 53 on B, 3 of 31 on Formbricks, 1 of 21 on Sim, 0 of 22 on Infisical); adding Greptile brings it to 52, all six on Product A; adding CodeRabbit to 53, the one on Product A. Benchmark, section 6: the truth about stacking.
- 70 % of the user-facing defects Heal proved on the code of the PRs tested (141 of 202, across the five products) had never been found by any of the three review bots. Benchmark, section 4: what code review bots don't see.
- The three bots' 1,003 distinct flags: 654 in files Heal does not read by design (CI configuration, migration history, dependency versions, analytics, upgrade scripts, the product's own tests); 202 on code Heal tests but did not report, mostly race conditions, not checked for whether they are real; 100 on code a later PR rewrote before Heal ran; 47 also found by Heal. Benchmark, section 5: what code review bots find and Heal doesn't.
- 98 to 99 % of the bots' claims come with a proposed fix. Heal's defects come with 1,085 regression tests across the five products, each one failing on the running product. Benchmark, section 7: Heal and code review find complementary issues.
- The bots' claims were not checked for relevance, while every Heal defect is proven by a test: a bias in the bots' favour. Whether the authors acted on a comment could not be checked, since the replays were isolated from the real history. A relatively small dataset: a useful signal, not a ranking. Benchmark, section 3: Limitations.