Heal · comparison · October 2026
Heal vs cubic: cubic reads the diff, Heal runs the product
cubic was the strongest review bot in our benchmark. It still missed most of the defects users would hit.
cubicCode reviewTL;DR
- cubic is an AI code reviewer: it comments on a pull request a few minutes after the push. Heal builds a production-like sandbox of the product and drives it as its users do, then proves every defect with a failing test.
- Of the three bots in the benchmark, cubic named the most Heal defects: 46 of 247. Adding Greptile and CodeRabbit on top of it raised that to 53.4
- 70 % of the user-facing defects Heal proved on the PRs tested were found by no review bot, cubic included.5
- Keep cubic for the diff, and for the files no user reaches. Add Heal for what users hit.6
1. Heal and cubic, side by side
Both aim to catch defects before they reach users. cubic reads what a developer changed; Heal runs what users will meet.1
| cubic | Heal | |
|---|---|---|
| What it is | A PR review bot: it reads the diff and comments on it | A runtime bug hunter: it runs the product and proves the defects it finds |
| Input | A diff, with the codebase as context | The whole codebase, and a diff |
| The question it answers | What would a developer reading this change fix? | What will a user hit when they use the product? |
| When | Minutes after the push: a median of 11 minutes in the benchmark | Minutes to hours: a full run over the product |
| Does it run code? | No: it reads the code and its context | The whole product, in the Heal application sandbox, through every entry point: UI, API, CLI, webhooks, jobs |
| Its proof | An argument in a comment, with a proposed fix | A test that fails on the running product |
| Best at | Style, consistency, internal guidelines, and files no user reaches: CI, migrations, scripts, tests | Behaviour, security and performance defects a user reaches, across files, PRs and repositories |
2. What the benchmark measured
cubic was the most talkative and the most useful of the three bots. On the open-source products it posted 510 comments, 220 of them raising a defect: more defect claims than CodeRabbit and Greptile together (115).3
It is also the bot that named the most of what Heal proved: 46 of the 247 Heal defects on the PRs all three bots reviewed. On Formbricks and Sim it was the only bot to name any.4
That is still about one in five. The defects it missed are the ones the diff could not show: a bug that takes two repositories, three PRs, or the frontend and the backend disagreeing about a route.5
3. What cubic catches that Heal doesn't
Most of what cubic flags sits where Heal does not look, by design: CI configuration, migration history, analytics, upgrade scripts, the product's own tests. No user reaches those files, so Heal never maps them. That is where code review is the right tool.6
Some of its flags sit on code Heal tests but did not report, mostly race conditions, which one pass through a feature does not always trigger. The benchmark did not check whether those are real.6,8
4. Use both
cubic is on the diff minutes after the push, before anyone reads it, and proposes a fix for nearly every claim. Heal runs later and longer, and hands back defects a user would meet, each with the test that proves it.7
If you already run cubic, a second review bot is not the answer: Greptile and CodeRabbit added 7 Heal defects to cubic's 46. The runtime adds what the review cannot see.4
Read the full benchmark, or run Heal on your repository.
5. Notes
Every number on this page comes from our benchmark of Heal against CodeRabbit, cubic and Greptile on 236 pull requests of five products (October 2026), or from its raw data.
- 236 pull requests of five products (two private, anonymised as Product A and Product B, and Formbricks, Sim and Infisical) replayed PR by PR into fresh private repositories, with CodeRabbit, cubic and Greptile installed on their default settings, Greptile with TREX on, and Heal's full run before and after the sequence of PRs tested. A language-model judge, calibrated on a manually curated sample, decides whether a bot's claim and a Heal defect on the same lines name the same problem. Benchmark, section 3: The benchmark.
- cubic reviewed 45 of Product A's 60 PRs and every PR of the four other products (30, 56, 44 and 46), on its default settings. Benchmark, section 3: Benchmark stats.
- The per-bot counts on this page (comments posted, comments raising a defect, median delay before a comment) are computed from the appendix's raw data: every comment the three bots posted on the 146 replayed PRs of Formbricks (56), Sim (44) and Infisical (46), as CSV. A comment raising a defect is one the appendix tags functional, security, data, performance or concurrency; summaries, docs, tests and style are left out. Benchmark appendix: the raw data.
- Figure 4, every Heal defect on the PRs all three bots reviewed (247 across the five products): cubic alone named 46 (29 of 120 on Product A, 13 of 53 on B, 3 of 31 on Formbricks, 1 of 21 on Sim, 0 of 22 on Infisical); adding Greptile brings it to 52, all six on Product A; adding CodeRabbit to 53, the one on Product A. Benchmark, section 6: the truth about stacking.
- 70 % of the user-facing defects Heal proved on the code of the PRs tested (141 of 202, across the five products) had never been found by any of the three review bots. Benchmark, section 4: what code review bots don't see.
- The three bots' 1,003 distinct flags: 654 in files Heal does not read by design (CI configuration, migration history, dependency versions, analytics, upgrade scripts, the product's own tests); 202 on code Heal tests but did not report, mostly race conditions, not checked for whether they are real; 100 on code a later PR rewrote before Heal ran; 47 also found by Heal. Benchmark, section 5: what code review bots find and Heal doesn't.
- 98 to 99 % of the bots' claims come with a proposed fix. Heal's defects come with 1,085 regression tests across the five products, each one failing on the running product. Benchmark, section 7: Heal and code review find complementary issues.
- The bots' claims were not checked for relevance, while every Heal defect is proven by a test: a bias in the bots' favour. Whether the authors acted on a comment could not be checked, since the replays were isolated from the real history. A relatively small dataset: a useful signal, not a ranking. Benchmark, section 3: Limitations.