Appendix: the raw material
Every comment the review bots posted on the benchmark's pull requests, as posted, with the Heal defect it names when one matched, and every defect Heal proved, with the test and the PR it blames. One page per product, with filters, search and CSV downloads.
- Formbricks →117 defect-claim comments on 56 PRs (108 distinct claims), 307 Heal defects (3 security defects left out)
- Sim →127 defect-claim comments on 44 PRs (105 distinct claims), 223 Heal defects (4 security defects left out)
- Infisical →100 defect-claim comments on 46 PRs (94 distinct claims), 538 Heal defects (16 security defects left out)
Private productsThe two private products of the benchmark (Products A and B) have no appendix: their comments and defects name routes, files and features that would identify them. Their figures on the benchmark page come from the same files, kept private.
Security defectsSecurity defects are left out of this appendix: Heal found several on these products, and their maintainers have not been told about them yet. Publishing an unfixed security defect before the people who can fix it have heard of it would put their users at risk, so those rows wait for the disclosure. They are counted in every figure of the benchmark page.