pROC 1.20 fixes 129 bugs and 77 documentation and test problems found in pROC 1.19.1. Almost all of them were found and fixed by an automated bug hunt with AI agents in October 2026, then reviewed and revised before the release. This post explains what was done, how it was reviewed, what changes for you, and lists every fix.

AI disclaimer. The bugs were found, fixed and documented by AI agents (Claude Code, by Anthropic), working from my instructions with a large degree of autonomy. The review was also done with Claude, in a separate session: it audited the agents' work, reproduced the findings and prepared the decisions. Every decision about what to fix, and how, was mine, and I reviewed the results. This post was drafted by Claude from the agents' reports and the review records, and edited by me.

Breaking changes

These changes can break code that worked with 1.19.1:

• geom_polygon_auc() has no legacy.axes argument anymore: it follows the axes of the ggroc() plot.
• The formula methods of plot.roc() and lines.roc() now take subset and na.action after ..., so these must be named: plot.roc(y ~ x, data, subset = g == "a"). A positional subset now fails.
• The unpaired DeLong test of roc.test() no longer returns a parameter (df): it now compares its statistic to the normal distribution (Hanley & McNeil, 1983), like the paired DeLong and the bootstrap tests, instead of a Student t distribution with ad hoc degrees of freedom. p-values change slightly with small samples.

Results that change

Most fixes change results. Of the 129 bugs fixed, 55 change a returned value (a wrong number, curve, confidence interval, p-value or flag becomes right), 41 return a result where 1.19.1 failed with an error, 10 fail with an error where 1.19.1 returned a wrong or meaningless value, and 21 change only printed or plotted output. The other two change where pROC writes its bug-report file, and add a paired argument to cov() and power.roc.test(). Every fix is listed below with its commit.

What was done: the bug hunt

The hunt ran several days, mostly unattended.

The brief. I gave the agent a written brief:

• The deliverable: one draft pull request with one commit per bug.
• What counts as a bug: wrong results, inconsistencies between interfaces, ignored arguments, input handling, side effects, C++ problems, platform problems, wrong documentation and wrong tests.
• The methods to use: review against the literature, parameter sweeps, property tests, differential tests, edge cases, static analysis and sanitizers, coverage, and the history.
• The evidence: every bug needed a standalone reproducible example, with its output before and after the fix.
• Autonomy: the agent was not to stop and ask about individual bugs. Anything that changed behaviour or the API went to an approval queue instead. Nothing destructive was allowed: no force-push, no push to the main branches, no merge.

The organisation.

• The main session kept a ledger on disk with one entry per bug, and ran the static checks itself: compiler warnings, cppcheck, codetools, lintr, coverage, and a sanitizer (ASAN/UBSAN) run of the whole test suite.
• Hunter agents each searched one area: the roc core; AUC, smoothing and multiclass; the CIs, bootstrap and coords; DeLong, var/cov and the tests; power; plotting and printing; and the documentation. They couldn't touch git, and they ran R against a fixed copy of the package.
• Fix agents worked in separate git worktrees. Each fix came with a regression test that had to fail before the fix and pass after it, and the full test suite ran on every commit.

Six passes. Each pass added new methods:

The first method to find nothing was the mutation testing of pass 5. Pass 6 still found bugs, but minor ones, such as a plot.roc() option that ignored a partial AUC unless another option was also set, a pairing rule that was not documented, and an unclear error message. The hunt did not stop by itself. On 13 October, during pass 6, the review session and I stopped it: read together, the commits showed that in several places each point fix had widened the previous one, and that needed rules before any more fixes ("My plan, on 13 October", in the next section).

By the end, the hunt had produced 233 bug reports, from BUG-001 to BUG-233, in one pull request: #148.

How it was reviewed

First reviews, by the agent itself.

• Every candidate was re-run before it got a bug number.
• An adversarial review agent re-read all the commits before the first push. It found two regressions and four incomplete fixes, which were corrected before anything was published.
• GitHub's continuous integration ran on every push.

These reviews looked at one commit at a time, though. Several later fixes turned out to repair regressions introduced by earlier fixes of the same pull request.

My plan, on 13 October. With the hunt stopped, I set policies for the fixes:

• P1: degenerate input gives NA, as base R does;
• P2: no matching on warning messages, and no blanket suppression of warnings;
• P3: no floating-point tolerances as fixes;
• P4: fix in one place;
• P5: test hygiene;
• P6: classes from packages pROC doesn't depend on are out of scope;
• P7: every statistical change needs a source that was actually read.

Root causes. For each cluster of related fixes, the agent wrote a design note, and I decided how to fix the cause once: argument routing, smoothing settings, rounding, input normalisation, degenerate input, and the bootstrap. The agent then rewrote the history so that each commit holds one complete, reviewed fix.

An audit of the whole pull request:

• every commit checked against the policies;
• the behaviour of the whole package compared with 1.19.1;
• objects saved by 1.19.1 loaded into the new version;
• performance compared with 1.19.1;
• every test that depends on a random seed measured over hundreds of seeds.

I took each finding as a separate decision.

A check that every bug was reviewed. Some fixes had only been accepted as a number in a list. So every bug was traced back to the evidence I had actually seen:

• the 15 fixes that change numbers, or turn a value into an error, were reviewed again in full: all 15 were real bugs, and a few needed changes;
• the other 117 got a lighter pass, with each fix re-run against 1.19.1.

Final check. After the last revision, 1520 test cases were compared between 1.19.1, the previous version of the pull request and the final one. Every difference traced back to a documented decision.

Other fixes

A few bugs were also fixed in the development version, separately from the hunt: seven bugs and two warning or error messages (DEV-1 to DEV-9). They are included in the counts and in the lists below.

Not fixed

5 bugs of 1.19.1 are not fixed, by decision:

• three floating-point rounding cases (BUG-112, the weighted ties of BUG-113, and the fraction case of BUG-226): no floating-point tolerance as a fix;
• integer64 predictors (BUG-174): a class from a package pROC doesn't depend on;
• a single density value (BUG-205): degenerate input, which returns a value as in 1.19.1.

Lessons learned

• Review by root cause, not only commit by commit. When a third fix lands in the same function, it is time for a design note, not another point fix.
• Settle the policies early, before the first fix, not after 230.
• Provide the papers at the start. The agent couldn't get the paywalled ones, and statistical claims made from memory had to be checked again.
• Treat a stopping rule as a minimum, not as a finish line.
• Record who decided what. Once, the review agent put into the agent's brief a decision the maintainer had never made. Since then, every decision file quotes the maintainer's answer, and anything else is labelled a proposal.
• Freeze history before writing about it. The branch was rewritten three times during the review, so every commit hash in the decision files and the pull request had to be updated

All the changes

Each entry links to its commit, with a reproducible example and the output before and after the fix.

Bugs fixed (129)

Documentation and tests (77)