# When a review comment disappears: peer review needs an audit trail

- Published: 2026-07-09
- Authors: CORTEXA
- Category: Opinion
- HTML: https://researchhub-vert.vercel.app/blog/neurips-review-transparency-2026

Review systems show authors the current state of their reviews. Almost none show them how that state changed, or who changed it.

A thread in r/MachineLearning this week — *NeurIPS Meta Reviewer comment gone. What gives?* — describes a meta-review comment visible one day and absent the next.

The specific case may have a mundane explanation. The general problem it exposes does not.

## State versus history

Conference review systems are built to show **current state**: here are your reviews, here is your meta-review, here is your decision. That is what authors need at the end.

What they almost never expose is **history**: this review was edited on the 14th, this score changed from 4 to 5, this comment was withdrawn, this reviewer was replaced.

For a process that determines careers, the absence of a visible audit trail is strange. The underlying databases usually have the information. It simply is not surfaced.

## Why this matters more than it used to

Two changes have raised the stakes.

**Scale.** Major venues now handle tens of thousands of submissions with reviewer pools stretched thin. More load means more mid-cycle corrections — reassignments, withdrawn reviews, emergency replacements. Each is legitimate, and each looks alarming to an author who sees only the before and after.

**LLM-assisted reviewing.** Whatever the stated policy, some reviews are now partly machine-written. That makes provenance more important, not less. An author who suspects a review is generated has no way to check anything about how it came to exist.

## What an audit trail would fix

The value is not catching misconduct, which is rare. It is **removing ambiguity**, which is common:

- A comment that vanished because the AC redrafted it looks identical, from the author's side, to a comment that vanished because someone deleted it.
- A score that changed after rebuttal looks identical to a score that changed for reasons unrelated to the rebuttal.

Both ambiguities generate exactly this kind of thread, several times per cycle, and consume real goodwill.

## What it would cost

Not much. The events are already logged — this is a display problem, not a data problem. A per-submission timeline showing *what changed and when*, with reviewer identities kept anonymous, would resolve most of it.

The main objection is that visible edit history might discourage reviewers from revising, for fear that changing their mind looks bad. That is a genuine concern, and it argues for showing that a revision occurred without exposing the diff — enough to distinguish "edited" from "deleted," which is the distinction that actually causes the anxiety.

## The wider point

Peer review is one of the few remaining processes in the field with essentially no observability, at a moment when everything else is being instrumented. The tooling has improved enormously at handling volume. It has barely moved on letting participants understand what happened to their own submission.