The Async Design Review: 48 Hours Instead of One Hour

Nineteen comments arrived in the first hour of a live design review, and eleven of them contradicted each other. The session was replaced by a 48-hour written window, and the contradiction rate fell to almost nothing because people could see what had already been said.

The team designed checkout flows for an e-commerce platform. Reviews happened weekly, involved six people, and had a reputation for ending without a decision.

Why the live review failed

In a live review the first comment sets the frame. Whoever speaks first names the problem, and the remaining hour is spent adjusting that framing rather than testing it. The most senior person usually spoke first, which meant the room optimised for a problem that had been chosen in the first ninety seconds.

The loudest objections also arrived at the end, when nothing could be changed without another meeting.

The written window

The replacement was a document with the screens embedded, a fixed structure of four questions, and a 48-hour window. Reviewers had to answer all four questions in writing. Comments were visible to everyone as they arrived, which was the point.

The four questions covered the problem being solved, the parts that felt unresolved, the parts that would break at scale, and a yes or no on whether the work was ready.

The first round

Round one produced 31 comments and a 62 percent agreement on readiness. Six of the 31 comments repeated a point already made, which was expected in the first attempt because the visibility rule was new.

Two reviewers wrote nothing and said privately that they had not understood which screens were new. The document had no marker for that.

Adding the diff

The second round added a diff view, so a reviewer could see what had changed since the previous version. Comments dropped to 18 and the proportion that addressed a change rose from a third to two thirds.

The diff also removed an entire class of comment, which was people objecting to something they remembered rather than something on the screen.

What the format forced

Writing a comment forces a reviewer to state a position. In the live review a remark like I am not sure about this could sit unresolved, because the room would move on. In writing the same remark must be finished with a reason or dropped.

Eleven of 31 comments in round one were dropped between drafting and saving, according to two reviewers who described the experience. That is a self-correction the live format could not produce.

The delay problem

A 48-hour window means a decision takes two days. For a weekly cycle that is acceptable. For an urgent fix it is not, and the team kept a separate path for changes that had to ship within the day, which ran three times in two months.

The urgent path had no review at all, only a note afterwards, and both times the note was written. It is a compromise that depends on the people using it being honest.

Four months on

Reviews that ended without a decision fell from 40 percent to 8 percent. The average number of comments per review fell from 27 to 15 while the number of accepted changes rose, which suggests less noise rather than less attention.

The reviewer who never finished

One reviewer completed all four questions in one of six rounds. The window closed and the work moved on without her input, which caused a short argument about whether silence meant approval. The team wrote a rule afterwards: no answer means no objection, and an objection arriving late is a request for the next version.

The rule is uncomfortable and it is the only one that keeps a fixed window honest. A window without a default answer waits for the slowest person indefinitely.

Nineteen comments once arrived in an hour and eleven contradicted each other. The window is now 48 hours, the comments are 15, and the contradictions have mostly become replies that say the same thing on purpose.