Performance Reviews Without a Rating Scale: One Cycle Tested

A review cycle without numeric ratings produced longer written feedback and a shorter calibration meeting: 90 minutes against four hours for the previous cycle. Fifty-four people were reviewed, and the written comments averaged 640 words per person, up from 210.

The company had used a five-point scale for three years, with the usual result: most people clustered at three and four, and the calibration meeting spent its time arguing about the difference.

What the scale was doing

The scale compressed disagreement into a single number, which made calibration possible and made the feedback useless. A rating of three could mean solid, could mean inconsistent, and could mean the manager did not have enough information.

Of 54 ratings in the previous cycle, 41 were three or four. Managers described the distinctions between them as largely social rather than evidential.

The replacement format

Three written questions replaced the scale: what this person did that the team relied on, what changed in their work over the period, and what they should take on next. Managers wrote answers before any discussion and shared them with the person two days before the meeting.

The two-day gap was deliberate. Feedback read in the room produces a reaction; feedback read beforehand produces a response.

The calibration meeting

Without ratings there was nothing to calibrate, so the meeting changed purpose. It reviewed whether each written review contained evidence, and whether two people in similar roles had been described in wildly different terms.

It took 90 minutes for 54 people, because the discussion was about wording rather than ranking, and wording can be fixed rather than negotiated.

What the managers found hard

Writing 640 words per person is a real cost: managers with seven reports spent about five hours each. Three managers said they had not realised how little they could evidence for one of their reports until they tried to write the answer.

That discovery is uncomfortable and is arguably the point of the exercise.

What the people found hard

No rating meant no clear signal about standing, which some people wanted and asked for directly. Four of 54 asked whether they were doing well, and the answer had to be given in prose, which is slower and more specific.

Two of those four said afterwards that the prose answer was more useful than a number, and two said they still wanted the number. Both reactions are reasonable.

What replaced the ranking

Compensation was decoupled from the cycle entirely and tied to a separate annual exercise based on role and market data, with a manager’s written case. Ratings had previously been the input to that case, so removing them changed what managers argued about.

The first compensation round after the change produced fewer disputes: three, against eleven in the previous year.

What the cycle could not measure

A written review is a sample of what a manager noticed. It rewards people whose work is visible and penalises quiet work, which is a bias the rating scale also had and did not solve.

The company did not claim to fix it. It recorded the risk and asked managers to name one piece of invisible work per person, which two thirds of them did.

One cycle on

Written feedback rose from 210 to 640 words per person, calibration fell from four hours to 90 minutes, and compensation disputes fell from eleven to three. Time cost rose by about 30 percent in total, which was expected and accepted.

The manager who could not write 640 words

One manager with six reports produced 180 words for each and said afterwards that the format had exposed a real gap: she had been managing output rather than people, and could describe what had shipped but not what had changed in anyone.

That is a hard thing to learn in a review cycle rather than a conversation, and it is the strongest argument the company found for removing the scale.

Fifty-four people once received 210 words and a number between one and five. They now receive 640 words and a conversation, and the number has moved to a separate process that nobody pretends is about development.