November 10, 2026

Human Judgement at Machine Speed

Designing AI-Assisted Teams That Still Think

There is an immediate attraction in a draft arriving before the kettle has boiled. The assistant has turned what used to be an hour's writing into a few seconds of waiting. Research summaries arrive quickly too. A task that occupied most of yesterday is finished before lunch. It is easy to count those gains. It is harder to know whether people are still examining the result as closely as they examined work they made themselves.

There is reason to question that assumption. Faster production changes the reviewer's work, including how much attention each recommendation receives.

The jagged edge

Several hundred Boston Consulting Group consultants took part in a field experiment led by Fabrizio Dell'Acqua and colleagues. Its first account was an HBS working paper in 2023; Organisation Science published the study in 2026. On tasks that sat within the capabilities of the AI model, consultants using it completed more work, faster and at higher quality. On a task deliberately placed just outside those capabilities, consultants using the AI were less likely to reach the correct answer than colleagues working without it.

The researchers described the boundary as a jagged frontier. The model's competence is uneven, and the line between what it handles well and what it handles badly is hard to see from outside. The consultants who went wrong had good reason to trust the tool, since it had been reliable on everything else, and its output looked equally confident on both sides of the line.

The difficulty for a team is that equally fluent answers can have different levels of accuracy. Human judgement is the only part of the system positioned to notice the difference, and it is the part most likely to relax after a run of good results.

How speed erodes evaluation

The change can happen without anyone deciding to check less carefully. A person who once built the argument now reads a completed draft. Writing meant working through each step and noticing where it was hard to join them. The reviewer gets the joins already made. Some will be wrong, but finding them means reopening an argument that looks finished.

This is related to what human factors researchers call automation bias. Once an assistant has supplied several good answers, why distrust the next one? Contrary evidence can get less attention than the familiar recommendation. Volume amplifies it. Thirty drafts require more checking than three, even if each reads well.

Judgement as a property of the team

'Check it more carefully' is an understandable response. The queue and deadline are still there. Look at who could help with the review and where to make room for them.

Put recent corrections beside the team's task list to mark where checking is needed most. Review those marks when the model changes. Keep the transcripts open when reading an interview summary. Let someone who has read the original conversations assess a recommendation based on them.

Visible reasoning comes next. When a recommendation arrives with its reasoning attached, a reviewer can test each link. A finished conclusion gives them less to examine. Bring its supporting evidence to the review. The reviewer can identify the particular step they want to challenge.

The review itself is then distributed. Two colleagues can read the same sentence and find different weaknesses: a data specialist in the figures, a customer specialist in the interpretation. Each has knowledge the other needs.

What the team should measure

Most teams measure AI adoption by time saved, because time saved is easy to count and pleasant to report. That records a useful benefit, but leaves the cost of any errors out of the account.

Read the hours saved beside corrections, client-facing errors and decisions reversed after faulty analysis. A fast draft that took days to repair has a different value from one that passed review.

Count the repair bill in hours, too. On illustrative assumptions, if assistance saves three hours on each of a hundred deliverables, the team gains three hundred hours. If five of those deliverables carry an undetected error that takes two days of senior time to unwind, and costs some client confidence besides, a large share of the gain has gone, and the reputational cost never appears in the hours at all. Perhaps more review time would have cost less than those repairs. The saving alone cannot settle that question.

A periodic sample review closes the loop. A small set of recent AI-assisted deliverables is examined in depth by people who did not produce them, with the specific aim of finding errors. The findings show where the frontier sits for this team, on this work, with this model, which no general guidance can tell it.

Where the human contribution concentrates

That recovered time has another use. Is the question right? Does the client's history change how the answer should be read? Someone may remember a failed campaign the prompt never mentioned.

Protecting that time gives the team something useful to do with the saving. It means people spend less time producing and more time thinking, which is the trade most teams hoped AI would offer in the first place.

The multiplier

The return depends on what the team can check. Share reviews and occasionally examine a sample in depth. Let the findings determine which tasks can move faster.

The hive's view is that the machine supplies the speed and the collective supplies the sense. Each depends on the other, and the teams that matter over the next few years will be the ones that build both into the same workflow.

‍