Cloud agents
Agents deliver, humans close
One rule holds the whole product together, and it is a rule about who is allowed to say finished.
The short answer
Reviewing AI agent work means a person, not the machine, decides whether it is finished. In Polaris a worker sets the task to in progress and delivers a comment; it can never set a task to done. A reviewer reads the delivery, comments on the file or document, sends the open comments back as a review job, and closes the task with a rating once satisfied.
- Who marks a task done
- A human, always
- Review job kind
- review
- Rating
- One per task, per reviewer
The rule, and why it is structural
Workers are told in their operating rules that they never mark a task done and hand it back to the person who owns the outcome. That is not only a prompt. The runtime sets task status to in progress at claim time and touches it again only to deliver, so there is no code path by which a session marks itself complete.
The reason is that delivery and acceptance are different judgements. A worker can tell whether it produced something. Only the person who needed it can tell whether it was the right thing. Collapsing the two is how teams end up with a board full of closed tasks and no confidence in any of them.
The review loop
A batch loop, deliberately. You finish commenting before anything wakes up.
- 1
Read the delivery in place
The deliverable is a comment on the task, next to the ticked acceptance criteria, any attached files and any drafted document pages. An unticked checklist item is the first thing to look at.
- 2
Comment on the artifact, not around it
Comments attach to a file or a document page, and can be anchored to a quoted passage, a specific block or a table cell, so feedback points at the thing it is about.
- 3
Send the whole batch
When you have finished commenting, send the open comments to a worker. That creates a review job, and a unique index permits only one live review per file or page, so the same batch cannot be started twice.
- 4
The worker works all of them in one pass
It is instructed to address every comment it can, apply the changes, and resolve each comment it handled. Anything it genuinely cannot address stays open and is explained in the summary.
- 5
A new version is written
A file becomes a new version with the previous path, size and author preserved. A document is updated in place after a full snapshot of the old title and blocks is stored.
- 6
A human closes and rates it
Marking the task done is a human action recorded on the activity feed, and a thumbs up or down is stored once per task per reviewer against the worker's member row.
Two ways to send work back
Pick by whether the problem is in the artifact or in the brief.
A task comment
- Best when the brief itself needs to change
- Queues a fresh task session for the owner
- The latest human comment becomes the brief
- The worker is told not to redo finished work unasked
- Naming another worker pulls them in too
A review session
- Best when the artifact is close but wrong in places
- Comments anchor to a passage, a block or a cell
- Every open comment is worked in a single pass
- Each addressed comment is resolved by the worker
- The previous version is snapshotted first
What a reviewer should check
In this order, because it gets faster with practice.
The unticked boxes
Acceptance criteria the worker did not tick are its own admission that something is unfinished. Start there rather than reading the prose first.
The sources
Research deliveries end with plain URLs. The skill that governs research says never cite a source you did not open, and the session's search events show what was actually queried.
The numbers
Figures are the thing to verify by hand. A summarisation method that says extract numbers exactly is a rule the reviewer should hold it to.
The hours line
The job stores its human-minute estimate beside the counters that produced it. If the search count on the bill does not match the search events in the stream, that is a specific, answerable question.
Related reading
Assigning a task to an AI worker
There is no prompt box. The task is the prompt, and the checklist is the contract.
When the deliverable is a file
A chat reply is not a deliverable if the thing you needed was a document somebody can open.
The audit trail behind every agent session
If you cannot reconstruct what happened three weeks later, you do not have an audit trail. You have a feeling.
Human-in-the-loop
The system cannot complete the loop without a person, by design.
Acceptance criteria
Written before the work, checkable after it, and binary either way.
Delivery comment
Work arrives where the task already lives, attributed and reviewable, and the task stays open.
Questions people ask
+Can an AI worker ever close a task?
No. The runtime sets a task to in progress when it claims a job and never sets it to done, and workers are instructed in their operating rules to hand the outcome back to the person who owns it. Closing a task is a human action, recorded on the activity feed as a completed event with the actor named.
+What happens to comments the worker could not address?
They stay open. The review rules tell the worker to leave a comment unresolved only when it genuinely cannot act on it, and to explain why in the delivery summary. An unresolved comment with a stated reason is a better outcome than a resolved one with a guess behind it.
+Can I get the previous version of a file back?
The previous version's path, size, version number and author are stored before a new one is written, and document updates store a full snapshot of the old title and blocks. The history exists in your own database, though a one-click restore is not in the product today.
+What does a rating do?
A thumbs up or down is recorded once per task per reviewer against the worker's member row, which builds a per-worker quality record rather than a per-session one. It is a signal for the humans deciding what to assign next.
+Does asking for changes cost more?
A follow-up or review session is a job like any other and records its own human-equivalent minutes, computed largely from comments resolved and content rewritten. A short round of corrections produces a correspondingly short line, and a session that fails records nothing.
Related
Cloud agents that keep working after you close the laptop
For the person whose agent is brilliant, local, single-player, and dead the moment the lid goes down.
Assigning a task to an AI worker
There is no prompt box. The task is the prompt, and the checklist is the contract.
When the deliverable is a file
A chat reply is not a deliverable if the thing you needed was a document somebody can open.
The audit trail behind every agent session
If you cannot reconstruct what happened three weeks later, you do not have an audit trail. You have a feeling.
Sharing the agent, not the transcript
There are two different things people mean by sharing, and only one of them survives contact with a second person.
Human-in-the-loop
The system cannot complete the loop without a person, by design.
Acceptance criteria
Written before the work, checkable after it, and binary either way.
Delivery comment
Work arrives where the task already lives, attributed and reviewable, and the task stays open.