Peer Review

Every quest is reviewed by AI first. The AI evaluates the submission against the quest's Acceptance Criteria — including its written text, attachments, and any connected GitHub pull requests — and can approve, request changes, or reject the submission directly when it is confident in its decision.

The system escalates to a single human reviewer when the AI is not confident, encounters content it cannot evaluate (an unsupported file type or an inaccessible resource), or has already requested changes twice. The assigned human then takes over for the rest of the quest's life.

Manager-assigned quests are the exception: when a manager or owner uses "Assign quest to member", the assigning manager becomes the sole reviewer immediately and AI is bypassed.

When the AI escalates to a human (or the AI is unavailable), QuestLore picks a single reviewer automatically using a priority-based fallback algorithm:

1. A colleague in the same department whose Experience Points level is equal to or higher than the submitter's.
2. A manager in the same department with a sufficient Experience Points level.
3. Any employee in the organisation with an Experience Points level equal to or higher than the submitter's.
4. Any manager in the organisation with a sufficient Experience Points level.
5. Any manager in the organisation — regardless of level.
6. Any owner in the organisation — the final fallback.

When multiple eligible reviewers exist at a step, one is selected at random. Once assigned, the human reviewer is sticky — they handle every subsequent submission on this quest until it reaches a final state.

To protect reviewers from in-flight pressure, the assigned human reviewer's identity is hidden from the submitter until the reviewer posts a final decision (approved or rejected).

• The submitter sees that "a reviewer has been assigned" but does not see the reviewer's name or avatar.
• If the reviewer requests changes, the submitter sees the feedback text but the reviewer remains anonymous.
• Once the reviewer posts approved or rejected, their identity is revealed retroactively across every event they posted on this quest.
• The reviewer always sees the submitter's identity — this anonymisation is one-way.
• AI Reviewer events, owner overrides, and manager-assigned quests are never anonymised.

The AI can request changes up to 2 times per quest. The third evaluation must escalate to a human regardless of AI confidence. This bounds both AI cost and worst-case submitter latency.

After escalation, the AI never reviews this quest again — the assigned human owns it for the remaining lifecycle, even across future resubmissions after changes requested.

A submission also escalates to a human whenever the AI flags that it is not confident, that the submission contains an unsupported content type, or that it references a resource it cannot access. If the AI provider is temporarily unavailable, the submission goes straight to a human reviewer.

The AI reviewer evaluates more than just the written submission text. It can also read:

• Attachments — images (JPEG, PNG, GIF, WebP), PDFs, Word documents, Excel spreadsheets, and plain-text files. File types outside this list cannot be evaluated and cause the submission to escalate to a human reviewer.
• GitHub pull requests — if a submission includes a GitHub pull request link and the submitter has connected their GitHub account under Profile → Integrations, the AI automatically fetches the pull request's title, description, and changed-file diffs and reviews the code alongside the submission.

If the submitter has not connected GitHub, the AI sees only the bare pull request link and will typically escalate to a human reviewer so the code can be assessed.

Human reviewers are rewarded for their time and mentorship. When a reviewer posts a final decision (approved or rejected) that resolves the quest, they receive review points and an Experience Points bonus equal to 5% of the Experience Points the submitting member earned for that quest.

AI Reviewer never earns review points — only human reviewers do.

A quest submission can be in one of four states during its review lifecycle:

• In Progress — The member is working on (or revising) their submission.
• Under Review — The submission is awaiting the reviewer's decision (AI or human).
• Completed — The reviewer accepted the submission. Experience Points and Coins are credited to the member.
• Rejected — The reviewer rejected the submission. No Experience Points or Coins are awarded. The member may re-attempt the quest if it is configured as repeatable.

Reviewers see all pending review requests in My Quests → Review Requests. Each request displays:

• The quest title and difficulty
• The submitting member's name
• The member's written work description and any attachments

The reviewer can approve the submission, request changes with written feedback, or reject it. The submitter cannot see the reviewer's identity until the reviewer posts a final decision (approved or rejected).