Review and finish.

11. Run the AI reading

1. On AI reading, choose the Model and the Spend limit for this run (USD) (at most $5 a run, unless the proxy is started with a higher --max-run-usd). Keep Include the approved brief ticked unless you have a reason not to. The other options (asking the fallback AI, reading again, asking again, sending as a batch) are under More options.
2. Press Plan the reading. Nothing is sent yet. Under Check what will be sent: the reading you see, for each approved submission, the most it could cost (a real run costs much less), and anything left out, with why. If nothing can be read yet, it says Nothing to read yet and why.
3. Press Confirm and send to send exactly what was planned, or Don't send. Below the button, it says the AI is working, and which reading it is on ("Reading sub-002 (2 of 3)…"), with how long it has been; each can take a minute or more, so keep the page open until it has finished. Every screen that waits for the AI (readings and proposals, drafts, a suggested edit, a suggested verdict or overall comment, a rubric read from an image, and checking or collecting a batch) says so in the same way, announced to a screen reader once (the time is shown, not read out), and it goes when the answer or an error arrives. The run stops at the spend limit.
   • A submission already read with exactly the same text, rubric, brief, instructions and model reuses that reading, at no cost, and the plan says so. Its record says it was reused and from which call. To have the AI read it again anyway, tick Ask the AI again even where nothing has changed.
4. What's recorded lists each sampled submission's reading: whether it is current or needs reading again (and why), by which model and instructions, when, how (directly, in a batch, or reused) and what it cost. Once every submission has a current reading, the form is folded away under Read again.
5. What came back starts with what was read, failed or not run, and what it cost, then lists each, with why, and anything to check (for example a quote the AI gave that isn't in the submission).

To pay half as much, tick Send as one batch, at half the price (under More options) before planning (it is offered only when the proxy's provider can send batches). The plan is priced at the batch rate; press Confirm and send the batch.

• The results come back within a day, usually much sooner, and you can close Feedbacker meanwhile. Waiting for a batch shows how far it has got: press Check now, then Collect the results once it has finished. Cancel the batch stops it; readings already done are still billed, and can be collected.
• Only one batch waits at a time, so nothing is sent twice.
• A batch has no automatic fallback. A submission the AI declined, or that didn't come back, is listed with why: plan the reading again without the batch to read it one at a time, with the fallback.
• If a submission, the brief, the rubric or an approval changes while the batch is out, its reading isn't kept: read it again.
• The provider keeps a batch's results for 29 days, so collect them within that time.

A reading is only ever a suggestion: a level for each criterion, the evidence it quotes, and a draft comment.

Each submission's approved figures (charts, dashboards, screenshots, diagrams) are sent with it, each as an image after its placeholder, such as [FIGURE_1]; the plan's Figures column says how many are sent, and Figures included but not sent lists any the AI can't be sent (an EMF or TIFF image, say). Untick Include each submission's approved figures to read without them. A figure you chose not to send, or one that can't be sent, is marked "(figure not sent)" for the AI, which treats it as evidence neither way: where a criterion rests on visuals it doesn't see, expect it to flag that it found too little evidence, and judge those criteria from the visuals yourself. The AI accepts at most 100 images in a request, each within 8000 pixels a side and 10 MB, and, with more than 20, each within 2000 pixels a side. A submission beyond those limits is left out of the plan, which says how many figures it has and how many fit, with the figures that don't fit listed under The figures of … that can't be sent with the rest. Press Read … with the figures that fit (taken in the order of the text, a figure being left out only where it would break a limit; the rest are marked "(figure not sent)") or Read … without figures; nothing is chosen for you. The reading records which figures it wasn't sent, so Review and, in marking, the proposals say the AI did not see them, and a verdict suggestion says so too.

12. Review each submission, openly or blind

Review opens once the source rubric is saved and every sampled submission has its original file and marking imported and its anonymised text approved; until then it lists what is left. The AI reading is optional, and doesn't hold it back.

On Review, choose a submission and press Review this submission. The first time, choose how to review it; the choice is kept and can't be changed:

• Review openly: the original marking and the AI reading are shown throughout. This is usual moderation practice.
• Review blind: they stay hidden until you have recorded a level for every criterion and press Reveal the original marking and the AI reading. You can then revise any level; your first level and the revision are both kept. Blind review isn't possible once you have confirmed that submission's marking.

The review fills the window in two panes, each scrolling on its own:

• left, the submission: the approved anonymised text, the brief (under The assessment brief), and each marker's overall mark, overall comment and inline comments;
• right, one criterion at a time: the original marking and the AI reading for that criterion, and your judgement.

Above the panes, the list of criteria shows each one's state (Judged, Out of date or Not yet judged). Choose one to open it, or move through them with Next and Previous. Comparison and verdict (once the marking and the AI reading are shown) is below the panes, full width: scroll down to it, or press Comparison and verdict in the list of criteria (or Next after the last criterion) to go there. Show in the text, beside a quote the AI reading found in the submission or a marker's comment whose passage is found as written, highlights that passage in the text and scrolls the left pane to it; Clear the highlight removes it. On a narrow window, or zoomed in, the panes are one above the other and the page scrolls as usual.

For each criterion, choose Your level from the rubric's levels. Its mark starts at the level's points (for example 68 for 2:1 (68)); to place the work within the level, press a quick pick (3 below, the level's points, or 3 above: 65, 68, 71) or enter another mark, which must be nearer that level than any other (or, for a level the rubric gives a mark range, within that range). Add a comment if you wish, and press Record the judgement. The comparison (on Comparison and verdict) sets your mark beside each marker's, and its Overall row shows the overall your marks imply, weighted by the rubric. Start from the AI reading chooses the AI's suggested level and puts its draft comment into yours to edit. The level is recorded as taken from the AI suggestion unless you choose another; the comment is recorded as adapted from the AI draft, however much you change it. The summary says how many levels were taken from the AI, since those agree with it by construction. Clear and write my own clears the comment, so you can write your own.

Comparison (headed with the submission and its pseudonym, on Comparison and verdict) sets your level and mark beside each marker's mark and the AI's suggested level, saying each difference in words ("Harsher than your mark (68) by 3 points"), and flags a marker's level label that doesn't fit their score. Its Overall row gives each marker's overall mark as awarded, the overall your marks imply, and the overall the AI's levels imply (never a mark); where one can't be worked out (no criterion weights, a criterion not yet judged, or the AI suggesting no level), it says why.

If the approved text or the source rubric changes after you judge a criterion (for example you add weights to the rubric), its judgement is marked Out of date beside it, and its button reads Record it again: check your level and mark, and press it. Nothing out of date is counted, in the Overall row or the export, until you do.

13. Record a verdict on each submission's marking

Under How was … marked?, on Comparison and verdict below the comparison, choose Agree, Generous, Harsh or Inconsistent, check the Suggested mark (once every criterion is judged, it starts from the overall your criterion marks imply, rounded; change it if you need to), add a comment if you wish, and press Record the verdict. Both the mark you suggest and the overall your marks implied are recorded, and the summary shows them side by side. For a blind review, this comes after the reveal.

A verdict rests on the marking, the approved text, the rubric and your levels and marks as they were when you gave it. If any of them changes afterwards (for example you move a mark), the verdict is flagged to check again: press Change the verdict to give it again.

Plan the verdict and comment. Once every criterion of a submission is judged, press Plan the verdict and comment to have the AI suggest one of the four verdicts and a comment explaining it, from your levels and marks, the original marker's overall mark and the AI's own reading. As for the AI reading, it first shows exactly what would be sent (never the submission's text, the brief or the marker's comments) and the worst-case cost, and sends nothing until you press Confirm and send. The suggestion is not recorded and nothing is chosen for you: press Use the AI's verdict and/or Use the AI's comment, then change what you like and Record the verdict. A verdict taken from the AI, and a comment that began as its suggestion, are recorded as such.

The Overview shows how far each submission has got and its verdict. Where the request was read from a document, it also shows The cohort the request listed: for each assessment, how many students submitted and how many of them are in the sample, the grade bands with how many of the cohort and of the sample are in each (and the count the request itself gives, where it gave one), and, under The rows of …, each student's row by number with the marks and band, never a name or ID. It says where the request's cohort size or band counts don't agree with its table, and never corrects them. Then it shows Agreement across the sample, by submission and by criterion. It says how many submissions and judged criteria the figures rest on. A sample of fewer than 10 submissions is small, and the Overview and the exported summary say so: the counts describe those submissions only, and are too few to show a pattern in the marking, so the summary doesn't name a criterion the marking tended to differ on. The counts themselves are still given, and the summary says how the sample was chosen where you recorded that.

14. Approve and export the record

On Export:

1. Export opens once the moderation is ready to approve. Until then, it lists anything left to do, by submission (for example a criterion still to judge, a blind review not yet revealed, marking not confirmed, or a verdict to check again because the marking, the rubric or your marks changed after it). Deal with each; the list updates as you go.

2. Write Your overall moderator's comment (it goes into the summary and the section for the moderation form), and press Approve the moderation record. Or press Plan the overall comment: once every sampled submission has a verdict, the AI suggests a short paragraph on the sample from your verdicts and comments (it shows exactly what would be sent, and the worst-case cost, first, and sends nothing until you confirm). Use it or not, as with the verdict; a comment that began as its suggestion is recorded as derived from the AI. A small sample can't show how the marker marked the rest, and the suggestion is told to say so. Approval is of the summary you were shown: if the moderation changed after it was shown (a change still being recorded, say), nothing is approved, and what there is now is shown for you to read.

3. Press Export the record and summary. It writes three files into the workspace's exports folder:

   • …-record.feedbacker-export.json: the structured record, with the full provenance of every value;
   • …-summary.feedbacker-export.md and ….docx: the readable summary.

   All three are pseudonymous. If you change anything after approving, the export is refused until you approve again.

The status line under Export says whether the record is ready to approve, or approved with nothing changed since. Before you approve, the summary is shown as it would be approved, just above Approve; once approved, What's recorded holds it, folded away under The approved summary, and the export buttons stay in reach below.

15. Return the moderation form

The summary ends with For the moderation form: the sampled items by grade band, each with your verdict, your suggested mark if you gave one, and the overall your marks imply (when it can be worked out), then your overall comment, ready to copy into the form. In the Sample overview, a submission without a suggested mark shows the overall your marks imply instead, labelled "(implied by your marks)".

If the form must say which submission is which, make a re-identified copy on Export: under Re-identified copy, press Make a re-identified copy. It asks first, saying what the copy will contain (each student's Turnitin ID, which identifies them, so it is personal data); press Make the copy, or Don't make it. It writes …-summary-reidentified.feedbacker-export.md and .docx, with each student's Turnitin ID in place of their pseudonym. Nothing else is restored: no names, and other redacted details stay redacted. You're asked to confirm each time. The standard exports are pseudonymous, so they need no such confirmation.

Copy what the form needs, add your signature as the form asks, and return it. Then delete the re-identified copy if you no longer need it (step 16).

16. Delete the workspace and the downloads

When the commissioning body's retention period ends, or as soon as you no longer need the material:

1. Press Delete this workspace…, beside the workspace's name above the steps. It deletes the workspace's whole folder and everything Feedbacker made in it: the source files, extracts, anonymised text, approvals, readings, marking, judgements, the pseudonym key and every export, including re-identified copies. It can't be undone.
2. It lists every export in the workspace, with its full path (a re-identified copy is marked as containing personal data). Exports are written only inside the workspace, so copy any you are required to keep, such as the pseudonymous record, to a folder outside it and outside any git repository, for example in the Finder. Keep them for no longer than required. Then tick I have kept the exports I need. If there are no exports, it says so: the record was never exported.
3. Type the workspace's name exactly, and press Delete this workspace permanently. Feedbacker also stops recording the folder's path, and the browser forgets it. The app returns to the start page and says what was deleted. If the browser could only empty the folder, it says so and gives the folder's path, so you can remove it yourself.
4. Delete the downloads (the originals and the marked views), and empty the Trash if they went there. Feedbacker didn't make them, so it can't delete them.

The proxy's egress log (egress.jsonl in its data folder, ~/Feedbacker/proxy by default) holds hashes, not text. Its entries are removed after 90 days, unless the proxy was started with a different --egress-retention-days. The proxy prints both when it starts, as "Egress log: … (kept … days)".

17. If something goes wrong

Feedbacker says what went wrong in words; it never hides a failure. Most messages say what to do next. A red This couldn't be done is a failure. An amber Please check is a note: the step worked, but something needs your attention.

To report a problem, open an issue in the Feedbacker repository, and describe:

• which step you were on and what you did;
• the message, word for word, with any name, ID or other real detail replaced;
• what you expected instead.

Never attach, paste or screenshot real material: no submissions, marked views, rubrics that identify the module, exports, workspace files, or anonymised text (it is still personal data). If you can, reproduce the problem with the made-up files in fixtures/synthetic/, and report that instead.