Set up a moderation.

1. Before you start

You need:

• a Mac or Linux computer: the workspace's permission safeguards (folders only you can read) don't work on Windows;

• Node.js 24;

• Chrome or Edge: the app opens your workspace folder through a browser feature that Safari and Firefox don't have;

• an Anthropic API key, for the AI reading. Everything else works without one;

• a copy of Feedbacker, installed once:

  cd ~/src/feedbacker/proxy && npm install
  cd ../ui && npm install && npm run build

  Run npm run build again in ui after updating Feedbacker.

Put the API key in a private file, once. Use an editor rather than echo, so the key stays out of your shell history:

mkdir -p ~/Feedbacker && touch ~/Feedbacker/.env && chmod 600 ~/Feedbacker/.env
nano ~/Feedbacker/.env        # one line: ANTHROPIC_API_KEY=…

Two folders matter, and they must be different:

• the Feedbacker source: the copy of the program you installed from (for example ~/src/feedbacker). Never put moderation material in it, or in any other git repository;
• your Feedbacker data folder, ~/Feedbacker: it holds the key file, the proxy's own data (~/Feedbacker/proxy), and is a good place for the workspaces and downloads (for example ~/Feedbacker/workspaces/ and ~/Feedbacker/downloads/).

Don't install the source as ~/feedbacker: macOS doesn't distinguish capitals in folder names, so it would be the same folder as ~/Feedbacker.

2. What stays on your machine, and what is sent to the AI

These never leave your machine:

• the original files and the marked views;
• the text extracted from them;
• the pseudonym key, which links each pseudonym (for example [STUDENT_A]) to a student's name and Turnitin ID;
• the original marking and the marker's comments;
• your judgements, verdicts and comments (in moderation), and your marks and comments (in marking), except as below when drafting feedback;
• the exports.

These are sent to the AI, and only when you confirm it on the AI reading step (or, in marking, the AI proposals step):

• each submission's anonymised text, exactly as you approved it;
• the source rubric;
• the anonymised brief, as you approved it (unless you choose to leave it out).

In marking, drafting feedback on the Feedback step also sends, for that one submission only and only when you confirm it: your own levels, marks and comments for it, and your approved feedback guide, both anonymised. Asking the AI to suggest an edit to flagged feedback sends that recorded feedback, its flags and your marking of that criterion, and not the submission. Nothing about any other student is sent, and the AI proposals are never sent anyone's marks. See section 18, step 13.

Nothing is sent until you have reviewed and approved the anonymised text. If a text changes after you approve it, it can't be sent until you approve it again; and if a rule you add later, or a name Feedbacker learns later, would redact something in it, it isn't sent until it has been anonymised and approved again. Everything leaves through the local proxy, the only program that holds the API key, and it records each request (hashes only, never the text) in its egress log: egress.jsonl in the proxy's data folder, ~/Feedbacker/proxy unless the proxy was started with --data.

Anonymised text is still personal data while the pseudonym key exists, and in practice beyond that, because a submission can identify its author indirectly (a project name, a workplace, a photo). Your review in step 10 is the safeguard for that.

3. Start Feedbacker

cd ~/src/feedbacker/proxy && npm start

The proxy prints an address like http://127.0.0.1:8765/#token=…, "API key: configured", and where its egress log is and how long it keeps it. Open that exact address in Chrome or Edge. The token in it is new each time the proxy starts. If you close the tab, open the printed address again. To stop Feedbacker, press Ctrl+C in the terminal.

The app's header shows the workspace's name and folder, and whether the proxy is connected and has an API key.

4. Create a workspace and set its retention

A workspace is a folder holding everything for one moderation. Use one per moderation. Feedbacker keeps every workspace in one folder, ~/Feedbacker/workspaces.

1. Press New moderation on Home (or choose Moderation at the top), and give it a Name, for example the module and year, such as CS101 2026. The name becomes the folder's name, so it can have letters, digits, spaces, hyphens, underscores and full stops only.
2. Under Options, set Keep for to how many days after the moderation the commissioning body lets you keep its material; the default is 90. Feedbacker records this, but it doesn't delete anything itself: deleting is step 16.
3. Press Start the moderation. The first time, your browser asks you to choose the workspaces folder, ~/Feedbacker/workspaces: choose it, and Feedbacker can open your work from then on (your browser may ask for your permission again on a later visit, with one click). The moderation then opens straight away.

Next time, Home lists your workspaces under Marking and Moderation, with when each was started, how long it is kept for, and how far it has got; press Continue to open one. A workspace whose folder has gone (for example, moved or deleted outside Feedbacker) is listed under Can't be found; Remove from the list forgets it, and deletes nothing. Its name, above the steps, says it is a moderation workspace. Close this workspace, beside the workspace's name above the steps, returns you to the list. Inside a module, the assessment also shows Back to the module and the module's other assessments, one press away. A workspace made by the command line elsewhere can be added under More options on Home, by its folder's full path. Its card says it is Stored somewhere else, with its path, and what Continue will do: ask you to choose that folder (once on this computer; the button then says Choose its folder to continue), ask you to allow access to it again, or simply open it. To stop it asking, press Move into the workspaces folder: Feedbacker renames the folder into the workspaces folder, keeping the workspace and everything in it exactly as it was, and it then opens like the rest. If its folder's name is already used in the workspaces folder (or can't be a workspace's name), the card first asks for Its name in the workspaces folder, with a free one filled in; the folder and the workspace's name take it, and nothing else inside changes. A module, an assessment in a module, or one on a different disk can't be moved this way; the card says why when you try.

The steps are listed along the top in working order: Overview, Request, Assessment (Rubric and Brief), Submissions (Original files and Original marking), Anonymisation, AI reading, Review and Export. Under each step is its status: Not started, Needs attention, Done, or Optional for a step you can leave out (the brief and the AI reading). The Overview shows how far each sampled submission has got.

On Anonymisation, the list of texts says how many things in each still look like an identifier (an email address, a web address, a phone number, a long number), and Review lists them, with the text around each, on your screen only. Each match has a Redact tick, ticked until you untick it; an unticked one can't be sent as it is, because the check before every request, and the proxy's, would still refuse it. Press Add the ticked ones and anonymise again (or Add the ticked ones to the anonymisation rules, then Anonymise now), and review and approve the text again. Only the first 30 matches of a text are shown at first; Show all lists the rest, and matches not shown are never added. A value you had told the rules to ignore is taken off that list when you add it. This finds them before the AI reading does.

If a submission "may identify someone", the plan leaves it out and lists, under What looks like an identifier, exactly what matched (with the text around it and how far through the text it is). These are shown on your screen only. A row of results (such as 0.156 0.20 0.206 0.00 0) is not taken for a phone number; if what is shown isn't an identifier, adding it only hides it from the AI. A match in the rubric can't be fixed by the rules, because the rubric is not anonymised: correct the rubric and import it again. For the others, tick what to redact and press Add the ticked ones to the anonymisation rules, then on Anonymisation press Anonymise now, review and approve each affected text again, and plan the reading again.

The same list appears when drafting feedback, suggesting an edit, suggesting a verdict or suggesting an overall comment would send something that looks like an identifier: nothing is sent, and what was found is listed. A match in your own marking, your feedback or the feedback guide isn't reached by the rules: change it where you wrote it, and record it again (or save the guide again).

Review and Export are Locked until the steps before them are complete. Opening a locked step lists what is left, each with a button to the step where it is done. A step locks again if a later change undoes what it needs (for example, a new rule that clears an approval); nothing you recorded is lost, and it opens again once that is put right.

5. Record the moderation request

On Request, you can read the request document (a Word file, or a PDF with text) first, or enter it by hand. A PDF's table is recovered from where its text sits, which is less reliable than a Word file's own structure: anything it can't place with certainty is left out and listed under Please check, so check the students listed against the document. A student's name that wraps onto a second line (a double-barrelled name, say) is joined to the name when the second line is plainly under it, in the name column, and nearer to its own row than to the next; the note under Please check says how many were joined, so check those names. A PDF that is only pictures of pages (a scan) can't be read; use the Word version. Under Read it from the request document, choose the document and press Read the document. On this computer, Feedbacker finds:

• the module, programme, cohort size, whether it was marked in groups, and the band distribution, and fills them in below. A band distribution that gives more than one number for a band (for example one for each assessment) can't be told apart, so it is flagged and left for you to enter;
• the students the document lists, with each one's initial mark, agreed mark and band. A student marked as not having submitted (for example NS) is shown as having no submission and can't be ticked. Blank rows are skipped.

Where the document lists more than one assessment, choose the one this moderation is of under The assessment this moderation is of; moderate the others in their own workspaces. Choose the sample. Under Choose the sample, a scheme can suggest it from the marks, and you can then tick or untick students by hand.

• The size is a percentage of those who submitted, always rounded up (a tenth, 10%, unless you change it; the percentage you last used is remembered on this computer). It shows the number it gives, for example "Sample size: 1 of the 6 who submitted".
• The scheme: A spread across the bands (the highest and lowest marks, then every fail below the pass mark you give, then the mark nearest each band boundary, then the rest spaced evenly through the marks; the usual choice), Marks near a band boundary (within the distance you give), At random, from a seed (anyone can repeat it from the seed, which is recorded), Marks changed by internal moderation (the furthest moves first), or Pick by hand.
• It suggests only students who submitted and whose ID can be recorded, and says where it found fewer than the size, or couldn't take every fail. The suggested students are ticked, and How the sample was chosen is filled in for you (for example "10% of the 6 who submitted (1), rounded up, chosen as a spread across the bands…"), with the seed where there is one. Tick or untick any student by hand and the note adds "Then adjusted by hand." The same marks and options always give the same suggestion.

Tick the students in the sample: the ticked students' IDs (and their bands, from their marks) fill in the sample below, and the count says how many of those who submitted are ticked. Anything uncertain (a mark that isn't a number, an ID Feedbacker can't record, a table that doesn't match the cohort size) is flagged for you to check, never corrected. When you record the request, the cohort the document listed is kept as well, without its students: for each assessment, its rows by number with the initial and agreed marks and whether the student submitted, the grade bands, which rows are in the sample, and the document's hash. No name or ID is kept for anyone not in the sample (a sampled student's go only into the private pseudonym key, as before). If you later read a different document, you are told it isn't the one the cohort was recorded from, and recording from it replaces that cohort. Nothing is recorded until you press Record the request. Staff named in the document are never read, nothing is kept of the document itself, and a student's name goes only into the private pseudonym key, where anonymisation uses it. Only a Word document is read; for a pdf, enter the request by hand.

The students' IDs in the document are the institution's own, which may not be the IDs in the downloads' file names (for example Turnitin's). Feedbacker finds each student's files by name instead, on the next two steps (see Match the students by name below). The marks the document listed for the ticked students are kept with the request, so a marked view's grade can be checked against the initial mark.

Or enter what the moderation request (the form you were sent) lists:

• Sampled submissions: a row per grade band, with the band (for example 60-69) and its submission IDs as on the form (for example Turnitin submission IDs), separated by commas or spaces. Add another band of the sample adds a row. If the form doesn't give bands, leave the band empty and put all the IDs in one row.
• The module, programme, cohort size and the cohort's band distribution (a row per band, with its number of students), as written on the form (all optional).
• Staff roles involved: roles only (for example "module convener"), never names.

Press Record the request. Each sampled submission gets a pseudonym ([STUDENT_A], [STUDENT_B], …), which is how it appears from now on.

The screen then shows What's recorded: each sampled submission's pseudonym beside its real ID, the only screen that shows real IDs, so you can match pseudonyms to the moderation form, and the context you entered. To correct it, open Change the request: the form starts from what is recorded; tick Replace the request already recorded and record it again. Every screen is laid out the same way, in the same order: a status line under the heading says whether the step is done and why (as the steps do), How this step works explains it, What's recorded shows what it holds, and its form is folded away once the step is complete. Two screens differ, on purpose: the Overview has no status line, as it isn't a step, and Export never folds its actions away, as you can export again whenever you need to ("Import the originals again", "Import the rubric again", "Import the brief again").

6. Import the source rubric and the brief

Use the rubric the module published (xlsx, csv, json, or a docx table), not the marking platform's copy of it: this source rubric governs the comparison.

A rubric that exists only as an image (a screenshot of the marking platform's rubric, say) is entered under Or enter it from an image of the rubric. Feedbacker doesn't read the picture: you type the grid beside it. Choose the image, and it is shown beside the grid; then enter the title, and for each criterion its title, weight and (if it differs from the weight) the points it is marked out of, and each level's label with its range as written (for example "Excellent (16-20)"), points and descriptor. If the levels are fixed percentage bands of each criterion's points, A pattern to start a criterion's levels from names each level and the percentage it starts at, and a criterion's Fill … level labels from the pattern button fills in its labels with their ranges; the points and descriptors you still type from the image.

As you type, anything that departs from the pattern the rubric itself follows is listed under Please check these against the image, and nothing is changed: the weights not adding to 100, a criterion with a different number of levels, a level's points outside its range, ranges that overlap, points that don't fall, and a level that starts at a different percentage of its criterion's points from the same level of the others (judged where three or more criteria give a range). A flag can be a slip in the rubric itself, so a flagged rubric can still be saved, as written. Have the AI read the image (optional). Typing the descriptors is the slow part, so the editor can ask the AI to propose the grid. It sends the image of the rubric through the local proxy, with its instructions and nothing else: no student's material, nothing from the brief. It needs the proxy running with your API key. Choose the Model and a spend limit, tick Replace what I have typed if you have started typing, and press Plan the image reading. You see the image (sent without its hidden metadata: names, places and the like that a picture can carry; your original stays as the rubric's source), the instructions it is sent with, and the worst-case cost against your limit; Confirm and send the image sends it, and Don't send it doesn't. What comes back fills the grid, marked as the AI's proposal. It can misread, so check every cell against the image and correct it; the checks below run on it as on anything typed. Nothing is saved until you save, and the saved rubric records that it began as the AI's proposal (which AI model, the time and the request's hash, not the image's content). Without a key the button says so, and you can still type the grid.

Save the rubric entered from the image keeps the image with the rubric as its source, with its hash, and records that it was entered from it, by you. Replacing a rubric already saved needs Replace the rubric already saved ticked. Import it before the original marking, which is mapped onto it.

1. On Rubric, choose the Rubric file. For a spreadsheet with several sheets, name the Spreadsheet sheet.
2. Press Read the rubric, and check the preview under Check the rubric before saving it: the criteria, their levels and points, and any warnings.
3. Under Criterion weights, check each criterion's weight, as a percentage. They start from the file; enter any it doesn't give (a rubric often has them only in its criterion titles, for example "Implementation 25"). The total is shown as you go. Weights are needed to work out an overall mark from levels on Review.
4. Press Save this rubric.

For the assessment brief, on Brief, choose the brief (docx or pdf) and press Import the brief. The Brief step is done once the brief is imported, as the submissions' steps are; anonymising and approving it is on Anonymisation, like a submission (step 10), which counts it with them. The AI reading sees only the approved brief.

A brief that describes several assessments. Under Which part of the brief applies, choose the part for this assessment. Only what you choose is anonymised, approved and sent to the AI; the rest is never sent, and isn't anonymised at all.

• If the brief has headings, each is listed with the number of words under it, and a part includes the headings beneath it. Tick the part for this assessment (for example "Element 1") and any module-wide parts that apply to it (the learning outcomes, the rules), then press Use these parts.
• If it has no headings (often a PDF), choose where the part starts and ends from the lists of its text, and press Use this range.
• The whole brief and the text this assessment uses are shown side by side, as written, on this screen only. Use all of the brief goes back to the whole.
• Choosing a different part clears the brief's anonymised text and its approval, so anonymise and approve it again; the same choice changes nothing. Importing the brief again starts from the whole.