Feedbacker. Share

How data is handled

Data handling.

Purpose

This note defines how Feedbacker handles real assessment material: where it lives, what may leave the machine, and when it is deleted. It applies from the first use of real material. The rules on what the AI may be sent, in PRODUCT.md, take precedence if the two ever disagree.

This note describes intended practice. It is not a claim of legal, regulatory, or institutional compliance.

Feedbacker runs as a browser app served by a local Feedbacker proxy (ADR 0004). The Python command line, kept as the reference implementation, follows the same rules and uses the same workspace layout. Where the browser app handles something differently, the difference is stated beside the rule.

Before using real material

Confirm with the body that commissioned the moderation that AI-assisted moderation is acceptable, and that sharing anonymised extracts with a model provider is compatible with any confidentiality or data-sharing terms. Record the answer in the moderation workspace, not in the repository.

Moderation judgements remain the moderator's own. AI output is a second reading, never a decision.

Kinds of material

Kinds of material
MaterialExamplesSensitivity
Source filesOriginal files and marked versions with feedback (e.g. Turnitin current views), usually as bulk downloads; rubric; moderation requestPersonal data; confidential
ExtractsText extracted from source filesPersonal data; confidential
Assessment briefThe task set for the assessment; may name staff and give contact detailsConfidential assessment material; staff personal data
Pseudonym keyMapping from pseudonyms such as [STUDENT_A] to real names and identifiersPersonal data; most sensitive
Approved anonymised textRedacted extracts approved by the moderatorPersonal data (pseudonymised); confidential
AI readingsSuggestions, quotes, drafts, call recordsPersonal data (pseudonymised); confidential
Moderation recordJudgements, comparisons, and the pseudonymous exportPersonal data (pseudonymised); confidential
Re-identified exportAn export with real names restored, on explicit requestPersonal data; confidential
Synthetic fixturesFictional rubric, submissions, and marks for testsNot sensitive; committed

Pseudonymised material is treated as personal data while the pseudonym key exists, and in practice beyond that, because submissions can contain indirect identifiers that redaction misses. Derived material, such as AI readings that quote anonymised text, is classified at least as highly as its source.

Where material lives

  • All real material lives in a workspace: a folder on the educator's machine, outside any git repository. The browser app keeps every workspace it creates in one workspaces folder, ~/Feedbacker/workspaces/<name>/ (ADR 0008). Someone running the proxy can name another folder when starting it; the command line can use any folder.
  • A module groups assessments (ADR 0010). It is a workspace of its own kind, made by name in the workspaces folder and registered and checked as any workspace is, with each assessment an ordinary workspace in a folder inside it. The module holds once what its assessments share: the anonymisation rules, the brief, the module's context (the programme and module), and the request document as an anonymous cohort, which holds every assessment's rows, marks and weightings and which rows are one student, but no name and no ID. The pseudonym key is shared only if the module was made to share it, a choice made then and never changed; the trade-off is told where it is made. A workspace made on its own is a module of one. Everything that decides what may be sent (the approvals, the anonymised text, the sample and the records) stays in each assessment, and deleting a module deletes its folder and so every assessment in it. The Python command line does not open a module or an assessment in one.
  • The application refuses to open or create a workspace inside a git working tree.
    • In the browser app, this check is made by the local proxy, because a browser folder handle reveals neither the folder's path nor its parents. The proxy creates every new workspace, and refuses any path inside a git working tree.
    • The app creates a workspace by name only. The proxy accepts a plain folder name (letters, digits, spaces, hyphens, underscores and full stops, with at least one letter or digit, not starting with a full stop or a space, not ending with a space, and at most 64 characters; the app trims surrounding spaces before sending it), makes the folder inside the workspaces folder, and checks and registers it. It keeps the workspaces folder itself readable only by its owner, and outside any git working tree. It writes a random ID into the workspaces folder (feedbacker-workspaces.json), and the app keeps the folder the educator chooses only if it holds that ID, since it can't learn the chosen folder's path.
    • An existing workspace, such as one made by the command line, is registered by its path, and the proxy makes the same checks.
    • To show the educator's workspaces, the proxy lists those it has registered, with what each one's manifest says (its name, type, when it was created and its keep-for period; an older manifest without a type or period is a moderation kept for the default 90 days, as the command line reads it). It reads only the manifest, never a workspace's records. The app works out each workspace's progress itself, from the workspace's files.
    • The app opens only folders the proxy has confirmed as registered. The proxy re-checks the registered path each time and refuses it if the path now resolves somewhere else, for example because a folder above it was replaced by a link.
    • The app must also prove the folder it was given is the registered one. The proxy writes a one-time value into the registered folder, and the app must read it back through the folder it opened (inside the workspaces folder, or one the educator chose for a workspace kept elsewhere), then deletes it. A workspace kept elsewhere can be moved into the workspaces folder: the proxy renames the folder (never copying it) and keeps its registration, and the registry records where it was (ADR 0013). A copy of a workspace doesn't contain the value, so it is refused even while the original is still in place. The app also shows the registered path whenever a workspace is opened.
  • In the browser app, the browser keeps only folder handles, never any records: the workspaces folder's, which the educator chooses once, and that of any workspace opened from elsewhere. Deleting the workspace still means deleting the folder.
  • The browser app is supported in Chromium-based browsers (Chrome, Edge), which provide the folder access it needs.
  • As a second safeguard, .gitignore excludes common workspace, key, and export paths in case material is ever placed in the repository by mistake.
  • Only synthetic fixtures are committed. Real material, including anonymised extracts, is never committed, attached to issues or pull requests, or used as a test fixture.

Bulk downloads

Bulk downloads contain the whole class, not just the sample. Examples are a zip of every student's original file, or of every Turnitin current view. A sample may be spread across several downloads, for example a main submission point and a late one, split zip parts, or single files. Each sampled identifier is matched across all of them, and an identifier found in more than one place is reported, never guessed.

  • Only the sampled submissions, selected by the identifiers in the moderation request, are imported. Other students' files are never opened, and their names, identifiers, and marks are not recorded.
  • Each selected file is stored in the workspace under its pseudonymous ID (sources/originals/, sources/marked/). Its real file name goes only into the pseudonym key.
  • The original marking (grade, rubric levels, general and inline comments) is parsed locally from the marked views. Comment text is anonymised with the same tokens as the submissions before it is stored in marking/. Replaced records are kept in marking/history/ so corrections are recorded. Marking is never sent to a model. Where the marked views carry no marking, the marks the request listed (kept with it, under the pseudonym) can be used as the original marking instead: overall marks only, recorded as taken from the request, and unconfirmed until the moderator confirms them.
  • Where the marked views print the platform's own copy of the rubric, that copy (criterion names, weights, maxima, and levels with their points: no student's mark, since the total printed beside the rubric's name is not kept) is kept as embedded-rubric.json, beside the marking and never as the source rubric, and is compared with the saved source rubric on the Rubric screen. It is never sent to a model.
  • To understand an unfamiliar file or archive, use feedbacker inspect. It reports structure only, with archive file names shown as shapes. Never share real files, or paste their content, into issues, pull requests, or AI chat sessions.
  • File names inside the archive may contain names or identifiers. They are recorded only in the pseudonym key.
  • A request document (a Word file, or a PDF that has text) may be read on the Request screen. Its tables and context are read in the browser and nothing is kept of the document itself. A PDF's rows and columns are recovered from where its text sits, so what can't be told apart reliably (a cell across two columns, text that wraps between cells, a table laid out differently on another page) is left out and said, never guessed, with one exception: a student's name wrapped onto a second line, in the name column, that sits closer to its own row than to the next line and has a student ID in the row above, is joined to the name and said (how many, never what), and a PDF that is only pictures of pages is refused (there is no OCR). Staff named in it (a convener, a marker) are never read. A listed student's ID and name go only into the pseudonym key, from which anonymisation learns the name; the other students it lists are not recorded by name or ID. What is kept of the rest of the cohort is anonymous (request-cohort.json): for each assessment, its rows by number with the initial and agreed marks and whether the student submitted, the grade bands, which rows were sampled, and the document's hash. The marks it lists for the sampled students (initial and agreed, as percentages) are kept with the request, under the pseudonym, to check a marked view's grade against.
  • When the request's IDs aren't in the downloads' file names, sampled students are matched to files by name, on file names only: no file's content is read. The moderator confirms every match, and the matched file's marking-platform ID is recorded in the pseudonym key (it is redacted from text like the other IDs).
  • Bulk downloads are not copied into the workspace; only the selected files are stored there, and only their hashes and the downloads' hashes are recorded. Delete the downloads from wherever you saved them once importing is done.
  • In a moderation, importing the whole cohort, for example to check a reported band distribution, requires a recorded decision.

A marking cohort

In a marking workspace the educator marks the whole cohort, so every submission in the bulk download is imported, not a sample (maintainer decision, 2026-10-04).

  • The platform names each file with the student's ID and name: Turnitin's <ID> - <NAME> - <file>, or Canvas's <name>_[late_]<user ID>_<attachment ID>_<file>. Feedbacker reads the ID from the name. A file whose name doesn't follow either form is listed (by its name's shape, never its name) and not imported, and so is an ID found in more than one file: nothing is guessed. The platform's report on the download (a .txt file) is not opened.
  • Each submission gets a submission ID and pseudonym, recorded in cohort.json, which holds nothing else. The real ID and the file name go only into the pseudonym key, so feedback can be matched back to each student for release. The Submissions screen shows each real ID beside its pseudonym, as a moderation's Request screen does (see Pseudonym key). Students' names are taken from Turnitin-style names for anonymisation; Canvas runs the name's parts together, so add a name to the anonymisation rules if it isn't redacted.
  • Importing again adds the submissions that are new (late ones, for example), and everyone keeps their pseudonym. A submission already imported is replaced only when the educator asks.
  • Each file is stored and extracted as an original is, and then anonymised, reviewed and approved in the same way. The educator gives the approval.

What may leave the machine

Only what PRODUCT.md's rules on what the AI may be sent permit:

  • approved anonymised submission text, sent only if its hash matches the educator's approval;
  • the approved anonymised assessment brief, under the same rule, with staff names and contact details redacted (add staff names on Anonymisation, under Add to the rules; emails, URLs, and phone numbers are caught automatically). Where the brief describes several assessments, only the parts the moderator chose for this assessment are anonymised, approved and sent, and the choice (as ranges of the extracted text) is kept with the brief;
  • the rubric's criteria and levels (a rubric entered from an image keeps that image in the workspace, as its source; Feedbacker doesn't read it, and it is sent only if the moderator asks the AI to propose a transcription, below);
  • Feedbacker's versioned instructions to the AI;
  • when drafting feedback only (ADR 0006): the educator's own final marks and comments for that one submission, and their feedback guide for the assessment, anonymised and approved like a submission;
  • when suggesting an edit only (ADR 0006, amended): the educator's recorded feedback on one criterion (or overall), anonymised, with the checks' flags on it, and their marking of that criterion; not the submission;
  • when suggesting a verdict only (ADR 0011): the rubric and one block of the moderation of one submission, shown to the moderator before they confirm: the original marker's marks, the moderator's own levels, marks and comments, and the AI reading's levels and rationales; never the submission, the brief, the marker's comments or the AI's quoted evidence;
  • when suggesting an overall comment only (ADR 0011): one block alone, of how the sample was chosen and, for each sampled submission, the original marker's overall mark and the moderator's own verdict, marks and comment; no rubric, submission, brief or AI reading;
  • when asked to propose a rubric transcription only (ADR 0009): the image of the rubric, alone, without its hidden metadata, with its instructions, after the moderator has seen what is sent and its worst-case cost and confirmed. It carries no student's material, and the proposal is only ever a draft for the moderator to check and save;
  • with a reading or proposals only (ADR 0007): the submission's figures that the educator reviewed and approved with its text, each sent as an image after its placeholder, only if its stored image still matches the hash approved, and only in a type the AI accepts (JPEG, PNG, GIF, WebP). Every other figure is marked "(figure not sent)" where it was. A run can leave every figure out (Include each submission's approved figures).

The pseudonym key, source files, document metadata, real names and identifiers, and any other student's material never leave the machine. Nor do the original marker's marks and comments, or the moderator's judgements, in moderation. A reading of a submission, which suggests levels, is never sent anyone's marks or comments, so it stays independent.

The local web interface binds to 127.0.0.1 only and is not reachable from other devices.

In the browser app, the local Feedbacker proxy is the only way anything leaves the machine:

  • The app's Content Security Policy allows network connections only to the proxy, and loads no third-party scripts.
  • The proxy binds to 127.0.0.1 only, checks the Origin and Host headers, and requires a per-session token that it gives the app at start-up.
  • It forwards only requests that carry the fields those rules permit, and the educator's marks, comments and guide only in a drafting request, their feedback only in a suggestion request, and the moderator's judgements only in a verdict or overall-comment suggestion request. It also runs leak checks for identifier patterns and refuses anything that fails them. These checks are a backstop; the app's approval gate is the control. The proxy says only the kind and the count, never the text, so that what it refuses can't be carried by its refusals or its logs. The app runs the same patterns itself before it plans a reading, proposals, drafted feedback, a suggested edit, a suggested verdict or a suggested overall comment, and shows the moderator, on their own screen only, exactly what matched and where (never sent, never logged). A match in a submission, the brief or the sample (its note) can be added to the anonymisation rules; one in the rubric, the guide or the educator's own marking or feedback is changed where it was written, because the rules don't reach it, and the screen says so.

Anonymisation

  • Text is extracted locally, and document metadata (docx author and properties, pdf info) is discarded, never carried into extracts.
  • Redaction runs locally and uses consistent pseudonyms: [STUDENT_A] for a sampled student, and [PERSON_n], [ORG_n], [ID_n], [EMAIL_n], [URL_n], [PHONE_n] for other values. The values behind the tokens are kept only in the pseudonym key.
  • Students' names come from the pseudonym key, including names taken from Turnitin-style file names (<ID> - <NAME> - ...). The moderator's added names, organisations, extra values, and false-positive exceptions are kept in anonymisation/rules.json, which is private (mode 600).
  • Text the educator writes (a comment on a criterion, an overall comment, a verdict's comment, the feedback guide, the note on how a sample was chosen) is anonymised the moment it is kept, in one place in the code, with the same pseudonym key and rules as the submissions and the brief. Anonymising again, with newer rules, applies them to the comments already kept.
  • The review of each text also lists anything in it that still looks like an identifier (an email address, a web address, a phone number, a number of seven or more digits), with the text around it, on the moderator's own screen only. These are the same patterns the proxy refuses on, so they can be dealt with, in one action that adds them to the rules, before a reading is planned. A row of results (0.156 0.20 0.206 0.00 0) is not taken for a phone number: a phone number is a plus, or a zero that isn't the tail of a longer number, then 9 to 14 more digits in groups, and the whole run (see the proxy's README). Nothing is redacted that the moderator has not seen and ticked (matches past the first 30 are added only once shown), a value they had told the rules to ignore is taken off that list when they add it, and the text is approved again after it changes. A match in the rubric can't be put right by the rules, because the rubric is not anonymised: it is corrected and imported again.
  • When a check stops something because the rules or the key now redact more of a text that was anonymised earlier (the record's approval, a reading or drafting plan, a suggestion), the app lists what they would redact, where, and with the text around it, on the moderator's own screen only (never sent, never logged); the message itself still names the text and never what was found. The moderator may keep a value that a built-in pattern found (it is added to the values not to redact), but a name, an organisation, a student's ID or a value they asked to redact is never offered to keep, and nor is a value that the identifier check before a request (and the proxy's) would still refuse, in text that is about to be sent.
  • The moderator reviews every redaction, can add or undo redactions, and approves each submission explicitly. Approval records who approved, when, and a hash of the approved text. Any change to the anonymised text clears its approval. Many texts may be approved in one action, after review by exception (ADR 0012): the app flags the texts likely to hold something to redact, saying why and never quoting them, and the moderator opens each flagged text and confirms they skimmed the rest. A flagged text that has not been opened is never approved by it (a submission with an included figure is always flagged, so its figures are looked at), the confirmation of the skimming is of the texts as listed and lapses if they change, and each approval records that it was made in a set, how many were flagged, and whether its text was flagged and opened.
  • Every model call obtains text only through the approval gate (feedbacker_core.boundary), which refuses anything that is not exactly the approved text.
  • Automated redaction can miss indirect identifiers, such as employers, repository or portfolio URLs, usernames in screenshots, and personal reflections. The moderator's review is the control for these, not a formality.
  • Figures. Extraction, on this computer, also keeps each figure (an embedded image: a chart, dashboard, screenshot or diagram) of a submission, and marks where it was in the text with a placeholder such as [FIGURE_1]. Alternative text the student gave a docx figure follows its placeholder as a line of the text (Alt text: …, the paragraph straight after its placeholder), so it is anonymised, reviewed, approved and sent with the rest of the text, and describes the image in the review. A docx figure is kept as its own image file, without its hidden metadata (one in a type that can't be cleaned isn't kept); a PDF figure as its pixels, saved as PNG. Images smaller than 32 points either way (bullets, icons, rules) are left out. Each is kept in the private area, private/figures/<submission>/ (mode 600), with its hash, page, size and media type in the submission's record, and is deleted with the workspace. Redaction can't see inside an image, so a screenshot may show a name, an email, a username or a face: the educator's review is the control. Hidden metadata the review can't see (a photo's camera details or location, an author's name) is removed when a docx figure is extracted, before it is kept or hashed: only what draws the picture is kept (EXIF, XMP, IPTC, text chunks and comments go, wherever they are in the file). This is done for the types the AI can be sent (JPEG, PNG, GIF, WebP); an image in another type (EMF or TIFF, say), or one that can't be read, is marked where it was, but its file isn't kept. A PDF figure is rebuilt from its pixels, without any. A figure kept before metadata was removed isn't sent until its submission is imported again. Each figure is shown, from memory only, where it was in the text under review, and every one is included unless the educator chooses not to send it (with a reason, if they give one); an excluded figure stays in the workspace, and its placeholder in the text, marked as not sent. Approving a submission approves its text and its included figures: the approval records each one's hash, and any change to which figures are included clears it. Approved figures are sent with a reading or proposals (ADR 0007): the provider's limits are checked first (a submission beyond them is read only once the educator chooses to send those that fit, or none), each reading records the figures it was sent by hash and the placeholders of those it wasn't, and the egress log holds only their hashes. An institution must confirm that its agreement with the provider covers images of students' work. Anonymisation never redacts a placeholder.

Exports

  • Exports are written into the workspace's exports/ folder by default and are named <name>.feedbacker-export.<ext>, a pattern that .gitignore also blocks.
  • Saving an export elsewhere is an explicit choice. The application refuses any destination inside a git working tree.
    • The browser app can't check a save destination, so it writes exports only into the workspace's exports/ folder. Moving an export elsewhere is the moderator's own action, outside the app's safeguards.
  • The moderator approves the moderation record before anything is exported, and an export is refused if the workspace has changed since the approval.
  • Anonymisation is re-checked before anything is sent and before anything is exported. The pseudonym key and the rules can grow after a text was anonymised (a later import can add a name; the moderator can add a rule), so a text the current key and rules would still redact is never sent to a model, and the record can't be approved or exported while any text in it would still be redacted. "Anonymise now" brings the submissions, the brief and every stored comment up to date.
  • The structured record (JSON) never contains the extracted original text: each submission carries only its approved anonymised text, the redaction offsets, and its approval. Pseudonyms stand in for names and external identifiers throughout.
  • Exports are pseudonymous by default. A re-identified export requires an explicit request each time. It is labelled as containing personal data, stored only in the workspace unless the moderator moves it, and deleted with the workspace.
    • It restores only the sampled students' external identifiers (such as Turnitin submission IDs), in place of their pseudonyms: never names, and never any other redacted value, which stays as its token (maintainer decision, 2026-09-27).
    • It is a copy of the readable summary, as Markdown and as a Word document. The structured record always stays pseudonymous.

Pseudonym key

The key maps each pseudonym to the real name and to any external identifiers, such as Turnitin submission IDs or VLE user IDs, for a moderation's sample or a marking cohort alike. External identifiers link directly to a student, so they are treated like names: they live only in the key and never appear in anything sent to a model. They reappear in three places only, all on the user's own machine:

  • the Request screen, which shows each sampled submission's external identifier beside its pseudonym (maintainer decision, 2026-10-02). It is where the identifiers are entered, and where the moderator matches pseudonyms to the moderation form and the marking platform;
  • a marking workspace's Submissions screen, which shows each submission's external identifier beside its pseudonym in the same way (maintainer decision, 2026-10-04), so the educator can match pseudonyms to the students in the marking platform. Every other screen stays pseudonymous, and real names appear only when asked for on Anonymisation ("Show the real values");
  • a re-identified export, for example when a moderation form needs them.

Showing them on screen means anyone who can see your screen, for example during a screen share, can see them. Close the Request or Submissions screen before sharing your screen.

The key is stored only in the workspace, at private/pseudonym-key.json, separate from the extracts and readings. The private/ folder is readable only by the moderator's user account (mode 700), and the key file only by the moderator (mode 600).

A browser can't set file permissions. For the browser app, the local proxy sets them when it creates or registers a workspace. It confirms the workspace whenever the app opens it and after every private write:

  • it refuses a workspace whose own folder is no longer readable only by the moderator;
  • it restores 700 for every folder inside and 600 for every file, because what the browser writes gets the system defaults. That is safe inside a 700 folder, but the stricter modes are restored anyway.

The key is append-only. Once assigned, a pseudonym always refers to the same identifier and is never reused, even if the sample changes, so no record can end up pointing at the wrong student. It is used only locally, to re-identify an export when the moderator explicitly asks for that.

Model provider requirements

The provider must:

  • exclude API inputs and outputs from model training by default;
  • retain data only for a limited period, for abuse monitoring or as its terms state;
  • be called only through the provider interface, which enforces the approval check and records the model, provider, prompt version, input hash, token usage, and timestamp for each call. In the browser app, the interface reaches the provider through the local proxy.

The first adapter is the Anthropic API (see ADR 0003). Review the provider's current terms before first real use, and again whenever they change.

Prompt caching. Nothing more is sent, but the provider may keep part of each request in its prompt cache for five minutes, so later readings in a run can reuse it at a lower price. That part is the prefix every reading shares: the instructions, the source rubric and the approved anonymised brief. The submission comes after the cache marker and is not part of the cached prefix.

Batches. When the moderator sends a run as one batch, the same approved requests are sent, all at once, and the provider processes them within a day. It keeps the results for 29 days after the batch is created, where the proxy can collect them. The proxy records the batches it sent (~/Feedbacker/proxy/batches.json, mode 600) with request hashes, models and amounts, but no text, and only those batches can be checked, collected or cancelled. The workspace records what it sent (readings/batches/), and the batch's results become readings only if the approved texts, rubric, brief and approvals are still exactly those sent.

Reused readings. A completed reading is kept in the workspace (readings/reuse/), and reused, with nothing sent, only for the same submission when everything that would be sent is identical. It is never used for another submission, and it is deleted with the workspace. See ADR 0005.

API keys and spend

  • The moderator's API key is read only from local configuration: an environment variable or a gitignored .env file. It is never logged, exported, or written into readings or audit records.
    • In the browser app, the key belongs to the local proxy's configuration and never enters the browser.
  • The proxy keeps an egress log of every request it forwards or refuses: the time, model, request hash, token usage, cost and outcome. The log holds no submission text and no names; the text stays in the workspace's call records. It is ~/Feedbacker/proxy/egress.jsonl (mode 600), next to the proxy's workspace registry (registry.json, mode 600) and its record of the batches it sent (batches.json, mode 600), in a folder only the moderator can read (mode 700).
  • Spend is bounded by:
    • a token and cost estimate the moderator confirms before each batch run;
    • a configurable limit per run, which halts processing when reached (in the browser app, the proxy enforces it);
    • a monthly spending limit set in the provider's console as a backstop.
  • Token usage is recorded with each AI reading.

AI readings

  • A reading sends one request per submission: the versioned instructions, the rubric, the approved brief, and the approved anonymised submission. The brief is included unless the educator unticks Include the approved brief, which is recorded in the run log. Immediately before each request, it is rebuilt from the current approved material and must equal the request the moderator confirmed; otherwise nothing is sent for that submission. The original marker's marks and comments are never sent.
  • In a marking workspace the same request asks for the AI's proposals, with instructions written for the educator (marking-v4): a level for each criterion, with reasons and quoted evidence, and a short draft comment for the student, which the educator may adapt (a comment adapted from it is recorded as derived from the AI). It is sent exactly what a moderation's reading is sent, and never the educator's marks or comments (ADR 0006). The provisional mark is worked out in Feedbacker from the proposed levels and the rubric's weights; the AI never gives a mark.
  • The educator's levels, marks, comments and overall mark are kept in judgements/ (private). Their comments are anonymised with the submissions' tokens when they are saved. When a submission is marked blind, its proposals and provisional mark are not even loaded until the educator has recorded a level for every criterion.
  • The default AI model is Claude Sonnet 5. If it declines on safety grounds, the same approved request is sent once to Claude Opus 5, and both calls are recorded.
  • Readings are stored as AI suggestions in readings/. Every call, including refused, truncated, and failed ones, leaves its call record in readings/calls/ and its raw response in readings/raw/. Each run's log (models, token usage, costs, outcomes) is in readings/runs/. All are private (mode 600) and are deleted with the workspace.
  • Every run shows a worst-case cost estimate (maximum output, plus a possible fallback call) that the moderator confirms, and it stops before exceeding its spend limit (default $5).

Drafting feedback

Decided in ADR 0006; built in marking and feedback.

  • A draft is requested for one submission at a time. It is sent what a reading is sent, plus the educator's own final level, mark and comment for each criterion, their overall mark and comment, and, if they have written one, their feedback guide for the assessment. It is never sent the AI's own earlier suggestions, or anything about another student.
  • The educator's comments and guide are anonymised with the same rules and pseudonym key as the submissions when they are saved. Before a draft is requested, the educator sees exactly what will be sent, and asking for the draft approves it; the request must match that approval when it is sent, or nothing is.
  • Each draft records what it was drafted from, so changing a mark marks that draft out of date. Drafts are stored with the workspace's other records, and deleted with it.
  • As built: drafting is on the Feedback step of a marking workspace, for one submission or a batch across the cohort, with the instructions feedback-v4. The marking is sent as its own block, after the submission, and the plan shows it exactly as it will be sent; confirming approves it, and each request is rebuilt just before sending and must equal what was confirmed. The proxy accepts that block only in a drafting request. Only criteria whose marks are current are drafted, and the overall summary only once the overall mark is current.
  • The feedback guide is written by the educator for the assessment, on the Feedback step. It is anonymised when it is saved (only the anonymised text is kept, in feedback/guide.json, private), each save is a new version, and it is sent only once the educator approves exactly that text. It is sent with every drafting request, as its own block before the submission, and each draft records the guide version it used; a batch sends the same guide and instructions for every submission. It is the only thing shared across submissions' requests: no other student's material is ever sent.
  • Drafting a criterion again after a praise flag may add a line to the marking block naming the words to avoid. They come from Feedbacker's own word lists, never from the student's text, and the plan shows them as they will be sent.
  • Suggesting an edit (ADR 0006, amended): on a flagged piece of recorded feedback, the educator can ask the AI to suggest the smallest edit that deals with the flags. It is sent the rubric, the educator's level, mark and comment for that criterion (for the overall feedback, the overall mark and comment and each criterion's level and mark), and the recorded feedback, which is anonymised, with the checks' unaccepted flags on it, with the instructions feedback-edit-v1. It is not sent the submission, the brief, the guide or the educator's other feedback. The plan shows everything that will be sent, the rubric included, exactly as it will be sent; confirming approves it, and the request is rebuilt and must match before it is sent; the proxy accepts the feedback block only in a suggestion request. The suggestion is kept in feedback/suggestions/<id>.json (private), naming the feedback it edits, and never replaces the educator's text: feedback recorded from it is derived from the AI, and only while that feedback is unchanged. The structured record carries every suggestion, as it does every draft.
  • If a rule or name added since would redact more of a comment that a drafting or suggestion request would send, nothing is sent until that mark is recorded again, and so anonymised as the rules are now.
  • The cohort view (feedback side by side by level, with outliers flagged) is worked out on this computer; nothing is sent.
  • Approving and exporting marking: each submission is approved on exactly what its student will receive; the approval (feedback/<id>--approval.json) records a digest of it, so any change clears it. The standard exports (each student's feedback, one file of them all, a marks table, and the structured record) are pseudonymous. A re-identified copy of the feedback and marks table, made only when the educator confirms it each time, restores each student's platform ID in place of their pseudonym, and nothing else, as in moderation; the record always stays pseudonymous. Feedback still containing an anonymised value (a token) is flagged before it can be approved.
  • Drafts are in feedback/drafts/, and the educator's feedback in feedback/ (anonymised, as their comments are). Feedback adapted from a draft is recorded as derived from the AI however much it is changed. Every call's record and raw response are in feedback/calls/ and feedback/raw/, each run's log in feedback/runs/, and replaced drafts and feedback in feedback/history/; all are private, and deleted with the workspace.
  • Feedback is checked against its mark in Feedbacker, on this computer: no request is made. A flag the educator accepts is kept, with their reason, on that feedback record. The workspace's praise words are in feedback/praise.json.

Why there is no authentication

Feedbacker runs only on the educator's machine, has no hosted endpoint, and uses the educator's own key. The local proxy is reachable only from the same machine and only with its per-session token. Anyone running the open-source code supplies their own key and pays for their own use. There is nothing for another person to sign in to, and no shared key to drain.

Authentication and server-side spend controls become necessary before any hosted deployment, and will need their own recorded decision.

Retention and deletion

The retention rule for a moderation:

  • Start the clock. When a workspace is created, record the commissioning body's retention or query-period requirement in it. If none is given, use a default of 90 days after the moderation report is submitted.
  • Delete at the end. When the report has been submitted and that period has closed, delete the whole workspace: source files, extracts, anonymised text, approvals, AI readings and cache, the pseudonym key, and any re-identified export. The app does this in one action, only once the workspace's name is typed, and after listing the exports (with their full paths) so the moderator can keep what is required, and confirm that they have: it deletes the folder and everything in it, the proxy forgets its registration (so no path to it is kept), and the browser forgets the folder. The downloads the moderator made are theirs to delete.
  • Keep only the required record. Retain only the moderation record the moderator is required to keep, in pseudonymous form unless the commissioning body requires names. Keep it outside the repository, for no longer than that body requires.
  • Never share the cache. Cached AI readings are never shared between workspaces.
  • Remind the moderator. The app records the retention period when a workspace is created, but doesn't yet remind anyone when it has passed: deleting at the end is the educator's own action. It never deletes without confirmation.
  • Provider retention is governed by the provider's terms (see above).
  • The proxy's egress log holds no assessment content. Keep it only while spend needs checking, and no longer than the longest retention period of the workspaces it covers. The proxy removes entries after 90 days by default (--egress-retention-days).

The application should offer a single action that deletes a workspace, and confirm what it removed. The browser app deletes the whole workspace folder in one action, after the moderator types the workspace's name.

Incidents

If real material is committed, posted publicly, or sent outside the boundary:

  1. Stop processing.
  2. Remove the material from wherever it was exposed. For git history, rewrite it and treat the material as exposed regardless.
  3. Record what happened, privately and outside the repository.
  4. Tell the commissioning body where its terms require it.
  5. Fix the safeguard that failed before resuming.

This page is generated from docs/data-handling.md in the code, so the two are the same words.