What I did
I used Feedbacker on one real external moderation, from the moderation request to the exported record. I took the sample, the original marker's marked views and the rubric from the marking platform, anonymised and approved each text, asked for an independent AI reading of each submission, judged every criterion myself, gave a verdict on each submission's marking, and exported the approved record for the moderation form. I can't name the institution, module, staff or students.
What I found
- It saved time, and it didn't compromise my judgement: it aided it.
- The AI reading was useful. It gave an independent second reading of each criterion, with quoted evidence and a rationale, beside the original marking.
- Its first instructions read too generously. It placed work a level or more above the marking. I revised them so it reads the points on the UK scale, raises a level only when the descriptor is clearly met, and doesn't credit fluency.
- It judged a visual criterion from a description. It saw only the text, so on an assignment where charts carried the marks it relied on the student's description of them. It is now told not to, and to say there is too little evidence. Feedbacker can now also send the images you approve.
What it changed
Real use found problems the made-up coursework hadn't, and I fixed each before I finished: progress while the AI reads, each outcome shown beside its button, matching the marker's differently named criteria by picking instead of typing, out-of-date judgements flagged, a mark within each level, and the overall mark my criterion marks imply.
The limits
This is one moderation, by one moderator: my own experience, not a measured comparison. I reviewed openly, so I haven't gathered the cleanest evidence, which would compare blind first judgements with revised ones. The interface worked for the whole workflow but needs more work.