Marking a cohort.

18. Marking a cohort

Feedbacker can also hold your own marking of a whole cohort, in a marking workspace. So far, it takes you from importing the submissions to marking them, with the AI's proposed levels if you want them, to writing each student's feedback from your marks, and to approving and exporting it to paste into the marking platform.

1. Press New marking on Home (or choose Marking at the top), give it a name, and press Start the marking; Options sets how long it is kept for, as in step 4. Its name, above the steps, says it is a marking workspace.
2. On Details, record the assessment's title, and its module and programme if you like.
3. Import the rubric and the brief as in step 6.
4. From the marking platform (for example Turnitin, through the VLE, or Canvas), make a bulk download of the students' own files. Save it outside the Feedbacker folder.
5. On Submissions, choose the download's zips (and any single files, named as the platform names them), and press Import the submissions. Every submission is imported and gets a pseudonym. The student's ID and name, read from the file's name, are kept in the private pseudonym key; the table shows each submission's real ID beside its pseudonym, on this screen only, so you can match them in the platform. A file whose name doesn't carry an ID, an ID found in more than one file, and a file that isn't docx or pdf are listed, not imported: put it right in the download and import again.
6. If late submissions arrive, download them and import again: only the new ones are added, and everyone keeps their pseudonym. Tick Replace submissions already imported only to replace files you've already imported.
7. On Anonymisation, set the rules, anonymise, review and approve each submission as in steps 9 and 10. Canvas file names run a student's names together, so check their names are redacted, and add them to the rules if not.
8. Delete the downloads once importing is done, as in step 16.
9. Optionally, on AI proposals, ask the AI to propose a level for each criterion of each approved submission, with its reasons and evidence. It works as the AI reading does in step 11: you see a worst-case estimate first, and nothing is sent until you confirm it. Your marks and comments are never sent.
10. On Marking, choose a submission and press Mark this submission. Choose whether to see the AI's proposals while you mark (Show the proposals) or to mark blind (Mark blind); the choice can't be changed. Blind marking hides the proposals until you have recorded a level for every criterion and press Reveal the AI's proposals.
11. For each criterion, choose your level (or press Take the proposed level, or Take the proposed level and record to take it, record it and go on in one action, for a criterion where the proposal is what you would mark), adjust the mark within it, write a note if you want one (or press Start from the AI's draft note and adapt it; it is recorded as derived from the AI however much you change it), and press Record and go to the next criterion. The Provisional mark comes from the AI's proposed levels: it is never your mark, and it is always shown apart from it.
12. After the last criterion, your Overall mark starts from the mark your criterion marks imply; change it if you need to, and press Record the overall mark (or Record and go to the next submission, which records it and opens the next one). You can also write a note on the overall mark: it is optional, it is your own record, and the student doesn't see it (what the student receives is written on Feedback). Like the notes on each criterion, it can be sent to the AI when you ask for feedback drafts, once you have seen exactly what would be sent and confirmed it. Next submission opens the next one. If you later change a criterion, the overall mark is flagged for you to check again.
13. The Feedback screen has three parts: a guide to how you give feedback (optional), first drafts from the AI (optional), and writing and recording each student's feedback, which is the part that counts. A Feedback guide tells the AI how you usually give feedback, so every student's draft starts from the same place; if you want one, write what students at each level of each criterion usually need to hear, and the common next steps (the screen gives an example). You don't need one. Press Save the guide (it is anonymised, and becomes a new version), read it as it will be sent, and press Approve this guide for the AI. It is then sent with every draft, so similar work starts from the same place; untick Include the approved feedback guide to draft without it. Each draft records the guide version it used. Under Draft feedback (optional), choose Every marked submission that needs drafts (or one submission), tick Send as one batch, at half the price if you like, and press Plan the drafts: this sends nothing, it shows what would be sent. Check what will be sent lists each submission, what will be drafted and the most it could cost; open What will be sent of your marking of … to read exactly what of your marks and notes will be sent. Nothing about any other student is sent. Press Confirm and send (or Confirm and send the batch), or Don't send. While the AI works, the screen says so under the buttons, with how long it has been. When the drafts arrive, What came back says what was drafted, what it cost, and offers Read and edit the drafts for each submission. The overall feedback is drafted only once your overall mark is recorded and current.
14. Under Write the feedback (press Read and edit the drafts in What came back, or choose a submission and press Write this submission's feedback), each empty box starts with its current AI draft, which is not saved until you record it: for each criterion and then Your overall feedback, read it, change it as you need to (or clear it and write your own), and press Record the feedback (or, once every box is as you want it, Record all pieces of feedback records them all in one action, and Record all and go to the next submission then opens the next one, unless a flag is still to check, or something is left to record, in which case it stays and says why). Text you type while it saves is kept and not recorded; and if a piece fails to record, it says which, the pieces before it stay recorded, and the rest stay in their boxes Anything you have already recorded or typed is kept as it is; Start from the AI's draft puts a newer draft in its place. Feedback adapted from a draft is recorded as derived from the AI, however much you change it; Clear and write my own undoes that. Your feedback is anonymised when it is saved.
15. If you change a mark or note afterwards, that criterion's draft and feedback are flagged as out of date (and the overall feedback's, which rests on every mark). Press Plan another draft on that criterion to draft it alone, then check your feedback and record it again. Plan another draft is on every criterion and the overall, whenever it is marked: use it too when a draft is current but not good enough (cut short, say). Nothing is sent until you confirm the plan. Once it is drafted, you are taken back to that criterion: if its box still held the old draft unchanged, the new draft is now in it, not yet recorded, so read it and record it; if you had changed the text, your text stays, and Start from the AI's draft puts the new draft in its place.
16. Each piece of recorded feedback is checked against its mark, in Feedbacker (nothing is sent): praise that belongs to a higher band than the mark's (for example "excellent" on a 58), no "Next time" step, another mark or level named (for example "68%" on a 62, or "2:1" on a lower second), an anonymised value (a token such as [ORG_1]) the student would see, and text that ends mid-sentence (as a draft can, if the AI's reply is cut short; draft it again). Flags appear under each piece of feedback, and Checks across the cohort lists how many each submission has. While a box holds text you haven't recorded (a new draft, or your changes), the checks shown are of that text as it stands; a flag can be accepted once it is recorded. Praise that is negated or said to be missing ("lacks an effective structure", "rather than a demonstration of effective …") isn't flagged. A flag never blocks you: change the text and record it again; press Plan another draft, avoiding "…" to have the AI draft that criterion again without the flagged praise words (they are added to what is sent, and shown in the plan); press Plan the edit for this flag to have the AI suggest the smallest change to your own recorded text that deals with the flags, keeping your wording (you see exactly what will be sent first: your feedback, its flags and your marking of that criterion, never the submission; press Confirm and send, or Don't send; the suggestion appears beside your feedback with its own checks, and Use the suggestion puts it in the box to record, as derived from the AI, or Don't use it); or keep the text by pressing one of the Accept: reasons, or typing your own and pressing Accept with this reason. The reason is kept with that text; recording new text clears it. A figure quoted from the submission (such as its test coverage) may be flagged as a mark; accept it with that reason. Praise words to check against the mark (under Checks across the cohort) lists the words it looks for: for each band, the words that suit that band's work, which are flagged in feedback on a mark below it (for example, "excellent" is flagged on a mark below 70%). They are checked in your feedback, not in the submission, and you can change them for this workspace.
17. Feedback across the cohort shows, for each criterion, every student's recorded feedback grouped by your level, and the overall feedback by band. Feedback much shorter or longer than the rest at its level (under half, or over twice, the median), and feedback nearly the same as another student's, are flagged. Edit opens that student's feedback.
18. On Export, press Read what they receive for a submission: it shows exactly what that student will receive, the overall mark, then each criterion's mark and feedback, then the overall feedback. A submission can be approved once everything is marked, its feedback is current, and every flag is accepted; if it isn't ready, it lists what is left. Read it, and press Approve (approving releases exactly this mark and feedback, to be copied or exported, and shows the next submission that is ready, so each takes one press). Approval is of what you were shown: if it changed after it was shown (a change still being recorded, say), nothing is approved, and what there is now is shown for you to read. If something needs changing, press Edit this feedback beside that part: it opens that submission on Feedback at that piece, with Back to Export to return once you have recorded your change and read it again. Any later change (a mark, the feedback, a flag's reason) clears the approval.
19. For an approved submission, Copy the mark, Copy each criterion's feedback, or Copy all the feedback, and paste it into the student's comment box in Turnitin or Canvas. Export the approved feedback and marks writes, into exports, each approved student's feedback as text, one file of them all, a marks table (CSV) and the structured record (JSON). All of these are pseudonymous.
20. To match students in the platform, press Make a re-identified copy: as in moderation, it asks first, and on Make the copy writes the feedback files and the marks table with each student's platform ID in place of their pseudonym, and nothing else restored. It contains personal data, is kept only in this workspace, and is deleted with it. A token such as [ORG_1] left in someone's feedback stays a token, so the checks flag it before you approve.