RSS Feedback has been live with my Sec 3 and Sec 4 Mathematics and Additional Mathematics classes for six weeks. One hundred and forty-three students. Many hundreds of submissions. Many more hundreds of AI hints.
When I sit at the teacher dashboard in the evening to review what came in that day, most of what I do is rubber-stamp. The AI graded; the AI's mark looks right; I accept. I am, in those rubber-stamps, mostly checking. The work is fast; the loop is short.
A small fraction of the reviews are different. The AI got it wrong, or partially wrong, or the AI's reasoning was right but the student's reasoning was also right and the AI did not see it. In those cases I override: I change the marks, I edit the feedback text, I flip a step's verdict from earned to not earned, or the other way.
That small fraction is where the work is. It is the only unbiased ground truth I have for what the tool is getting wrong. The bulk of teacher reviews that accept the AI's verdict are mostly a vote of confidence; the small overriding fraction is the actual signal.
This piece is about what that signal has been doing. Over six weeks, the patterns in my override pile-up have driven specific changes to the tool. Some are pedagogical: the rule for how reasoning gets judged. Some are about how the system holds a student's attention: when the canvas accepts more work, when a verdict is final but practice can continue. One is about the cost structure of running this at school scale.
I write here as the person on the inside of the loop, watching it run, and serving as its closer. The next essay in this series, when the data lets me write it honestly, will be about whether my students are learning more. This one is about what running the tool in my own classroom has been teaching the tool.
What the loop is
Most accounts of edtech AI focus on the loop the student is in: the AI presents, the student responds, the AI corrects. That loop is the visible one. The one I want to name here is the other one, the one between the teacher reviewing and the tool itself.
It works like this. A student submits a piece of working on an iPad. The AI grades the working against the rubric, awarding marks step by step, and returns either a celebration or a short hint. The submission lands in my review queue, with the AI's verdict pre-filled: marks per step, total marks, the AI's feedback note ready for the student to read once I have approved it.
I look. Sometimes I am looking during a free period, sometimes at the end of the day, sometimes late at night. Most of the time the verdict is right. I accept; the student sees their mark; the loop is closed.
Some of the time the verdict is wrong. The AI accepted something it should have queried, or queried something it should have accepted; or the marks per step add up but the reasoning behind them is wrong; or the feedback note is fine but the AI's understanding of what the student did is off. In those cases I override.
The overrides accumulate. There are not many in any given evening; over weeks they form a pile. A script in the codebase pulls the genuine overrides (the ones where I changed something, as distinct from the rubber-stamps) and reads them as a corpus. That corpus is the ground truth I have for what the tool is missing. When I find a pattern in it, the prompt gets tightened, the rule gets restated, sometimes the schema itself gets changed.
What follows is four times that loop has produced a specific change in the last six weeks.
Four changes the loop has driven
Changing how the AI judges reasoning
The deepest change of the six weeks was about geometry.
A student in one of my Sec 3 classes submitted a working that included a step where two angles were equal, with the student's stated reason being alternate angles on parallel lines. The marking scheme had been built around a different valid path: corresponding angles on the same parallel lines. The AI compared the student's stated reason against the scheme's expected reason and judged it wrong. The student lost the method mark.
When I saw the override I had to make I noticed something I should have set up the AI to handle from the start. The student's reasoning was correct: alternate angles did justify the deduction they were making, on the path they were taking. The marking scheme had chosen one valid reason; the student had used another. The AI had been doing what the schema told it to do, which was to match the scheme's chosen property, and that turned out to be the wrong job for an AI in this position.
The rule I had given the AI was: does the student's stated reason match the property the marking scheme lists for this step? The rule I changed it to was: does the student's stated reason validly justify the deduction they have made, on their path, in this configuration? It is a different question. The first is matching. The second is reasoning.
I also went and patched two questions that had been duplicated across the Sec 3 assignments, requiring both reasoning paths where the schemes had been lenient. That was the second-order consequence: when I changed the AI's rule, I had to make sure the marking schemes I had written were not still under the older assumption.
This change was driven by one student's working in one class on one question, but it produced a rule that now applies to every reasoning step in every math class on the tool. That is what the override loop, in its best form, does.
Keeping the canvas open past the cap
The second change was about what happens when a student runs out of AI hints.
The tool gives each student up to twenty hints per question. After that, the AI stops generating: the student sees a fixed message, the Submit Answer button changes to Save Work, and the teacher gets a note in the review queue that this submission needs eyes on it. The cap exists so that one student cannot run away with the day's AI spend, and so that students who are stuck do not stay locked in an infinite loop with an algorithm.
I had originally built the canvas to also lock at the cap. The thinking was: if the AI is not giving any more hints, there is nothing left for the student to do; they should wait for the teacher.
A student showed me I had thought wrongly. They hit the cap, the canvas locked, and they wanted to keep working on the page. They had ideas. They wanted to keep trying. The AI's silence did not mean the student's reasoning had stopped; it meant the AI was no longer helping. There is a difference.
I shipped the fix the same day. Past the cap, the canvas stays open. Submit relabels to Save Work; the student can keep iterating without the AI in the loop, knowing that whatever they save will be what their teacher sees when reviewing.
This was a small change in code and a real change in posture. The tool's job is not to govern the student's effort. The tool's job is to help the student while it can, and to step out of the way when its turn is done and the teacher's begins.
Letting students practise after the verdict
The third change was the inverse of the second.
When a student finishes a question, sees the AI's verdict, and gets my approval on it (the verdict that becomes their final mark for that piece of work), the natural thing is to move on. The mark is in. The question is closed.
A student wanted to do something different. They had got their verdict but they were not happy with what they had done. They wanted to keep practising the question, not for the mark (the mark was settled) but to see if they could solve it cleanly on a second pass.
The system would not let them. Once the teacher's verdict was logged, the canvas locked. The thinking on my side had been: the question is closed, the next thing for the student is the next question, the AI should not be wasted on a question whose mark is already final.
The student's request reframed it. The closure of the question for the marking is not the closure of the question for the practice. A student who wants to revisit what they did wrong, on a question whose mark is already settled, is doing the thing the whole tool exists to encourage.
I unlocked it. The teacher's verdict stays sticky as the final mark; the student can iterate after, up to the same twenty-hint ceiling, getting more AI feedback without the option of having the mark change. The mark is theirs. The practice is theirs too.
Raising the spend cap
The fourth change was about money and a student who tripped a guardrail.
Every student has a daily ceiling on how much AI can be spent on their hints. The ceiling existed so that a runaway loop, or a bug in my code, could not produce a single student's outsized daily session at the school's cost. It was set at a deliberately conservative number.
One student showed me the number was set too low. Over a long session at home, they made several dozen legitimate hint calls on a difficult assignment, hit the cap, and were blocked from making any further calls for the rest of the day. The cap was working exactly as designed. Every one of those calls was a real student-shaped use of the tool.
I raised the cap. The conservative version had been protecting against bugs that had not happened; the cost was a student who could not get help they wanted at a moment they wanted it. The raised cap accepts a higher worst-case dollar exposure in exchange for not cutting off real learning. If a real bug ever does show up, the daily error monitor catches it; the cap is the second line of defence, not the first.
Six weeks in, what I still cannot tell
Six weeks is too short to tell you whether my students are learning more.
The bulk submission count has grown each week. The variance across students has stayed wide: some complete every assignment, some complete one or two, some have not yet returned to the tool. The AI's feedback gets revised each day by something I notice in the review queue, so the version of the tool a student meets in week six is not the version they met in week one. None of that is the same as learning gains; none of it lets me make a claim about whether they are more confident, more accurate, more patient with their own working than they would have been without the tool.
The piece after this one will be about that, if and when the data lets me write it honestly.
This piece is about what I can tell you. The tool gets better because I am paying attention. The override pile-up is small but it is what counts. Every prompt iteration, every rule change, every UX adjustment is anchored to a specific moment in a specific classroom with a specific student. The work of running this tool in my own classroom has, in large part, been the work of being its closer: being the human who sees what the AI does not, and catching when the rule the AI has been following turns out to be the wrong rule.
What this term has shown me, beyond the loop itself, is what an AI assistant in the room actually changes about how learning works. Two things a math classroom has never quite had before, an AI assistant gives it. The first is personalisation at the scale of the class: a teacher with forty students can attend to a few at any moment, while the rest wait; an AI assistant can be in conversation with all of them at once, each conversation different, each one shaped to the working the student has in front of them. The second is presence during the working: feedback used to arrive after the working was done, in red on a marked paper or as a printed solution sheet handed back later; an AI assistant is in the room while the student is thinking, able to nudge before the working commits to a wrong path. Personalised, in the moment, at the scale of a real classroom. These are the two unlocks the tool is built around. The loop I have been writing about is how I make sure the assistant is actually delivering on them.