Why AI Feedback Often Fails the Transfer Test
education exchange
Who remembers the first time they tried to ride a bicycle? I certainly do. It was with my friend Simon and his family, a year older than me. Simon was already a pro. I remember our parents running behind my bike until they eventually let go, and I could zoom around the park by myself. I also remember that when I eventually had to stop the bike, I lost my balance and fell into a deep pile of stinging nettles!.
If my parents hadn't let go of the bike, I would have developed what we call tool-bound competence; I could "ride" the bike, but only because they were doing the hard work of balancing for me. This is the exact challenge we risk with the latest surge of AI feedback tools. A recent study into AI-supported storytelling reveals a fascinating, if slightly cautionary, tale about the difference between performance and learning.
The study
A recent article by Zhang et al (2026), shows both the promise and the risk of AI support. One of the limitations of previous digital technology is an inability to provide the true individualised micro-adaptations of support (Sibley et al., 2025). While AI has provided a quantum leap in this ability, not all AI is created equally, and this study highlights a significant design risk.
- The study investigated how different levels of AI support influenced the oral storytelling abilities of young children. They considered three distinct levels of AI-powered narrative support:
Passive-Active Retelling: Children listened to a standard story narrated by the AI and then retold it. - Autonomous Storytelling with AI Feedback: Children narrated a story picture-by-picture. After each picture, the AI provided real-time, personalised feedback. Children then proceeded immediately to the next picture without revising their work
- Iterative Storytelling with AI Feedback: This followed the same process as the Autonomous condition, but with a critical addition: after receiving AI feedback, children were prompted to revise their narrative for that specific picture before moving on.
The Iterative condition was significantly more effective than both the Passive-Active and Autonomous conditions. Children in this group found the activity more enjoyable and produced narratives with better structure and higher complexity, and more frequent use of internal state terms during the supported phase.
However, when children were asked to narrate a new story without AI assistance, the researchers found the pupils were not able to translate these superior skills into the new task. As soon as the "training wheels" were removed, the pupils’ independent narrative skills hadn't improved. They had simply learned to follow instructions.
Here is an example interaction from the report:
Child's initial narration: “One morning, the mother bird gave birth to two babies, and then the babies chirped. The mother knew they were hungry.”
AI feedback: “… I think you can also add some details, for example, the baby birds cried, or begged for food …”
Child's revision: “A morning, a mother bird gave birth to two little babies. The babies were very, very hungry, chirping loudly, almost about to cry. The mother bird now knew they were hungry.”
Classroom Impact
On the plus side, this shows that AI is doing something that previous digital technology could not do: identify the next step for that child, then respond in natural language. It also highlights the following:
- Importance of practice - what made the AI model successful was allowing children the time to practice from the feedback
- Importance of withdrawing support - The AI provided some great models that were useful to the child. However, these become a crutch. The pupil relies on these to develop their answer. We must skillfully withdraw models and make children do the ‘hard thinking’ required to improve their schemas and metacognition.
- Importance of questioning - if we know a pupil already has the required knowledge to improve the story, we would ask questions such as “how do you think the birds being hungry would make them feel?”. By using questioning to connect schemata, the pupils would start to consider feelings in other contexts.
When you choose your new AI tool, take the time to ensure that it doesn’t allow pupils to cognitively offload their thinking, nor remove the questioning strategies that help develop their metacognition. The process of learning still requires the child to struggle. Without the gradual fading of scaffolding, the AI remains a crutch rather than a staircase to success.
Reliability of the study
One interesting aspect of the study is that the researchers didn’t have a fourth intervention - human intervention. We don’t know how far off the iterative storytelling AI was from being able to replicate human-level success. I suspect that a human teacher would significantly outperform the AI in this case.
The study used a sample of 180 children, which was justified by a priori power analysis to ensure the results were statistically meaningful. Children were randomly assigned to conditions, and baseline tests confirmed no initial differences in storytelling ability between the groups. The researchers also used MANCOVA to statistically control for age, baseline performance, and the duration of interaction. Interrater reliability was high (Cohen's kappa .89–.93), meaning two independent researchers agreed strongly on the scores when assessing the pupil’s narrative work.
Sources
- Sibley, L., Fabian, A., Plicht, C., Pagano, L., Ehrhardt, N., Wellert, L., Bohl, T. and Lachner, A. (2025) ‘Adaptive teaching with technology enhances lasting learning’, Learning and Instruction, 99, 102141. Available at: https://doi.org/10.1016/j.learninstruc.2025.102141
- Zhang, N., Xu, J., Wei, R. and Wang, Y. (2026) 'From scaffolding to transfer: The impact of AI-powered support on children's narrative skills', Learning and Instruction, 104, 102345. Available at: https://doi.org/10.1016/j.learninstruc.2026.102345
