Great Conversation, No Idea What the Student Learned

We spend a lot of time evaluating our AI tutor. Did it ask a good follow-up? Did it scaffold instead of giving away the answer? Did it respond to the learner’s misconception? Was the conversation engaging? Those are reasonable questions, and getting them right is important. But they tell us much more about the quality of the tutoring than they do about what happened to the learner’s understanding.

A learner can have what looks like a great conversation with an AI. They answer several questions. The AI responds appropriately. There’s productive back-and-forth. Maybe the learner even says the explanation helped. We still don’t know whether their understanding changed.

Several recent pieces of research on AI tutoring expose different parts of this problem.

A recent evaluation from Comprendo ran 779 simulated tutoring conversations using an eighth-grade math learner with a known misconception. Without a tutoring-specific prompt, the models gave away the answer in 97% of conversations. Across 298 attempts, there were zero successful tutoring outcomes by the researchers’ definition.

I’ve watched versions of this happen while designing AI learning experiences. A learner gives an incomplete or incorrect response, the AI correctly recognizes what’s wrong, and then immediately explains it. The explanation can be excellent. It can also remove the exact thinking we need the learner to do.

A 2025 Harvard study published in Scientific Reports shows what can happen when intentional learning design is put around a model. Researchers compared an AI tutor with an active-learning physics class and found that students using the AI tutor had substantially higher learning gains while spending less time on the task. I’m generally wary of big claims about an “AI tutor” outperforming a classroom, but the design of this study is what makes the result interesting to me.

The experience was designed around principles including active learning, scaffolding, cognitive load, targeted feedback, and self-pacing. The researchers also found places where prompting alone wasn’t enough. For multipart problems, they added structure that controlled how learners progressed through the work. An LLM can produce a good explanation, but teaching requires decisions about what the learner should do before receiving that explanation, what information to withhold, and how their response should change what happens next.

Even with that kind of design, there is another problem to solve: determining what the learner actually demonstrated during the interaction.

Think about a learner who believes that multiplying two negative numbers produces a negative number (I miss my 8th grade math teaching days). The AI asks them to reason through an example, and their first response exposes the misconception. Instead of explaining the rule, the AI gives them a smaller case to consider. The learner revises their reasoning. The AI then gives them a new problem without the previous support, and the learner correctly explains why the product is positive.

The sequence tells us something a final correct answer alone cannot: the learner began with a specific misconception, changed their reasoning in response to a scaffold, and then applied the idea correctly when the support was removed.

Conversation length, number of turns, or a learner reporting that the interaction was helpful wouldn't capture any of that.

Conversational learning gives us access to pieces of the learner’s reasoning that traditional signals like right/wrong answers and time on task largely miss. That evidence is harder to interpret. A correct answer after the AI supplied most of the reasoning carries less weight than one produced independently. Revising an answer after a carefully chosen scaffold tells us something about how the learner’s thinking changed. A long, engaged conversation can still end with the learner holding exactly the same misconception they had at the beginning.

The conversation itself is evidence, but interpreting it requires us to be precise. We need to distinguish what the learner could do independently from what they could do with support and pay attention to how their reasoning changed over the course of the interaction.

Reducing all of that to a single “understanding score” would throw away much of what makes conversational data valuable. The better goal is to make careful, bounded claims about what a learner has actually demonstrated and how strong the evidence for those claims is.

Persistent learner memory makes bad interpretations harder to contain. If we carry an inaccurate assumption about a learner from one interaction into the next, future instruction can start adapting around something that was never well supported in the first place. An AI can remember that a learner struggled with a concept last Tuesday. It can store the entire conversation. Neither tells it what should happen on Wednesday. For memory to improve learning, what gets carried forward needs to be more useful than a transcript: a grounded account of the learner’s current understanding, with enough information to know which conclusions are well supported and which still need evidence. Otherwise, we are giving the AI a longer history without giving it a better basis for deciding what to do with that history.