What Exactly Are We Trying to Measure?

People ask me now, working in AI, whether technology can prove learning. Funny thing is, I don't remember feeling that confident about proving it when I was actually in the classroom.

I gave assessments constantly. Grades needed numbers, parents and learners wanted updates, and I needed some signal on whether to keep moving or slow down. But what stuck with me was never the grades. I remember sitting after class with a stack of quizzes, staring at one learner's paper because it contradicted what I'd seen an hour earlier at the whiteboard, where they'd walked me through the idea almost perfectly. On paper, they looked like they'd understood nothing.

Which one was I supposed to believe?

That happened often enough that I stopped treating assessments as proof. They were evidence, sometimes strong evidence, sometimes evidence that sent me back to rework a lesson because half the class had made the same mistake for the same reason.

I don't think assessment is broken. It's still one of the better tools educators have. But I wonder if we've quietly started asking it to answer a question it was never built to answer.

Now people ask whether a conversation with AI can prove learning. Reasonable question. I'm just not sure it's a new one. Educators have always been making inferences from imperfect evidence. A quiz score is evidence. A project is evidence. A learner explaining an idea out loud is evidence. None of it ever gave me certainty, and I don't remember expecting it to.

So when someone asks if AI can prove learning, I get stuck on a different question: prove what, exactly? Two universities can both say they want "evidence of learning" and mean completely different things. Retention a month later? Transfer to a new context? The ability to explain it to someone else? Recognizing a misconception you used to hold?

I don't know the answer, and I think that's the actual conversation worth having. AI didn't create this uncertainty. It just made it harder to look away from.

So, when you say a learner has learned something, what evidence convinces you? And what evidence doesn't?