By Dr. Jessica Smagler, EdD
Head of Research and Outcomes
Kyron Learning
What happens when middle school students—especially English Language Learners—don’t feel confident participating in math class?
In 2024, Kyron Learning set out to explore whether AI-powered instruction could meaningfully increase motivation and engagement in those classrooms. With support from a Gates Foundation grant and the greater AIMS community, we launched a multi-cycle research study to test, refine, and measure what actually works.
The AIMS (Advancing Innovative Mathematics Solutions) Collaboratory
The AIMS Collaboratory is a community that brings together researchers, technology developers, and school districts to build, test, and refine math products and strategies. It is a rare and energizing space where dozens of organizations work collectively toward a shared goal: improving outcomes for students and teachers, one math classroom at a time.
Being part of this community has shaped Kyron in meaningful ways beyond the grant itself, influencing how we approach research, collaboration, and product development.
Kyron Learning: AI-powered Instruction
Kyron Learning delivers AI-powered Instruction designed to augment great teaching. Educators create interactive, conversational lessons that feel like personalized 1:1 instruction while staying fully aligned to their classroom goals.
Teachers set the learning objectives, define the audience, and optionally include course materials. Kyron then generates a structured, multimodal lesson grounded in learning science. Every element can be edited, refined, or replaced, including visuals and instructional prompts, ensuring educators remain in control of both content and pedagogy.
As students move through a lesson, Kyron provides real-time support. Instead of simply giving answers, it guides learners through dialogue that surfaces misconceptions, reinforces concepts, and builds confidence. The result is individualized support available when students need it most, designed to motivate and engage learners while staying aligned to educator intent.
Study Design
For the 2024-2025 school year, Kyron Learning partnered with 22 teachers and almost 1,000 students from 17 schools throughout the United States to explore motivation and engagement of English Language Learners in math class (see Figure 1). The students in the study were in grades five through eight and came from schools with large percentages of historically disadvantaged youth (see Figure 2)—an important component of the missions both for Kyron Learning and for AIMS. Each teacher taught two comparable sections—similar in size, grade level, and academic profile—of about 25 students with one section using Kyron (treatment) and the other not using Kyron (control).
With invaluable help from Leanlab Education, we designed a four-cycle rapid iteration study. Data collection for each cycle lasted eight to ten weeks for teachers and six weeks for students. During each cycle, teacher participants built lessons on Kyron’s platform and used those lessons with their treatment class for 30 minutes per week for six weeks.
For each of the four cycles, we collected pre- and post-survey data from both learners and teachers consisting of items designed to measure motivation and engagement, gathered product usability feedback, conducted teacher interviews, and captured qualitative insights throughout the process.
Research Questions
The key questions we sought to answer with this research were:
Key Findings
Qualitative
The qualitative findings below capture what teachers and students experienced while using Kyron in their classrooms. Drawing from surveys, interviews, and classroom feedback, these themes highlight how the platform shaped instruction, supported diverse learners, and influenced student confidence and engagement. Together, they offer a grounded look at Kyron’s impact from the perspectives of the people using it day to day.
Teacher Perspectives: Empowering Instruction & Inclusion
Standards-Aligned Lessons That Surface Student Thinking
Teachers consistently reported that Kyron lessons aligned well with standards while helping them better understand student misconceptions and promote independent learning.
"The strong points are how it gradually introduces the student to what the lesson is about. My students really enjoyed the introduction. They also liked the way they could answer questions, and the AI was able to answer their questions and gave instant feedback.”
Transformative Support for ELL Students
Teachers highlighted Kyron’s particular impact on English Language Learners, noting improved access, understanding, and performance.
“I have not truly understood the problems with ELL students until this year. It is difficult to start working with students that have limited language, but we had the best scores ever from them in the past 6 weeks.”
“[The ELLs] benefit from the ability to pause, replay, and review instructions and explanations at their own pace. This helps reduce frustration and allows them to process the language more thoroughly.”
Engagement & Confidence Across All Learners
While statistically significant effects were limited and varied by cycle, educators reported clear qualitative gains in student participation, confidence, and willingness to engage. Teachers described classrooms where more students were contributing, asking questions, and thinking more expansively about the content.
“The lessons have engaged every student regardless of their deficiency. They are excited about getting into the lessons, and they are responding more and with greater confidence.”
“Before I started using [Kyron lessons], I would ask students how well they understood what I was teaching, and many of them were scared to admit in front of peers that they had no clue how to do some things. I liked the fact that they used the computer AI to ask questions.”
“Students are engaged and looking to think wider. They are feeling like their voices matter, and they should feel that way.”
Easy to Use With Powerful Potential
Teachers generally found Kyron Studio intuitive and easy to use, and emphasized that ongoing product updates strengthened its classroom value. With continued improvements—such as increased student interactivity, enhanced visuals, and ready-to-use resources—teachers saw Kyron as having strong potential to become a go-to tool for inclusive, efficient, and data-informed teaching in the K-12 classroom.
Student Perspectives: Confidence, Clarity, and Control
Building Understanding, Confidence, and Independence
Many students reported that Kyron helped them better understand math, feel more confident, and learn more independently.
“Since using Kyron, I feel more confident in math. It helps me understand problems better and makes learning more fun and less stressful.”
“I like how these lessons adapt to your knowledge. It allows students to start from their own understanding, and be able to build more of that.”
Personalized Support That Meets Students Where They Are
Students appreciated adaptive questioning and the ability to ask for clarification when they were confused.
“What I liked about Kyron Lessons was that the questions for each lesson were different, and what helped me when learning was when I could ask them to simplify the question if I didn't understand what they were saying.”
Strong Resonance With Spanish-Speaking Students
Spanish-speaking students shared especially positive feedback, emphasizing clarity of explanations and learning support.
“Lo que me gusto de Kyron, fue que me explicaba bien y me ayudaron a respaldar mi aprendizaje.” (What I liked about Kyron was that it explained things well and helped me support my learning.)
A Judgment-Free Learning Partner
Students described the AI Assistant as supportive and non-judgmental, reducing stress and encouraging help-seeking.
“I liked talking to the AI Assistant because it was always available to help and explained things in a simple, clear way. It made learning feel more comfortable and less stressful since I could ask questions without feeling judged.”
Across interviews, surveys, and classroom observations, a consistent theme emerged: teachers and students experienced Kyron as a tool that expanded access, strengthened confidence, and made participation feel safer and more attainable. While the magnitude of impact varied by classroom and learner group, the qualitative evidence pointed toward meaningful shifts in engagement, particularly for English Language Learners. The quantitative data below provides additional context for how those shifts appeared in measured outcomes.
Quantitative
Across the four rapid iteration cycles, quantitative findings revealed modest and sometimes fluctuating differences between treatment and control groups. While the pattern was not linear, student self-reported outcomes more often trended in favor of Kyron, with clearer directional movement emerging in later iterations. In Cycle 4, student math self-perceptions reached statistical significance in favor of Kyron.
Math anxiety showed more variability across cycles. In Cycle 2, students in Kyron classrooms reported a statistically significant reduction in math anxiety compared to their peers in control classrooms. While this reduction was not sustained at the same magnitude in subsequent cycles, anxiety levels remained relatively comparable between groups in later iterations.
Teacher-reported measures of classroom engagement demonstrated clearer and more consistent differentiation. Across the school year, teachers rated students in Kyron classrooms as exhibiting higher levels of emotional engagement and, in some cases, lower emotional disaffection. Emotional engagement reached statistical significance in Cycle 2 and trended higher in subsequent cycles. Teacher observations of participation, attentiveness, and interest also showed small but steady advantages for Kyron classrooms over time.
For English Language Learners specifically, teacher-reported engagement followed a similar pattern. While differences were not statistically significant, Kyron classrooms showed gradual strengthening in participation and attentiveness across the year, aligning with the study’s focus on inclusive instructional design.
Taken together, the quantitative findings point to incremental but meaningful shifts—particularly in teacher-observed emotional engagement and in specific reductions in math anxiety. While student self-reported measures showed modest differences, the patterns observed by teachers align closely with the qualitative accounts of increased participation, confidence, and willingness to engage. In combination, these findings suggest that the most pronounced impact of Kyron was experienced in the lived dynamics of the classroom.
Product Iteration
The most valuable part of the study was the open and candid feedback we received from both teachers and students about the product. As a result of the feedback, we made three critical improvements to our product to support English Language Acquisition:
Figure 3 below shows that ELLs disproportionately took advantage of many of these features throughout cycle 4 as compared to their peers.
Progress Since June 2025
Just as important as the outcomes themselves was how this study shaped our product roadmap. Teacher and student feedback led us to deepen the back-and-forth dialogue within lessons, strengthen visual scaffolds for complex concepts, and refine how the AI adapts to student confidence and uncertainty in real time. We also enhanced our analytics to surface misconceptions more clearly for teachers and learners.
This study did not simply measure impact. It directly informed how we continue building more responsive, inclusive AI-powered instruction.
Reflections and Next Steps
This study began with a focused question: how can AI-powered instruction meaningfully support motivation and engagement for middle school students—particularly English Language Learners—in mathematics? While shifts in motivation were difficult to capture consistently through quantitative measures, the qualitative findings were clear and compelling. Teachers saw new visibility into student thinking. Students described greater confidence, clarity, and independence. English Language Learners, in particular, experienced more accessible entry points into mathematical understanding.
Just as importantly, this study reinforced something core to Kyron’s approach: AI-powered instruction must be educator guided and shaped in real classrooms. The most meaningful advances did not come from assumptions about what students needed. They came from intentional cycles of classroom feedback, rapid iteration, student experiences, and partnership with teachers.
We do not view this research as a verdict on AI in education. We see it as progress. As proof that building AI for learning requires discipline, evidence, and educator oversight at every step.
For Kyron, the work continues. We are focused on amplifying teacher expertise and designing instructional experiences where students can think out loud, receive guided support, and build confidence in real time. Through our continued partnership in the AIMS Collaboratory, we remain committed to building, testing, and refining alongside educators and researchers who share this mission.
AI in the classroom should not replace teaching. It should augment it. And it should always be rooted in learning science, shaped by educators, and grounded in the lived experiences of students.