Background

Lumi is an AI-powered EdTech platform that uses comic book storytelling to improve literacy rates and student engagement in K-12 schools. By combining visual narratives with interactive reading experiences, Lumi helps students develop reading comprehension and writing skills through a medium that naturally captures their attention and imagination.

Teachers create projects on Lumi for their students; customizable learning outcomes align to their curriculum.

Students follow a step-by-step process for creating their stories.

Strategic research

The Head of Customer Success and I received feedback from teachers across 5 districts, representing hundreds of students, during biweekly calls about how students were actually using the AI-driven Lumi Chat feature. What we discovered fundamentally challenged underlying product assumptions.

Teachers were raising serious concerns about students using Lumi Chat suggestions to create documents rather than developing critical thinking skills. Districts evaluating our tool based on literacy outcomes were threatening to discontinue programs they saw as enabling academic dependency rather than supporting learning growth.

Posthog sessions confirmed what we heard from teachers - students were using the suggestion feature to complete entire documents.

Students were relying on the Lumi suggestion feature to write entire documents for them.

What teachers told us

Through interviews with educators across different districts, a clear pattern emerged:

"Kids are using Lumi suggestions to write the entire document for them."

"The hardest part about teaching is that I am spread too thin to coach all of my students, so lots of them don't get the attention they need."

Teachers revealed they weren't opposed to AI assistance - they desperately wanted tools that could provide the individual coaching they couldn't offer every student. But our current approach was replacing student thinking rather than supporting it.

What districts told us

School administrators were evaluating Lumi through literacy outcome improvements, and our AI writing assistance was actively working against those metrics:

"If you can't prove that kids are getting better at reading and writing, we won't continue this program."

This was our wake-up call: Lumi's original AI direction positioned it as a productivity tool that allowed students to generate entire documents with minimal effort. While this delivered on efficiency, it fundamentally undermined the educational outcomes our customers were measuring.

What students actually need

The most revealing insight came from teachers who understood their students' capabilities:

"These kids know a lot. Sometimes, they just need help with stepping stones to get to the answer."

Students weren't lacking ideas - they just needed scaffolding to access and organize what they already knew. Our AI was doing too much of the work for them.

Lumi was fundamentally misaligned with educational pedagogy

Students bypassed real learning via Lumi’s writing suggestions
Teachers were frustrated by students bypassing the cognitive work necessary for skill development
Districts were expecting improved literacy outcomes
Lumi was seen as an obstacle to education rather than support

Meanwhile at MIT Media Lab

Research revealed that dependence on AI-generated content negatively impacted cognitive development.

❌ Learning transfer

LLM users had worse performance across neural, linguistic, and scoring measures over time.

❌ Cognitive dependency

When asked to work without AI support, they exhibited weaker neural connectivity and under-engagement.

❌ Ownership and retention

LLM users reported low ownership of their essays and struggled to quote from work they had just completed.

❌ Cognitive load patterns

ChatGPT users showed weakest overall cognitive coupling compared to brain-only writing.

Implications for educational AI design

While AI can provide immediate benefits, unrestricted use may undermine the cognitive processes fundamental to learning and skill development.

What all of this meant for Lumi

Our original Lumi Chat experience positioned us as a productivity tool that allowed students to generate entire documents with minimal effort. While this delivered on efficiency, it fundamentally undermined the educational outcomes our customers were measuring and posed an existential threat.

Translating findings into product opportunities

This research illuminated a clear path to market leadership and a defensible business model: position Lumi as a teaching partner rather than a content creator.

✅ Cognitive load monitoring

Lumi Chat should have mechanisms to ensure students remain cognitively engaged rather than becoming frustrated, passive, or disengaged.

✅ Learning process integration

Lumi Chat should support the learning process, not replace it. Lumi Chat should provide stepping stones for idea development, critical analysis, and knowledge synthesis rather than content generation.

✅ Personalized support system

Rather than writing for students, Lumi Chat should offer scaffolded support that acts as an always-available teacher. This maintains cognitive challenges while offering personalized help.

There was an additional opportunity for assessment adaptation that we addressed in parallel work through insights & analytics for teachers. Traditional assessment methods may need revision when AI tools are involved, focusing more on process, understanding, and transfer rather than final products.

Testing opportunities with rapid prototyping

The Head of Customer Success and I wanted to quickly test the 3 product opportunities. We gamed out a realistic conversation that Lumi Chat might have with a 9th grader working on an analysis of “Animal Farm.”

Lumi’s responses should…

Support the student like an always-available teacher
Prevent cognitive overload
Encourage individual idea development

Building team alignment

The Head of Customer Success and I presented our research and prototype to the CEO, Head of Engineering, and the Engineering Team to build strategic alignment.

This gave everyone at the company a voice and stake in the strategic problem at hand: they could voice concerns, share feedback, and offer alternative solutions.

They shared tactical feedback for the prototype, which influenced design and engineering requirements.

“What if this is for a science class, and the teacher doesn’t want the student spending time on character analysis? How do we make sure Lumi talks about the right things?”

“This feels like a moment where a student will just go to ChatGPT. How can we keep them motivated in our product?”

“This feels like an area where AI can assist with busy work. How can we still leverage the time-saving potential of AI?

Additional feedback

ML needed a framework for what Lumi Chat does and what Lumi Chat does not do
ML needed examples of Lumi responses

Strategic skepticism

Ultimately, the team was skeptical of making such a seismic shift to our product experience, and ultimately our company strategy.

Addressing skepticism

Customer Success and I believed we could alleviate hesitation through customer feedback and building team-wide ownership.

We ran two parallel tracks of work:

Validate the strategy with customers to address skepticism
Collaborate closely with engineering on the user experience

Early customer validation

Without a research team, Customer Success and I utilized standing biweekly calls with teachers to gather feedback about our approach.

We shared the prototype from earlier and heard the following:

“This is way better than having Lumi write for students. I’m still worried about them copying and pasting from GPT, but at least this way Lumi isn’t just serving answers to them on a platter.”

“This seems like it would really help lift up my students who get stuck. I don’t have time to work with all of them one-on-one, and I don’t have a teaching assistant this year, so this sort of thing can help fill gaps for me.”

“Would this know what I am trying to teach? I don’t want it to focus too much on sentence structure if I’m using Lumi for a project about genetics in my biology class.”

Customer alignment

We quickly learned that this experience addressed a serious problem for teachers: They simply don’t have the bandwidth to provide personalized learning to all students. If Lumi could behave like an always-available version of themselves, they believe it could materially improve learning outcomes.

At the same time, it also addressed educator and district admin concerns about “cheating” with AI that does all of the work for a student.

Co-creating with engineering

At all companies, but especially a small, 7-person organization, I believe it’s important for team members to feel a sense of ownership and conviction in what they’re building. In order to unlock this, I held design sessions with engineers to address their feedback and customer feedback.

Lumi Chat framework

The ML engineer was excited to set up a framework to test a chat experience with Lumi behaves more like a teacher, and less like a creator. We worked together on a framework for chat patterns, which should mirror education best practices and provide “stepping stones” for students.

2. Training Lumi to “talk about the right things”

During our feedback session, the ML engineer identified that Lumi Chat would need to “talk about the right things” with students. This was also flagged by teachers.

As a principle, we view teachers as the experts on what students should learn, so we wanted to provide a way for teachers to easily add detailed context to projects. This also informs how Lumi “talks” to students in order to mirror the actual teacher.

We co-designed a step-by-step workflow for teachers to add overall project goals, guidelines for how Lumi responds to students, and goals for individual project steps.

3. Student engagement

One of the full-stack engineers, who has a passion for front-end and consumer experiences, was excited about cracking the problem of student engagement. He and I paired with Customer Success to address this, and we identified 3 key issues: Students become disengaged when they don’t know where to begin, when they get stuck and don’t know how to get unstuck, and when they don’t receive positive reinforcement.

Below are explorations we designed to address these issues.

a. Engaging students who don’t know where to begin

Hypothesis: Inactivity on a new document can indicate a student doesn’t know where to start.

What if Lumi intervened after 30 seconds of inactivity?

b. Helping students get unstuck

Hypothesis: “Stepping stone” questions are used by students to guide students to their own conclusions.

How might Lumi mirror this behavior?

c. Positive reinforcement

Hypothesis: Gamification can be a method for positive reinforcement.

What if we had a system of badges

4. Cognitive development vs. busy work

The Head of Engineering identified an opportunity for Lumi to still generate for students when it does not negatively impact cognitive development. We partnered with Customer Success to create frameworks for tasks that are tied to cognitive development and tasks that are not tied to cognitive development.

5. Insight into student processes

The Lumi team and our customers flagged a loophole: students can still go to programs like ChatGPT in order to complete assignments for them. In the world of generative AI, process becomes as important as the final product.

How might we give teachers insight into how students are creating on Lumi, as well as actionable next steps to support real learning?

This was addressed in a separate analytics project, but just for fun, here’s a glimpse of the MVP experience we launched:

Pitching the vision

With external validation, as well as internal and external feedback, Customer Success and I brought a blue-sky proposal back to the team.

The student experience demonstrates how Lumi mirrors teaching methodology, supports students with level-adjusted stepping stone questions, and keeps students engaged and encouraged with a badge system.

The teacher experience demonstrates how teachers can train Lumi on their curriculum, project goals, and personal teaching methodology to provide a personalized learning experience to all of their students.

Can we de-risk this work?

There was still significant resistance from founding team members to undergoing a total product pivot. Even with early positive signal from customers, what if we got it wrong? It was time to figure out what lower-risk, lighter-lift experiments we could run to de-risk this proposal.

Impact-effort analysis

The CEO, Head of Engineering, Customer Success, and I held a workshop to chunk the proposal into independent pieces and map them against impact and effort.

An experiment everyone believes in

Our matrix made the decision obvious: we could test our hypothesis that Lumi would be more effective as a teacher rather than as a co-creator by adjusting the chat experience.

We could quickly adjust our prompts for Lumi so that it guides students to their own answers instead of providing the answers for them. We would also shut off the Lumi suggestion experience during this experiment.

This change only impacts the chat experience, and leaves other parts of our product untouched.

This simple, easy-to-implement experiment ultimately got team-wide buy-in and alignment because it allowed us to validate the hypothesis without upending the entire product.

Measuring success

Ultimately, we want to know if students see an improvement in cognitive development when using Lumi.

We are tracking longitudinal data with districts related to state testing and reading levels, but it can take years to see impact here.

As a proxy for this, Customer Success is qualitatively measuring feedback from teachers using Lumi, as well as feedback during demos to potential customers. We are still running this experiment, but I’m happy to report that initial reactions are positive going into the 2025 school year.

Impact

This research and product vision is now informing a comprehensive business strategy evolution that aligns our core value proposition with how customers measure success and make purchasing decisions.

It also strengthened our relationships with educator customers who see Lumi as thoughtfully considering long-term learning outcomes.

Lastly, this work established an internal framework for responsible AI in education that continues to guide product discussions. The teacher interviews and district feedback provided crucial insights that helped the team better understand the tension between AI efficiency and educational effectiveness - learnings that inform ongoing product strategy.

Lumi AI learning companion