UC Berkeley professor flips the script on classroom AI

PupilBot pilot applies learning science to rethink how AI fits into undergraduate education

As artificial intelligence becomes more prevalent in higher education, faculty in the Department of Industrial Engineering and Operations Research are examining how these tools can be integrated into undergraduate courses in ways that reinforce — rather than replace — student learning.

Assistant Teaching Professor Daniel Pirutinsky is piloting a new classroom tool, PupilBot, that asks students to take on an unfamiliar role when interacting with AI: teacher.

The project is part of a Berkeley Engineering–led initiative supported by the Center for Advancing Women in Technology, which awarded funding to faculty across multiple engineering departments to explore pedagogically grounded uses of AI in undergraduate education. Rather than introducing AI as a tutor or problem solver, PupilBot is designed to leverage a well-established learning principle known as the protégé effect, in which students deepen their understanding by teaching others.

“A really, really good way of learning is by teaching,” Pirutinsky explained during an interview about the project’s development. While AI is often positioned as an instructional aid that delivers answers, he said his goal was to create a system that requires students to articulate reasoning, define assumptions and confront gaps in their understanding.

PupilBot operates as a Gemini-based custom agent that is intentionally prompted to behave like a novice learner. The AI asks clarifying questions, expresses confusion about conceptual elements and resists providing direct solutions. In practice, this design pushes students to slow down and explain their thinking step by step, rather than relying on answer-focused study strategies.

The tool was first piloted last semester in IEOR courses, where students were offered an opportunity to engage with PupilBot as part of an exam follow-up activity. Pirutinsky described the initial results as mixed but encouraging: student participation was high, and feedback collected through end-of-course evaluations suggested that many students found the experience helpful for revisiting material they had not fully mastered.

At the same time, the pilot revealed clear limitations. Large language models are optimized to produce correct answers, Pirutinsky noted, which can make it difficult for them to consistently sustain the role of a confused learner. In some cases, early versions of the tool made simplistic errors or reverted to providing solutions. Those behaviors have since informed revisions to how the system is prompted and structured.

Using PupilBot in undergraduate courses required a platform that could be shared reliably and securely with all enrolled students. Pirutinsky initially experimented with Microsoft Copilot but ultimately transitioned to Google Gemini, which allowed custom agents to be shared securely with students through UC Berkeley’s campus-authorized environment. Pirutinksy emphasized that ensuring equal access, data privacy and compliance with FERPA requirements were nonnegotiable constraints to deploying PupilBot.

With grant support, Pirutinsky is continuing to refine PupilBot this semester while recruiting undergraduate research assistants to contribute to tool development and evaluation. A longer-term goal is to design an IRB-approved study that formally assesses whether — and under what conditions — the approach improves student learning outcomes. That study would compare performance and retention between students who use PupilBot and those who do not.

Beyond the tool itself, Pirutinsky views the project as part of a broader effort to help students engage responsibly with AI technologies that are already shaping professional practice. He noted that some students approach AI tools with anxiety or uncertainty and argued that low-stakes, institutionally supported experiences can help build confidence and fluency.

“Pretending that AI isn’t going to have a significant impact on education isn’t realistic,” he said. “The challenge is figuring out how to integrate it in ways that keep students actively thinking, reasoning and learning.”

As PupilBot continues to evolve, Pirutinsky plans to share lessons learned through campus teaching networks and disciplinary communities, with the aim of adapting the approach for other problem-solving courses within and beyond IEOR.

Undergraduate students interested in contributing to the development of PupilBot are encouraged to apply to join the research team. Apply here.