Bringing AI into the work of learning

This fall, UC Berkeley industrial engineering and operations research professor Huiwen Jia will ask students in an undergraduate engineering course to step into the role of operations managers, using a newly developed resource allocation simulator to make and refine decisions over 12 simulated months.

Using an AI assistant built into the web-based dashboard, students can turn their ideas into code and test those policies in a simulated cloud service operation, where changing customer demand, limited server capacity and uncertain service times force them to make difficult trade-offs.

Getting AI to produce working code is only the beginning. After each simulated month, students must evaluate how their policy performed, compare it with established approaches and decide what to change before the next round. The process makes AI part of a continuous cycle of decision-making, testing and revision rather than treating an AI-generated response as the answer.

Jia and four UC Berkeley Master of Analytics students developed the dashboard to address a challenge confronting educators as generative AI becomes increasingly common in higher education: how to give students meaningful experience with the technology without allowing it to replace the critical thinking, subject-matter knowledge and judgment they need to develop.

Jia will introduce the simulator this fall in INDENG 150, Production Systems Analysis. Her work is part of a Berkeley Engineering-led initiative, with support from the Center for Advancing Women in Technology, that funds projects exploring pedagogically grounded uses of AI across engineering disciplines.

Jia focuses on modeling courses, where students translate complex problems into mathematical models and use them to guide decisions about issues such as allocating resources, managing inventory and routing customers through a system.

“In class, we often introduce models in a relatively simple environment,” Jia said. “But real-world applications are more complicated. With the simulator, students can see how the models they learn in class perform when some of those assumptions no longer hold.”

Stepping into the role of an operations manager

Students take on the role of an operations lead for a simulated cloud service platform. Customer requests arrive over time, and students must decide whether to reject each request or assign it to one of 10 server clusters with limited capacity. Requests vary in value and resource requirements, and students do not know in advance how long each request will occupy capacity.

That uncertainty makes the problem more difficult than simply accepting every request that fits. A lower-value request with a long service time, for example, could tie up capacity that might otherwise be used for a more valuable request arriving later.

Students begin by examining a year of historical operating data for patterns in demand, service duration and revenue. Using those observations, they develop an admission-and-routing policy, essentially a set of rules for deciding which requests to accept and where to send them.

The simulation progresses one month at a time. After each round, students examine revenue, accepted and rejected requests, unfinished work and server utilization and compare their results with benchmark policies discussed in class. They must also decide what successful performance looks like. High server utilization, for example, may produce poor results if lower-value work fills the system or requests remain unfinished at the end of the month.

“In reality, we have to design our own metric,” Jia said.

Giving AI a supporting role

The dashboard integrates the AI assistant directly into the policy-development process.

A student might decide, for example, that the system should preserve more capacity for high-value VIP requests. Instead of writing the Python implementation entirely line by line, the student can describe that strategy to the AI assistant, ask for feedback and use it to generate code that reflects the idea.

But Jia does not want students to simply ask AI for the policy that will maximize revenue. The dashboard provides sample prompts that encourage students to begin with their own operational reasoning. AI can help explain concepts, brainstorm strategies, generate or debug code and interpret results, but students remain responsible for understanding the code and determining whether the resulting policy works.

Once students implement a policy, they examine the results, compare their performance with established approaches and decide what to change. The dashboard preserves results from previous months, allowing students to revise their strategies as conditions change.

For Jia, that process reflects a skill students will increasingly need beyond the classroom: the ability to work with AI while retaining the expertise to recognize when its output is incomplete, ineffective or wrong.

“If students directly use AI without knowing whether the answer is correct, it can be harmful,” Jia said. “They need the ability to evaluate and improve those solutions.”

Students building for students

Jia developed an initial version of the simulator before the summer, but four UC Berkeley Master of Analytics students helped transform the prototype into the web-based platform that students will use this fall. The student development team included Yiran Guo, Wen Chia Huang, Munazza Nadir and Junxing Wu.

The group met with Jia weekly as they moved the simulator from a version that students would have had to download and install on their own computers to an online service accessible through course credentials. Their work required them to consider not only whether the platform functioned correctly, but also how undergraduates would experience the problem-solving process.

“One of the most interesting challenges was turning a proof of concept into a platform where students could clearly understand the entire problem-solving process,” Huang said.

Knowing that undergraduates would eventually use the platform also changed how Huang approached its design.

“Knowing that undergraduate students would actually use the dashboard made me think less like a developer and more like someone encountering the optimization problem for the first time,” Huang said. “One question I kept coming back to was not just how students should use AI, but when AI should enter their problem-solving process.”

The team designed the workflow so students first examine the resource-allocation problem and historical data before turning to AI to help develop a strategy. That sequence gives students an opportunity to form their own understanding before evaluating AI-generated suggestions.

For Wu, designing for students also meant making the platform’s technical behavior predictable enough that it would not distract from the underlying lesson.

“It made us think much more carefully about whether the system’s behavior would be understandable, not just technically correct,” Wu said. “For example, when testing different resource-allocation scenarios, I focused on making sure that the simulation results and API outputs remained consistent so that students would not be distracted by unexpected system behavior. The goal was to make the technology fade into the background so students could focus on understanding the decisions and tradeoffs in the problem itself.”

What comes next

This fall, Jia will have her first opportunity to observe how undergraduate students use the dashboard.

After the semester, she plans to collect student feedback and examine usage data, including whether patterns emerge between how frequently students interact with the AI assistant and how their policies perform. The results could help guide future versions of the simulator and its possible use in other modeling courses.

For Jia, the larger goal extends beyond a single dashboard or course. As AI becomes increasingly capable of generating code and proposing solutions, students will need the expertise to determine whether those answers make sense and the judgment to improve them when they fall short.

Jia’s approach makes AI part of that intellectual work, not a substitute for it.