How do we know precisely what students in our classes have learned? And what they haven’t?
Traditional assessments don’t tell us quite enough to create a meaningful feedback loop between students and instructors. But there may be a better way.
Jon Reifschneider, Director of Masters Studies, AI for Product Innovation at Duke University, joined the Enrollment Growth University podcast to discuss how artificial intelligence can help empower faculty to provide more personalized instruction.
Why test scores aren’t enough & what we need instead
Do students understand what we’re teaching them?
A homework assignment, quiz, or test can help determine the answer. Still, these tools are generally designed to provide aggregate scores and nothing more.
Take homework, for instance. Let’s say a student got an 85. Then on quiz one, they earned a 92. That information doesn’t give you insight into what a student understands or where they’re struggling. To figure that out, you’d have to dive much deeper.
“As I started teaching,” Jon told us, “I really struggled with this. But I also realized that now more than ever, we have such a vast amount of data about our students through online homework.”
Everything these days is online — lectures, quizzes, tests, and more. How could we use all this data to gain more insight into what the class as a whole understands and where individual students may be struggling?
The Intelligent Classroom Assistant
“So I started building a tool,” Jon said. “We call it the Intelligent Classroom Assistant.”
This tool looks at data such as quizzes, homework assignments, lectures, and tests and maps it back to a set of pre-defined learning objectives. By doing that, the instructor can evaluate how students are doing on each of the topics that they’re supposed to be learning.
“I am highly adaptive in my teaching style,” Jon explained. “Each week I start preparing for the following week. I spend time in my tool, the Intelligent Classroom Assistant, and I look at the aggregate picture as well as pictures of the individual students.”
That approach helps identify two things:
- Topics with which students are struggling
- Individual subtopics where students aren’t thriving
With this knowledge, the instructor can prepare for the following week by making sure the class works through areas of weakness and reinforcing old ideas in new ways.
Over time, instructors ask: Are students progressing in their knowledge? Or do I need to continue to make additional adjustments and spend more time on these ideas?
Can AI help us in one-on-one student support?
When students show up for one-on-one help sessions, it’s hard to know where they are struggling. You have to ask them and hope they are self-aware enough to answer you correctly.
“I started using the tool also on an individual student basis to look at what I call a mastery map for each student,” Jon said.
That’s basically a map of all the topics covered in the class and which ones learners have reached the level of mastery on and which ones they have knowledge gaps in. By preparing ahead of time, instructors can prepare for the areas where students show weakness and provide a personalized approach to the time together.
“One of the things I’m prototyping right now is a student-facing aspect of the tool,” Jon said.
The tool now lives on a public URL, intelligentclassroom.org, and students can log in and view their own data. This lets them map how they’re doing to date on each topic covered in class. Students can also get personalized help recommendations thanks to a machine learning algorithm that analyzes all Jon’s lectures.
“One of the great things about teaching online is that we’re able to auto-transcribe all of our lectures,” Jon told us. “I can use my algorithm to direct the student to exactly which lecture and specifically where in the lecture they should go for help on whatever particular topic they have a weakness.”
Next steps to leveraging AI-empowered instruction
The first places to start don’t have anything to do with AI.
In fact, the first prerequisite is good teaching practice, specifically formatting assessment in the classroom to create a feedback loop between learners or instructors. Without that feedback loop, it’s impossible to know what’s in a student’s brain. And until we figure that out, there’s really nothing we can do with any type of analytics. The second prerequisite is making sure you’re capturing all that data so it’s available for you to use.
Once you’ve done those two things, you can start to apply machine learning or other types of analytics to extract value from the data. That gives you options. You can build something yourself, or you can use commercial software. A lot of new systems are coming on the market.
“But the immediate focus is on those two things I mentioned as prerequisites before you even start thinking about what the right tool is,” Jon said.
If you don’t use iTunes, you can listen to every episode here.