Emerging Pedagogies Symposium speaker discusses impacts, limitations of AI in the classroom

Duke Learning Innovation hosted the “Emerging Pedagogies Symposium: Teaching and Learning in the Age of AI” event last Monday, featuring keynote speaker Ashok Goel, professor of computer science and human-centered computing at the Georgia Institute of Technology. 

In his keynote address, Goel spoke about the impact and limitations that artificial intelligence might have on teaching, learning and educational research. 

AI’s benefits and limitations

Goel began by pointing to several ChatGPT features, such as how the program can tackle the “bread and butter of computer science” by writing and analyzing code. He also highlighted ChatGPT’s success in standardized testing, including several Advanced Placement exams, the SAT, the Graduate Record Examinations and the Uniform Bar Examination. 

Although he recognized the benefits of using AI, Goel mentioned how people must be aware of its limitations. For instance, Goel noted that ChatGPT has difficulty processing “common sense reasoning” and can only answer prompts because a human has done so before. 

“It’s just completely replicating what it has gotten from [somewhere else]. There’s no deep understanding,” Goel said.

Goel then spoke about how artificial intelligence can enhance education. He highlighted educational psychologist Benjamin Bloom’s findings on how one-on-one tutoring increased students' test scores by over two standard deviations, regardless of that student’s previous performance. 

Goel believes that one-on-one tutoring for all students is not scalable, considering the limited resources available in the education system. However, he hopes AI could use Bloom’s findings by analyzing and responding to individual students to replicate tutoring.

Another measure is to use ChatGPT “with only reliable documents, so there is no possibility of bias.”

The creation of Jill Watson

Goel spoke about his own experiences using artificial intelligence to enhance the classroom experience, even before the release of ChatGPT. He described how he created an online class that hosted and answered questions from thousands of students. 

“The students were frustrated because the questions were not getting addressed; the teachers were frustrated because they didn’t want to spend time [answering] mundane questions,” Goel said.

His solution was to use artificial intelligence to create an automated question-answering agent named Jill Watson, which could answer questions about the class or the content. Goel claimed that by using Jill Watson, teachers could save hundreds of hours per semester. 

Initially, Jill Watson took 100 hours to build when it was first created in 2016. Eventually, that build time was reduced to five hours per course — and was reduced even more. 

“If you give me a class … I’ll ask you for all of your educational materials, the content that you have, the course syllabus, the books that you use, the documents that you pass and I’ll train a Jill Watson for you and build a Jill Watson for you in about five minutes,” Goel said.

Goel concluded his talk with a call for policymakers to determine how to regulate artificial intelligence properly.

“None of us really understand fully the impact that AI will have on teaching and learning in education. We need to identify what our values [and] what our goals are,” Goel said.

---