“Hallucinations”

By Scott Hamilton

I have been using Microsoft’s Co-pilot for the last several months to help me with generating quizzes for an American history class that I teach at local homeschool co-op, but the last several weeks it has become challenging. I don’t know if you have heard about a phenomenon called AI Hallucinations or not, but I have experienced them the last several weeks. In fact I have perfect examples of it from teaching this history class.

We were studying World War II politics and that’s where the problems began. I took the same pdf file I have been using for more than a year and uploaded it to Co-pilot. It is the answer key for the curriculum we use in the course. It has questions and answers from the text with references to the page number in the book for every unit and lesson.

Up until about a month ago I would upload the file and ask Co-pilot to “extract the questions and answers for Unit 20” for example. It worked perfectly, making an exact copy of the questions from just Unit 20 rather than having to copy, paste and reformat the questions by hand, making my life very easy. The second thing I would do is ask it to just extract the questions to make a handout for the students; this worked beautifully as well. My final request was for Co-pilot to make a quiz from the question set. I loved it as it saved me hours of effort, up until a month ago.

Last month when I followed the same process, Co-pilot began to do something strange. It started making up its own questions that were nowhere in the original document. For example, during the World War II unit it came up with “What was the Cold War?”, “What was the Berlin Airlift?”, and “Who was Martin Luther King Jr.?” Not only were these questions completely off topic, they were nowhere to be found in the original file. Instead of extracting the original questions, Co-pilot decided to do its own thing and ask random history questions.

At least it provided answers, but were they correct? That is where it begins to get even more scary to me – more than half of the questions were not answered correctly, at least from my view of history. If you study history, you learn very quickly that all history books have a slant toward the author’s world view. For example if you read an American Civil War history book written by someone from the north, the war was all about slavery. If you read the same history written by someone from the south, it talks about being taxed without representation. I hold a Christian world view and look at American history through the viewpoint that the early settlers came for Christian religious freedom. Co-pilot does not share the same views, but many scholarly history books, including the curriculum in use share my viewpoint.

I guess I am saying all this as a warning, that while AI can be a great tool to speed up many processes, like extracting specific information from documents, it cannot be trusted. There are several theories as to why AIs hallucinate, and the first is that they are constantly trying to improve the accuracy of their results, based on the worldview of their developer and trainer. I think the reason behind such poor behavior in my particular case is that the worldview of my document and the AI differed greatly, and it was trying not to compromise what it perceived to be the truth. I did not ask for it to correct the document or research the topic, I simply ask it to do a copy and paste of a large section of the document and it failed to comply because it did not agree with the subject matter.

I have the students in the class writing a term paper and some of them have tried to use AI to assist with the paper. It does an amazing job of generating a very nice looking term paper, including references, but in nearly every case I have been able to easily identify the AI-generated papers, partly because of the sources that were utilized and mis-quoted, and partly because of the graduate student level language utilized throughout the paper. So if you are a student and are thinking about using AI to assist in writing a paper, I highly recommend you check all the reference sources and just ask for an outline with references. Any teacher will spot an AI-generated paper, but you might get by with automating the research. The big risk is that of hallucinations – maybe it decides to use a historical fiction work as a source and you get completely false history. Or it scans scholarly journals and misinterprets the information, giving you a different view than you would have seen yourself.

While I believe AI has a few valid use-cases, it cannot be trusted because you cannot trust its sources of information, which primarily come directly from the internet without any reliable filtering. Until next week stay safe and learn something new.

Scott Hamilton is an Expert in Emerging Technologies at ATOS and can be reached with questions and comments via email to shamilton@techshepherd.org or through his website at https://www.techshepherd.org.