What is all the fuss about ChatGPT?

Unless you have been living under a rock for the last month, you have likely heard of ChatGPT and the concerns that teachers have due to this software. I understand these concerns for teachers who assign writing projects, which I do, but can a student use ChatGPT to write a research paper?

Since we are getting ready to discuss copyright in the Math 371 class, I wondered if ChatGPT was smart enough to not use copyrighted material. I decided to ask an SDState librarian who is the resident expert on copyright, Elizabeth Fox. Another concern I had is whether ChatGPT would produce a bibliography. Fox asked it “I need 5 peer-reviewed articles on artificial intelligence.” ChatGPT produced a list of 5 articles. It gave her a list, but the links to the articles didn’t work and when she searched databases for these articles, the articles did not exist. Therefore, I am not convinced that ChatGPT could write a good research paper with citations and a bibliography. Fox said that she thinks that ChatGPT is likely more of a danger with plagiarism than copyright violation.

But it certainly can write a persuasive essay—which colleges and teachers need to watch for. A computer science major at Princeton recently released an app, called GPTZero, which can determine whether something was written by a human or ChatGPT. This app gives a piece of writing a “perplexity” score and a “burstiness” score. The perplexity score is a measurement of how complex the writing is—the more complex, the more likely it is written by a human. The burstiness score looks at the variation in sentences—an essay written by AI is likely to have sentences of the same length, while a human varies the length of sentences.

I just copied the above paragraphs into GPTZero Classic and these paragraphs earned a perplexity score of 133.62 and a burstiness score of 282, and a final score of 79.43, which indicates that the text is likely human-generated.

I did ask ChatGPT to “create an engaging lesson plan to teach systems of linear equations.” It gave me a decent lesson plan, but it wasn’t very engaging because we need humans to be creative. I decided to take some of this AI lesson plan and put it into GPTZero so that I could compare scores. It gave the lesson plan a perplexity score of 119.44 and a burstiness score of 282, with a final score of 74.65, indicating that it is also human-generated. I am suspicious that ChatGPT found a lesson plan online and plagiarized it, which brings up another problem if students try to use it for a writing assignment. Keep in mind that I am using GPTZero Classic, which isn’t the latest version. One needs to sign up to use the newer version of the app.

ChatGPT curates the web for material before 2021—therefore, it isn’t going to have the most up-to-date information. I do understand why so many teachers are concerned about ChatGPT, especially when writing essays. We as educators need to be very clear with our expectations with assignments and likely we need to add something about using AI for writing to our syllabus. Software like ChatGPT is why many educators have a love-hate relationship with technology.

I will blog later this semester about ChatGPT and mathematics.