Can ChatGPT do mathematics?

My colleague, Dr. Matt Biesecker, decided to play with ChatGPT as he was preparing the Probability & Statistics final exam.  He asked ChatGPT, “What is a good exam problem involving conditional probability?”

Below is his exchange with ChatGPT:

ChatGPT:   A jar contains 5 red balls and 5 green balls. A ball is drawn at random from the jar. What is the probability that the ball is red, given that it is not green?

Dr. Biesecker: What is the solution to the problem you told me about?

ChatGPT:  The solution to the problem is that the probability that the ball is red, given that it is not green, is 4/10.

Dr. Biesecker:  Could you explain your solution to me?

ChatGPT:  The solution to the problem is that the probability that the ball is red, given that it is not green, is 4/10. This is because there is a 4/10 chance that the ball is red, and a 6/10 chance that the ball is not red.

Dr. Biesecker:  I think you are incorrect, because if the ball is not green, then it is certainly red.

ChatGPT:  This is not necessarily the case. The ball could be blue, for example.

Dr. Biesecker:  But you told me the jar only contains red and green balls.

ChatGPT:  This is true, but the problem states that a ball is drawn at random from the jar. This means that any ball, red or green, could be drawn. 

Dr. Biesecker: Then how could you get a blue ball?

ChatGPT:  The ball could be blue, but there is still a 4/10 chance that it is red.

Dr. Biesecker: But the jar does not contain any blue balls!!

ChatGPT:  This is true, but the problem states that a ball is drawn at random from the jar. This means that any ball, red or green, could be drawn.

Based on Dr. Biesecker’s exchange with ChatGPT in December, I am pretty sure that ChatGPT isn’t very good at probability questions. So, can we use ChatGPT in math class? Dr. Biesecker also asked ChatGPT to do three proofs. ChatGPT did one of the three proofs correctly—so, we could give our students the incorrect proofs and ask them to find the errors in the arguments.

During History of Math in January, we were looking at different number systems: Babylonian, Mayan, Egyptian, and Roman. My students were completing a worksheet where they needed to add two numbers in each system. I decided to ask ChatGPT to add two Roman numerals, MCXLVII and MMCDLXXXIV. It was able to identify the first Roman numeral as 1,147, but it misidentified the second Roman numeral so the sum was incorrect. I decided to try again and now (end of February), ChatGPT did the problem correctly.

While we have seen improvements in ChatGPT in answering mathematics questions, it is still not that great. In the paper (shout out to Mike Ion, University of Michigan for telling me about it), MathematicalCapabilities of ChatGPT, the authors ask ChatGPT to answer some mathematical exam-type questions. The authors created datasets and ran these datasets through ChatGPT. One of the datasets, labeled MATH (Mathematics Aptitude Test of Heuristics), included problems taken from various mathematics competitions. Two characteristics of the problems in the MATH dataset are: each problem has a unique answer and all answers can be condensed into a few characters. ChatGPT was given a random sample of problems from the MATH dataset and only answered 26% of those problems correctly.

While ChatGPT has been generating lots of interesting conversations, it doesn’t seem to be very good at mathematics. I suspect it will continue to improve, but better AI applications exist for doing mathematics problems.

Gamification in Math Class

Making math fun for kids, rather than work is a huge task that a lot of math teachers have taken on over the years. “When am I ever going to use this?” “Ugh! Why did our math teacher assign so many problems?” “Math is so boring.” These are all comments I have either heard during my time in grade school or personally said myself. I am sure teachers are getting sick of hearing it, as I am sure students are tired of doing textbook assignments with no color or taste. Well, there is a solution to that: gamification. Gamification, in the classroom, is the incorporation of games into learning which allows teachers to better engage students. 

Math is one of those subjects that students typically either understand or don’t. For those that don’t understand it, it is typically because they fail to find interest or appreciation of the subject. Using gamification when teaching math allows the students to see math in a different way, a more fun way if you will. Studies have shown that within classrooms that use gamification, students feel more motivated to learn as the gaming setting takes away the feeling of solving repetitive questions over and over again. Gamification has also had a direct impact on reducing math anxiety. Students are more open to learning in the comfort of games compared to the traditional classroom. Gamification also enables students to pay attention for a longer time because they are playing games, and few students would prefer traditional homework over gaming activities. Naturally, students see games as a challenge. At the same time, they see traditional homework as work. Moving from the traditional classroom to a gamified classroom or teaching style will motivate students to not give up as easily. From personal experience, I know I have given up on a math problem a lot faster than I have given up on beating a level of a video game. 

“But how will gamifying my classroom affect me as the teacher?” is a question that most teachers reading this blog will ask. Well, gamification will benefit you in many ways. It is easier to gauge how well students are doing in your classroom and where they are struggling through scores, charts, etc. provided through the games. It also allows teachers to reward students more often which leads to multiple different outcomes: students are more engaged, motivated, challenged, etc.

Overall, I believe that gamification brings an aspect to the math classroom that can engage, motivate, challenge, and reward students. Not only may teachers stop hearing complaints about how boring or the usefulness of their beloved topic, but they may also see growth in students like never before.

The Industrial Revolution and its Consequences: Math-Solving Apps Edition

With the introduction of more and more technology into our lives, classrooms have adapted to this change of technology over the years to better suit the students, which has led to many cool programs, apps, games, and practices being used in the classroom.  It has also led to some controversy as math calculators and help tools become widely more popular as every day passes.  But why, and what are the effects of using these calculators? 

In the year of our lord 2023, there are dozens, if not hundreds of different math calculators and math help tools available to anybody with a computer and Google.  Typing in simple phrasing like “polynomial calculator”, “integral calculator”, and “inverse calculator” into Google can give a student many results that are not only helpful, but some consider to be downright cheating.  In my humble opinion, math apps and calculators are fair game, just as long you don’t use them just to get homework done, but to actually learn the material.  This is where the problem lies entirely, how do you get students to use these math tools and calculators for the betterment of themselves, rather than using them to do homework?  That is a question I cannot answer because it is impossible to answer.  It is impossible to know a student’s intentions when doing homework. However, you will know if they “cheated” by looking at their test scores.

This also raises the question; how can you prevent students from using math help tools and calculators?  This depends entirely on how you run your classroom.  If your classroom is a traditional classroom where the students are given the material in class and are to do homework outside of class, it is nearly impossible to monitor or stop this.  Students today, myself included, have no patience to wait for a teacher to return their email.  That’s why they go to math help tools, solely out of convenience for them, which I think could be a good thing because it eliminates a teacher’s stress in responding to students’ questions, but also could be a bad thing because students could be receiving the wrong information.  If you are in a flipped classroom, you don’t have to worry about students using these math tools because there are doing their homework in class and the teacher is there to answer questions.

Overall, I believe math-solving apps and calculators are very helpful to students needing quick help on their math homework but can be abused to the point where the students are not learning the material.

Strategies for Having Successful Math Discussions

Many math teachers might struggle with finding the right problems for their students to work on during class. For new teachers in particular, this task may seem daunting or nearly impossible to do. However, at the South Dakota STEM conference, Sharon Rendon did a presentation that provided tools to help ease this process. The presenter talked about five practices that a teacher can use to help find problems that would help develop math discussions that are productive, not destructive. The five practices she mentioned are anticipating, monitoring, selecting, sequencing, and connecting.

First is the anticipating step. In this step, the teacher should do the problem themselves to try and make sure the problem is appropriate for the students. The teacher can use their technology to see if they want students to use specific websites like Desmos for coming up with the solution. Another part to this step could be anticipating what the students might get out of the problem or reviewing whether the problem is a good application of the current unit. If the students see no connection, they may not want to work on it.

Next comes the monitoring and selecting steps. This one can come in multiple parts. The first part could be the teacher identifying techniques that they used to solve the problem and make a list with the second part being walking around the classroom to see what techniques the students used. If the students get off track, then the teacher could ask a question to get them on track. The selecting step is crucial because this is where the teacher chooses what problems to give to students. The problems should be challenging because they will help understanding go up but not too challenging, so the students give up.

The last two steps are sequencing and connecting. These steps assume that students are wrapping up their work and the class is ready to come together. The sequencing step is where the teacher determines how the students should show their work. They could decide to show common misconceptions or errors first and then show the correct solution. They could also have the students show their work by drawing on the whiteboard or other surfaces to help illustrate to other students. The connecting step can help students understand how that problem ties into others that they did and possibly increase their understanding of other concepts not discussed.