Here’s a post I shared with my fellow faculty members at Texas A&M University. It may be useful to others.
Greetings everyone,
As I mentioned near the beginning of last semester on this listserv, I decided to incorporate/embrace the use of AI for the student term papers for the two graduate courses I taught in Fall 2023. After reading more on this topic, I thought that trying to fight it may be a losing battle. The approach I used was to include this note in the instructions I provided for submitting the term papers:
“Important note: The term paper is to be done individually and must be original work. ChatGPT/Bard can be used as a preliminary tool, but the final product has to be solely written by the student. If AI is used, the student needs to write a statement at the bottom of the title page indicating how AI was utilized as a preliminary tool (generate an outline, to search the literature, etc.). If the statement is not present nor accurate and AI was found to be used, this will be considered as plagiarized work.”
Now, I would like to share my experience in grading/evaluating the term papers as well as how I will move forward on embracing AI. I am providing this information in point form.
- There was a total of 43 students in my two graduate courses (many took both courses at the same time).
- Some students asked if they could use Grammarly to improve the text. I said yes, but they needed to include this information in their statement on the front page.
- Overall, about 90% all the students provided a statement explaining how they used ChatGPT, Bard or Grammarly on the front page. From those, most were specific, but a few students just mentioned they used one of the tools or used a generic statement taken from the web (see attached paper). Examples:
“Grammarly was used to refine the report’s grammar, sentence structure, and tone. This tool is essentially a more powerful version of the generic word text editor in that it can only be used to improve the presentation of original thoughts and ideas, not create them.”
“Note: For this report, AI tools (such as ChatGPT, Grammarly, and Quillbot) were used for organizing content, correcting grammar, and rephrasing to improve clarity, coherence, and overall English expression in this document.”
“Note: ChatGPT was initially used to explore some potential topics and then to determine the layout of the paper.”
- All the papers were uploaded on Canvas. I turned on the Turnitin function that automatically screens the paper when they are uploaded, but I did not provide the results to the students. I found out this semester that the Turnitin output results included AI detection (or used to, since I no longer see it as of when this message was sent to the listserv).
- The Turnitin function flagged three or four papers where AI seemed to have been used, but no statements were provided on the front page. I contacted the students explaining that their paper seemed to have included AI generated material, but I did not see a statement on the front page. All the students said they forgot to include the statement and apologized for forgetting to include it. At this point, the AI detector seemed to have worked to identify the worked that used such tool (again it looks like this is no longer available on Canvas). None of these students claimed that they did not use AI or it was a mistake generated by the detection program.
- An important issue is that if someone used Grammarly, the AI detector flagged the text as being produced via AI in many cases. For those, I examined the statement on the front page and, if the student wrote that he or she used Grammarly for improving the text, I let it go.
- Overall, I found that students who used AI or Grammarly to improve the grammar or sentence structures provided bland manuscripts or sections of the paper. In other words, the text had no soul, so to speak*. It really felt that it was written by a computer program. I attached a term paper (without info about the student) that overused AI (flagged with very high %) (notice how the references are incomplete at the end). I reduced the grade for these cases.
- *I saw a segment on The Today Show a few months ago in which the journalist wrote two essays as if she were to apply to a university, one by her and one create by ChatGPT. The admission officer, who was interviewed for the piece, was quickly able to know which essay was written by ChatGPT. The officer said that the one created by ChatGPT was too impersonal and used words that high school students would not use in a real text. The Washington Post reported a similar experiment five days ago with the same conclusions: https://www.washingtonpost.com/technology/interactive/2024/chatgpt-college-essay-ai-harvard-admission/
- Since I have been requiring term papers in my graduate courses for almost 20 years now, I did not see any improvements with the quality of the work based on the use of AI. One reason is that the student misses the fundamental principles and subtilities associated with the topic he or she is writing on. This lowered the grade for the criterion related to “understanding the subject.”
- I had a few students who did not use any AI (no statement and no flag) and the quality of the paper varied from marginal to excellent.
- Three students were caught plagiarizing (identified via Turnitin). For all three, the students did not know how to properly cite existing material (the references could be found in the list at the end). The first question of the first assignment in all my courses requires every student to take the Academic Integrity Training from Evans Library (http://library.tamu.edu/services/library_tutorials/academic_integrity/index.html). I also tell the students that if they are not sure about how to cite previously published text properly to ask me before getting into trouble.
- Summary statement: in general, I feel I could trust the students to do the right thing with allowing them to use AI in writing term papers and reports (despite the few instances of plagiarism). However, I will need to revise the statement I used last semester to account for the issues I mentioned above. I will be stricter with the application of these tools.
Moving Forward:
- I will ask every student to write a statement on the front page about whether or not they used AI in the submitted work. For those who did not, I want to avoid students claiming they “forgot.”
- I will warn the students that using Grammarly, ChatGPT or similar program to improve the text could negatively affects the quality of the overall paper and could even show that they did not fully understand the topic of their work.
My apologies for any typos/errors (dyslexia at work). No AI program/Grammarly was used in the text above.
You all have a nice spring semester.
Best regards,
Dom