If you’ve been reading this blog for a bit, you may remember that I’ve been experimenting with using generative artificial intelligence (GenAI) to support my students’ writing. Last fall, I wrote several posts on my efforts using ChatGPT to provide feedback on drafts that my students wrote. My goal was to use GenAI to provide some immediate corrective feedback to students on things like grammar, spelling, and formatting which would allow me to focus on higher-order feedback. If you’re interested in that journey, check out these blog posts from the fall:
Feedback with ChatGPT
Discussing Feedback with ChatGPT
Responding to ChatGPT Feedback
My graduate students submitted their first writing assignments last week and I incorporated GenAI into the writing and revision process again. This time, I chose to use Claude to provide feedback. Claude allows users to upload documents the the tool can then interact with. Last fall, my students had to copy and paste segments of their paper into ChatGPT to get feedback. With Claude, the students could upload an entire PDF of their paper and have the tool interact with the whole document. ChatGPT (and other GenAI tools) offer file upload functionality in their pay versions, but I didn’t want my students to pay for a tool. So, after playing around with Claude, I decided to implement the tool in this iteration of the activity.
To provide some support and guidance with using Claude, I offered the following prompts (and recorded a short video demonstrating the process):
- Acting as a writing tutor, can you offer suggestions for improving the structure and organization of this paper without revising it?
- Acting as a writing tutor, can you offer suggestions for improving the grammar and word usage of this paper without revising it?
- Acting as a writing tutor, can you offer suggestions for improving the in-text citations of this paper without revising it? Please follow the APA 7th Edition guidelines.
- Acting as a writing tutor, can you offer suggestions for improving the references of this paper without revising it? Please follow the APA 7th Edition guidelines.
- Acting as a writing tutor, can you offer suggestions for improving the paper formatting of this paper without revising it? Please follow the APA 7th Edition guidelines for a student paper.
You may notice that the prompts have become more specific than the one I offered to students last fall. After my students submitted their revised drafts last semester, I analyzed their common mistakes and the types of feedback that ChatGPT gave them. I realized that the prompts needed to be more specific to better address the errors my students were making. To chart their revision process, I had students submit their first draft, a copy of Claude’s feedback, and their revised drafts. This gave me a window into the overall feedback process and how it informed my students’ work.
Here are some takeaways from this iteration of GenAI feedback:
- Claude mostly gave indirect feedback. Unless prompted for more specifics, Claude tended to give indirect feedback on students’ writing. For example, for one student’s paper, Claude provided this feedback: “Review verb tenses. Some sentences shift between present and past tense. Using past tense consistently for summarizing and discussing studies is appropriate.” This type of indirect feedback is helpful to students, but it requires that students can recognize the difference between verb tenses. Students tend to respond better to direct feedback, where specific errors are identified. Looking at the Claude feedback that my students submitted, some asked follow-up questions seeking specific examples of some of the issues. I’ll probably include more guidance with asking follow-up questions next time.
- Claude made some mistakes. This shouldn’t be a huge surprise, but Claude didn’t always get it right, especially with APA formatting. The most common mistake that Claude made was with the number of authors to cite for a work with three or more authors. While the reference includes all of the names of the authors (up to 20), the citation only includes the first author, followed by an “et al.” Claude consistently bungled this.
- My students sought and appreciated the feedback. Despite the mistakes that Claude made and the type of feedback it offered, my students responded favorably to the activity. While I suggested that students could use whichever prompts they felt they needed, most of them used all five prompts to receive feedback on their papers. I also surveyed the students after this exercise and asked how likely they were to choose to use an AI tool again to guide their writing and revisions. Almost all of them said they planned to use GenAI to support their work. For example, one student wrote, “I would likely use an AI tool again to guide my writing and revisions. I feel that, as long as I double check that its recommendations align with formatting guidelines, it doesn’t really hurt to have another set of (artificial) eyes look over my paper.” It’s clear that students saw the benefits and also recognized the limitations of AI-generated feedback, at least concerning their own learning.
- Student use did not always translate to their practice. All of the students in this graduate class are teachers. While they work in different settings and environments, they all support their own students’ learning. When asked whether they planned to use GenAI with their students, half of the respondents did not foresee using it. As one student wrote, “I do not think that I would use an AI tool in my classroom. To be frank, I’m afraid that students would use this technology to do the work for them, and not to just assist them.” While my students saw the benefits to their own learning, they weren’t ready to incorporate GenAI tools into the classroom.
The 8 Blog 