How AI Is Changing Academic Grading for Professors
The sudden emergence of large language models (LLMs) like ChatGPT and related AI tools has caught the academic world off guard. For the first time in human history, we have tools available to a wider public that can write text on virtually any topic, producing content almost indistinguishable from human-written text.
Universities worldwide still grapple with this new reality, struggling to agree on common guidelines for the accepted use of AI in academic settings. Meanwhile, students are quicker to adopt this new technology and already use AI for their studies, outpacing their institutions' ability to develop coherent policies.
As educators, we find ourselves at a crossroads. On the one hand, this powerful technology has the potential to revolutionize learning and education. On the other hand, it is evident that we need to fundamentally adapt our traditional paradigms of teaching and assessment. In this article, we'll explore how AI is reshaping the teaching and grading of academic writing, in particular written assignments like essays and reports, and offer insights into how educators and students can effectively use AI in this domain.
Let’s face it – our students are using AI
It's no secret that students are increasingly turning to AI tools to assist with their essays and assignments. This trend has sparked debates about academic integrity and the future of education (McKenzie, 2023). However, rather than viewing AI as a threat, we can see it as an opportunity to reimagine how we teach writing and assess student work.
First, we must teach the principles of research integrity early in the curriculum so students understand the long-term benefits of ethical research practices for their own intellectual and professional development. Guarded by these principles, students can navigate the complexities of the digital age, whether they're using AI tools or traditional research methods.
Second, students must learn about AI and its responsible use early in the curriculum. This includes teaching them how to critically evaluate AI-generated content, understand the limitations and potential biases of AI tools, and avoid common pitfalls such as halucinations and plagiarism.
Rethinking Writing Instruction in the AI Era
Essays and other forms of written assignments are an important part of many academic disciplines, in particular in the humanities and social sciences. But writing reports and short papers is also an integral part in the STEM disciplines.
With the rise of text generation tools, we may be tempted to abandon written assignments in our curricula altogether or to use AI detection tools to sanction their use. But I think neither of these solutions are practicable. First, AI supported writing is here to stay and we must prepare our students for this new reality. Second, AI text detection is a loosing battle. Already now many AI detection tools have high false-positive rates and are often biased against non-native english writers.
So what is the alternative? As educators, we must remember that writing is more than just producing text. Writing is also the foundation of academic communication. Writing is an indispensible skill that allows students to develop their full mental and cognitive abilities. A report by the National Commission on Writing (2003) emphasizes that writing is not just an option for young people and adults, but a necessity in today's complex, information-rich society.
Shifting the Focus of Assessment – Writing as a Guided Process
So how can we assess written assignments in the age of AI writing tools? The answer is that our approach to assessment needs to evolve, too. Instead of solely evaluating the end product (the essay or report), we must focus on teaching and assessing the skills required to write effectively. We must teach writing as an iterative process where feedback and assessment are possible in each iteration (or revision of a draft).
Traditionally, this would have been too time-consuming to implement effectively. However, with the right AI tools, it is now possible to give formative feedback at almost every step of a student's writing journey.
Large Language Models (LLMs) like ChatGPT, Claude, or Gemini in particular can help us analyze students' essays as they develop from the first outline to the final version. At each stage, we can use them to provide feedback that motivates and guides students, fostering a growth mindset and encouraging continuous improvement (Joksimović et al., 2019). This approach aligns with the principles of formative assessment, which has been shown to significantly improve student learning outcomes (Black & Wiliam, 1998).
Implementing AI-Assisted Feedback
Let's explore how we can use AI to support this process-oriented approach to writing instruction. For the sage of simplicity, we consider 3 different writing stages plus the final document. At each stage, we use AI to process the draft and to give feedback according to our criteria. We can then read, correct or extend the feedback. Here are some example prompts for each stage:
Outlining Stage: Prompt: "Review this outline and provide feedback on structure, logical flow, and potential areas for expansion."
First Draft: Prompt: "Evaluate this draft for clarity of argument, use of evidence, and adherence to academic writing conventions. Suggest areas for improvement."
Revision Stage: Prompt: "Compare this revised draft to the previous version. Highlight improvements and identify areas that still need attention."
Final Assessment: Prompt: "Analyze this final essay against the rubric criteria. Provide a detailed evaluation of strengths and areas for future development."
By using AI in this way, we can provide more frequent and targeted feedback, helping students develop their writing skills more effectively.
Structured AI Tools for Academic Writing
General purpose tools such as ChatGPT or Gemini are already very helpful, now imagine what tools that are tailored to academic writing instruction could do for you. Thesify.ai was designed specifically for academic writing and provides structured and actionable feedback on essays, reports, and other documents. Thesify uses standard rubrics, aligning with established academic standards and making it easier for educators to maintain consistency in their assessments. The platform also recommends relevant sources for citation, promoting thorough research and proper attribution. Most importantly, Thesify is built on the principles of academic integrity, supporting students in improving their writing without doing the work for them.
The Future of AI in Academic Writing
The future of academic writing instruction lies in striking a balance between leveraging AI capabilities and preserving the core skills that make writing valuable. As educators, our role is to guide students in developing these skills while teaching them to use AI tools responsibly and effectively (Chami, 2023). As we are learning to live with AI tools as integral part of our daily academic routine, we must always remember that they should enhance, but not replace, our judgment and expertise. In our teaching and assessment practices, AI tools can not only provide more personalized and effective support to our students but also create a more engaging and effective learning environment that prepares students for the chances and challenges of our new AI-driven world.
References
Lumina Foundation (2024) The Latest Insights into Academic Integrity
Technical University Munich. (2023). Guidelines for AI and Assessment: How do top-ranking universities respond to the challenges? https://www.edtech.tum.de/guidelines-for-ai-and-assessment-how-do-top-ranking-universities-respond-to-the-challenges/
National Commission on Writing. (2003). The Neglected "R": The Need for a Writing Revolution. College Board. https://files.eric.ed.gov/fulltext/ED475856.pdf
Black, P., & Wiliam, D. (1998). Inside the Black Box: Raising Standards Through Classroom Assessment. Phi Delta Kappan, 80(2), 139-148. https://kappanonline.org/inside-the-black-box-raising-standards-through-classroom-assessment/
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education, 16, 39. https://doi.org/10.1186/s41239-019-0171-0
WEF (2023). The future of learning: How AI is revolutionizing education 4.0 https://www.weforum.org/agenda/2024/04/future-learning-ai-revolutionizing-education-4-0/
Balfour, S. P. (2013). Assessing writing in MOOCs: Automated essay scoring and calibrated peer review. Research & Practice in Assessment, 8, 40-48. https://eric.ed.gov/?id=EJ1062843
Chami, G (2023) Artificial intelligence and academic integrity: striking a balance. https://www.timeshighereducation.com/campus/artificial-intelligence-and-academic-integrity-striking-balance