Thinking about our Assessments in the Age of Artificial Intelligence (AI)
This post was written in January 2023, for a more current piece on the topic please see the blog post series from July 2024, Addressing Academic Integrity in the Age of AI
by Krys Ziska Strange, Associate Director of Instructional Design & Faculty Development, TTS & Carie Cardamone, Associate Director of STEM, Professional Schools & Assessment, Tufts CELT
The recent release of Chat GPT and media coverage has prompted instructors to explore AI tools to understand their capabilities and limitations. It has evoked anxious conversations about how well these tools are able to perform tasks we ask our students to complete – from writing code to creating essays and reports – and resulting implications for learning and academic honesty. At the same time, many instructors are creatively thinking about how to leverage AI tools for learning.
Past experiences with new technologies like calculators and smartphones have shown us that initial worries about how the technologies will help students cheat or detract from what students “need to learn” can give way to innovative new approaches for education. Eventually, we adapt and begin to leverage these new tools for learning. Battling the tide of innovation in technology is an exhaustive fight we will not win. What if we can use this moment as an opportunity to refocus our attention on a different question: “What activities and assignments will have the most value to my students’ learning today?”.
There are many ways to promote and assess student learning that are not easily replicated by AI, and some that leverage its capabilities. After all, students will be entering careers where they will have to navigate and critically evaluate rapidly evolving sources of information and new technologies to solve complex problems. Each of the sections below offers ideas that are resilient in terms of both their effectiveness for assessing learning, doing work that cannot easily be gamed, and developing skills that will support continuous learning. Many of these are tried and true, but under-utilized, others embrace new opportunities!
Embrace authentic assessments
Consider refocusing course activities where possible so that students can apply their learning to real world contexts and problems and find personal relevance. This can positively influence both motivation and long term memory. These authentic assessments teach students how to approach multifaceted, complex questions where there is not always a right answer, and develop their analytical skills. You might engage students in creating a portfolio of their work with critical reflections on how course experiences, and activities have impacted their learning and, if relevant, the impact on the communities in which they are situated. A learning portfolio is a visible way to help students develop metacognitive (thinking about thinking) skills, become critical thinkers, and open a space for reflective dialogue with you to track and adjust their learning progress.
- See this blog with principles of authentic learning experiences (User Generated Education)
Create social learning opportunities
Social interactions have always been essential for learning. Motivating students to engage in their learning with peers can also help them feel a sense of belonging within a classroom community. Group activities with well designed questions or prompts can provide students the opportunity to articulate and challenge their own understanding of concepts, integrate other perspectives, and reveal gaps in their knowledge. They can also motivate students to come prepared to engage in these activities with their peers.
Varying the structure of these collaborative engagements (whole group, dyads, small groups) and the form (problem solving, games, idea or question generation, case studies, hypothesis testing, collaborative annotation of readings using Perusall or Hypothes.is) can promote both individual and collective learning. Creating opportunities to scaffold peer feedback on assignments engages students in understanding their peers’ approaches to an assignment thereby expanding their own thinking and approaches.
- See Universal Design for Learning in Higher Educations page on Defining Social Learning (UDL on Campus)
Focus on process over product
Traditionally, the end product of a paper or project has carried most of the weight for assessing learning. As AI improves, and it will improve rapidly, reframing our priorities by weighting process and value, revision and improvement more heavily will be important. When designing a writing assignment or a project, you might break it into smaller components that assess skill development at each stage, using self and peer feedback submitted with changes prior to your feedback. This refocusing communicates the value of engaging in the process and circumvents possible shortcuts that will deprive the student of pride in their work and learning. Many instructors already do this by requiring drafts and rewrites of thesis statements, citation searches, reflection, brainstorming documents, etc.
- See Assessing the learning process not the product (Modern Learners)
- See CELT’s Developing Syllabus Statements for AI for advice and example syllabus policies
Ask students to get creative
Projects that engage students in thinking beyond the traditional disciplinary modes of communication can give students valuable experience translating dense concepts for different audiences. Consider alternative models for students to express their learning, such as conference posters, information graphics, podcasts, maps, artwork, storyboards, song writing, poetry, or research presentations. These creative expressions build on universal design for learning core principle, “Providing Multiple Means of Expression, which increases equity in the classroom.
Be transparent about your expectations for the use of AI
One of the easiest ways to encourage authentic engagement with course activities and to discourage academic dishonesty is to be transparent about your expectations. Include a statement in your syllabus that tells students what is (and is not) allowed in your course. Be explicit about how students should cite the work of others and a policy for citing the use of AI tools in learning or in their work. You might also include specific guidelines on when (human or AI) collaboration is encouraged, and when it is not.
A key component of transparency in this instance is to explain and continuously reinforce how your various assessments will benefit them in your course, their other courses, or in their career. Be explicit about the purpose, tasks, and criteria for success on your assessments so that students may understand the value and benefit of the effort.
Have students explore AI tools
Rather than avoiding AI tools or trying to catch students misusing them, you might challenge them to discover examples where AI tools provide inaccurate or incomplete information, reinforce misconceptions, or are missing alternative perspectives. You could ask your students to deconstruct a text written by AI for its strengths and weaknesses. Students can compare AI generated responses to their own work, or use AI in specific stages of an assignment (always with attribution). Or, you might ask them to use an AI tool to generate a piece of writing to compare and contrast with their own writing, and reflect why they would incorporate or reject differences. See Teaching Students to Write with AI: The SPACE Framework for an example. Students might even help you develop a classroom policy for the appropriate use of AI tools.
Get creative with technology
There are some creative applications of free or adopted tools that you can use to help dissuade use of AI-generated text. You can have students share links to their Google Docs and explore the versioning history of the text. You can also embrace the commenting features on Canvas assignments and have students respond to questions to earn back points or add bonus points. Or you can have students create marginalia on printed documents and have them scan/upload the documents using free apps such as Adobe Scan.
So, what’s next?
The rapid and constant change in the use of AI makes a cure-all assessment approach next to impossible. Even during the writing of this article, several AI-assisted research and writing tools have begun to gain popularity and notice in academia, and dozens of new articles have appeared. We recommend you explore one of these new AI tools (e.g., ChatGPT, MidJourneyAI, Consensus, Elicit, The Good AI, etc.), while keeping in mind that their capabilities are quickly evolving.
Regardless of how each of us chooses to adapt to the development of AI systems in our individual courses, it’s time to move beyond thinking about AI systems as something to “battle against” and begin to think of how these systems can help us to learn. They can be tools for revising writing, guiding studying, answering questions, searching references, and for reflecting on and expanding our ideas. Perhaps these systems can instead provide an exciting opportunity to reimagine a more resilient, equitable, creative, system of assessment, and even have fun in the process.
See Also
Resilient and Equitable Teaching and Assessment Require a Paradigm Shift (by Annie Soisson, Faculty Focus)
Designing Courses in the Age of AI (Teaching@Tufts)
Getting Beyond Google: Simple Ways to Help Students Critically Evaluate their Sources (Teaching@Tufts)
ChatGPT: Understanding the new landscape and short-term solutions, (Resources compiled by Cynthia Alby)
Chat GPT and Artificial Intelligence Tools (Georgetown University)
Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.9
Image credit: Image created with Adobe Firefly
