Guidebook: Designing an Alternative Assessment Where Learners Use GenAI

11. Learner Meta-Cognition

A student is engaged in thought while interacting with a robot. Supporting the development of students’ meta-cognitive skills (learning how to learn) is a goal of many post-secondary educators. Indeed, the ability to gauge where one stands with regards to learning a new task and how much more needs to be learned is a key skill that professionals must master (Lockyer et al., 2004).

Teaching students how to reflect becomes particularly important when introducing a new tool, such as GenAI, that seems to be able to do many of the same tasks that a student has historically done in their academic work.

After using the GenAI tool, you may ask students to reflect upon some of the following:

TOOL EFFECTIVENESS. How “good” was the GenAI tool? Were its responses accurate? Precise or general? Contextualized? Did it hallucinate? Given these observations, how useful would the use of such a tool be for different purposes? (e.g., in developing new medicines, in supporting students for learning, in helping computer programmers, in helping them with their next writing assignment, etc.)
CRITICAL EVALUATION. What approaches did the student use to sift through the GenAI’s responses and determine the ones to trust from those that needed to be investigated further? Under what circumstances is the tool most accurate? Should users have a priori knowledge of a topic before using the tool? How would you recommend that a user work with a GenAI tool on a class project (which steps should a learner use to conduct research and when in the process should they use a GenAI tool?)
USAGE. When using the GenAI tools, which prompts resulted in the best/worst responses? Why was this a good/bad response? Did students discover any tricks to write an effective prompt? When did the GenAI tool get trapped in a loop or when was it more likely to hallucinate? How did students do when they encountered this situation? How did they make the most of the iterative nature of the interaction with the tool? Were some GenAI tools better than others and why? What recommendations would they make to a new user of the tool? (i.e., to students doing this assignment next year)

To Do

Review your assignment’s learning objectives and consider whether including a meta-cognitive component (i.e., a student reflection on the use of GenAI and what they learned from it) makes sense. If this would help learners achieve the learning objective of the assignment, consider what you will ask learners to reflect upon and how they will share their thoughts.

Resources

Aronson, L. (2011). Twelve tips for teaching reflection at all levels of medical education. Medical Teacher, 33(3), 200-205. https://doi.org/10.3109/0142159X.2010.507714

(One of the fields where reflection has been well-integrated into education and professional practice is medicine. This article provides 12 concrete suggestions for teaching students how to use reflection to improve their practice.)

Mollick, E.R., & Mollick, L. (2023). Assigning AI: Seven Approaches for Students, with Prompts. arXiv preprint arXiv:2306.10052. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4475995
(This article has been referenced many times in this course. In it, the authors provide suggested assignment instructions including instructions for a reflection at the end of the assignment.)

Tanner, K. D. (2012). Promoting student metacognition. CBE—Life Sciences Education, 11(2), 113-120. https://doi.org/10.1187/cbe.12-03-0033
(Note: this article is written for an audience of biology educators but can easily be applied to other disciplines and indeed is a staple article in the field of metacognition in undergraduate learning)

Centre for Innovation and Excellence in Learning (n.d.). Metacognition [compilation of blog posts]. Vancouver Island University. https://wordpress.viu.ca/ciel/category/metacognition/
(Series of blog posts on the topic of reflection and student meta-cognition. The post Metacognition: Valuable Resources on Thinking about Thinking shares many resources on the theory and practice of meta-cognition in the classroom.).

Note: Images created using Bing Image Creator (September 2023)