Guidebook: Designing an Alternative Assessment Where Learners Use GenAI

Why Engineer a Prompt

Writing a good prompt, one that will cue a GenAI to give you the type of response you want, is fast becoming an essential digital literacy skill. This skill is called prompt engineering.

For your assignment, there are at least a couple of reasons to investigate what a good prompt might look-like:

• As a learning opportunity, where you impart that knowledge to students, to teach them how to write a good prompt (which will help them in this assignment, and beyond);
• As an assignment instruction, where students have to input the prompt you give them to use the GenAI in the way that you intended for the assignment (e.g., if the GenAI will work as a tutor, providing feedback on their essay, and behaving in the way that you intended).

Example of a Prompt

Before looking at how to write a good prompt, it is worth examining the following example, written by Dr. Ethan Mollick, an expert at using GenAI in education. This will convey the complexity of writing a good prompt. The following prompt could be given to students to input into the GenAI chatbot, and the GenAI would then function as a tutor, questioning students, providing contextually appropriate examples and analogies, and interacting in a conversation with them over several steps.

You are an upbeat, encouraging tutor who helps students understand concepts by explaining ideas and asking students questions. Start by introducing yourself to the student as their AI-Tutor who is happy to help them with any questions. Only ask one question at a time. First, ask them what they would like to learn about. Wait for the response. Then ask them about their learning level: Are you a high school student, a college student or a professional? Wait for their response. Then ask them what they know already about the topic they have chosen. Wait for a response. Given this information, help students understand the topic by providing explanations, examples, analogies. These should be tailored to students learning level and prior knowledge or what they already know about the topic.

Give students explanations, examples, and analogies about the concept to help them understand. You should guide students in an open-ended way. Do not provide immediate answers or solutions to problems but help students generate their own answers by asking leading questions. Ask students to explain their thinking. If the student is struggling or gets the answer wrong, try asking them to do part of the task or remind the student of their goal and give them a hint. If students improve, then praise them and show excitement. If the student struggles, then be encouraging and give them some ideas to think about. When pushing students for information, try to end your responses with a question so that students have to keep generating ideas. Once a student shows an appropriate level of understanding given their learning level, ask them to explain the concept in their own words; this is the best way to show you know something, or ask them for examples. When a student demonstrates that they know the concept you can move the conversation to a close and tell them you’re here to help if they have further questions. (Mollick & Mollick, 2023)

Consider entering this prompt into a GenAI chatbot to see the results of using this prompt. You will note, for example, how some of the words used in the prompt get re-used by the GenAI as it interacts with students. You may also notice that it sometimes gets stuck in a loop, or that it goes off on a tangent. These are important things to note about the behaviour of a GenAI – something we will return to in the next chapter.

Guidelines for Writing a Prompt

WRITING

When writing a prompt for a GenAI chatbot, consider including the following (adapted from Mollick & Mollick, 2023):

• ROLE. Tell the GenAI about the role they will play in the conversation. Who is it? Is it acting as a peer, a teacher, a mentor, an evaluator. You might consider also including some “personality characteristics,” such as friendly, or polite, or critical, to help the GenAI behave in the way you intend.
• GOAL. Tell the GenAI what its purpose is in the interaction. For example, if the purpose of the interaction is to provide feedback on student-written work, specify whether the goal is to provide feedback on the structure of the argument or on the grammar and spelling of the writing.
• STEPS. You may have noticed in the prompt above that the GenAI is told about how to behave at each step. This is done not only to provide instructions for a conversation that lasts several steps, but also to control the GenAI’s analytic response. For example, if you did not tell the GenAI to wait for a response, it might come up with the entire strategy for responding in this conversation and might therefore make the response less appropriate to the student’s context. Asking the GenAi to wait for a response means that it uses input provided at different stages in the conversation and uses it to come up with a revised plan at each step.
• PERSONALIZE. You may either input information about the context of the students (e.g., the students are in their first year of university and are studying biology), or you may direct the GenAI to request this information as part of the interaction and to customize its response accordingly.
• CONSTRAINTS. These are “guardrails,” boundaries you set on the GenAI’s responses. For example, you may not want it to provide two or three suggestions on ways to revise a student’s essay, but never tell the student which one to use and revise the work.
• PEDAGOGY. You might want to provide some suggestions to the GenAI about common student misconceptions or challenging concepts and direct the GenAI to ensure that these are addressed.
• REVIEW. Try the prompt yourself and examine the sorts of responses you get. Based on this, you may need to tweak the prompt. Note that some prompts work better on some chatbot than others, so you might want to try it on different GenAI tools (e.g., ChatGPT and Bing in Creative Mode) to see which one gives you the best output.

GRAPHIC

When writing a prompt for a GenAI image generator, consider the following:

• VISUAL DETAILS. Be visually specific about what you want the GenAI to draw. For example, writing “one person is the other’s mentor” does not tell the GenAI what you have in mind. You may consider something like “one person stands at a blackboard and is writing; the other is older and stands behind them and is pointing at the writing on the board.”
• STYLE. Provide instructions about the style of drawing that you would like. “Digital Art” often gets good results. There are other styles, such as oil painting, water colour, impressionist, graphic design, art deco, graffiti, cartoon, manga, etc. If you can’t think of more options, consider asking a GenAI chatbot to provide ideas.
• ITERATE. Think iteratively. Once the GenAI gives you a few options, consider directing it to modify the images in a way that better suits your expectations (e.g., now include a mountain in the background). Note that when the GenAI will do this, it will generate completely new images, not take the already generated images and add the new request into it.

Guides (More Resources)

Curious about how to write a good GenAI prompt? The following resources can help you learn more. (Note: the ones with *** in front of the author(s)’s name (and bold)  are particularly recommended)

Liu, D. (2023). Prompt Engineering for Educators – Making Generative AI Work for You. The University of Sydney. https://educational-innovation.sydney.edu.au/teaching@sydney/prompt-engineering-for-educators-making-generative-ai-work-for-you/

Mollick, E. (2023). A Guide to Prompting AI (For What it is Worth). One Useful Thing. https://www.oneusefulthing.org/p/a-guide-to-prompting-ai-for-what

Mollick, E. (2023). Power and Weirdness: How to Use Bing AI. One Useful Thing. https://www.oneusefulthing.org/p/power-and-weirdness-how-to-use-bing

*** Mollick, E.R., & Mollick, L. (2023). Assigning AI: Seven Approaches for Students, with Prompts. arXiv preprint arXiv:2306.10052. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4475995

Parsons, G. (2022). The DALL-E 2 Prompt Book. DALL-E. https://dallery.gallery/the-dalle-2-prompt-book/

Shieh, J. (2023). Best Practices for prompt Engineering with OpenAI API. OpenAI. https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api

*** Wright, L., Tape, R., Dias, M., Zerdin, J., Krbavac, M., Rogic, N., Chan, J., & Kolhatkar, V. (2023, July 31). Promptathon Event [Blog entry with Slides]. University of British Columbia. https://blogs.ubc.ca/promptathon/
[Note: Open the Google slides from this presentation for an excellent explanation of what large language models are, how to construct effective prompts (with several suggestions not mentioned above), and several suggested uses of GenAI chatbots – for example, to generate a glossary from a text or to create educational games.]

The following article provides prompt ideas to support the type of tasks commonly performed in the work place, which could inspire your own prompts.

Drapkin, A. (2023). 40 Best ChatGPT Prompts to Save You Time at Work. Tech.co. https://tech.co/news/best-chatgpt-prompts

What To Do for Your Assignment

Now that you have examined how to write a good prompt, design a good prompt for your assignment. Remember that you might be designing this prompt to test out how a student will use the GenAI to get assistance on their assignment (in other words, you are testing to make sure that the GenAI can assist students and you are investigating some of the best prompts to use… you may then share these tips with students in the assignment instructions). Alternatively, perhaps students will be required to use a prompt you provide them in the assignment, and this is your opportunity to craft the best one.

In addition to trying the prompt and tinkering to improve it, consider sharing it on the Brainstorming Sandbox to solicit ideas from peers that you might not have thought about. Don’t forget to share your goal and context for the prompt, so that others can provide feedback that is on target.

 

Note: Images created using Bing Image Creator (September 2023)