Introduction to Generative AI in Assessments

GenAi can support education by assisting in the work of educators, as well as learners.

Educator Uses

GenAI can be used by educators to help them design and develop their courses. Ethan Mollick, an Associate Professor at the Wharton School of the University of Pennsylvania who studies the impact of GenAI in education has written this highly readable article Using AI to make teaching easier & more impactful where he suggests five (5) ways in which educators may use GenAI in their work.

Here are some examples of the ways in which it may be used:

• SYLLABUS. Tell the GenAI (ChatGPT would be a good choice for this one) some parameters for your course’s pedagogy (e.g., I would like to scaffold all assignments and ensure there are opportunities for peer learning). Then ask the GenAI to take you through the steps of creating a syllabus that matches this pedagogy, requesting your input to customize the syllabus to your class context along the way.
• LESSON PLAN. You can ask the GenAI (e.g., ChatGPT) to create a lesson plan to help teach a concept using specified pedagogy. You can also simply ask the GenAI for an idea for a class activity to suit your audience and the learning objective. It can inspire some new learning activities in your class.
• EXAMPLES. One of the most challenging aspects of teaching is to find good examples of a concept. You can ask the GenAI (ChatGPT would be a good one to use here) to generate a few examples of a concept you plan to use in class. These examples can be used in your lecture, or they can be used to help learners gain an understanding of the concept through Inductive Learning (where you give learners a few examples and ask them to extract the commonalities to define that concept). Or, consider asking the GenAI to create three examples of essays (simulated sample student work): one that is excellent, one where the student is developing, and one that is sub-standard. Then give these examples to students and ask them to create a rubric to assess the work based on what they observe about these three examples. This will help students understand what excellent work looks-like. It also doesn’t require you to approach past students who created substandard work and ask them to share their essays for this assignment…
• QUIZ or PROBLEM SETS. You can ask the GenAI to generate a few problem sets or quiz questions – complete with solutions and answer keys – to help learners practice applying a concept they recently learned.
• RUBRIC. You can ask the GenAI (e.g., ChatGPT) to help you create a rubric that can be used to assess an assignment using criteria you specify and suggesting others.
• IMAGES. To support your slides and text, ask an GenAI image generator (e.g., DALL-E) to create original artwork. If you teach a topic where the illustration can help students understand (e.g., how an enzyme binds to a substrate), consider using the image to increase understanding, not just to make the slide more visually interesting.
• TRANSLATION. Certain GenAI tools, like Power Point Live, can live transcribe a lecture in a variety of languages, helping learners who are not native English speakers to follow a lecture more easily.

The more specific the prompt given to the GenAI in these requests (e.g., specifying the audience, learning outcomes, pedagogy, etc.), the better the output. However, expect that in nearly all cases, the GenAI will be used as a “brainstorming tool” that may inspire your own work but will likely not be specific enough to be used as is.

Faculty Focus published a wonderful article, which includes specific prompts, that educators may use to help them in their work.

The best way to think of the GenAI is as your assistant. It will provide useful resources but likely won’t be able to replace your work and do it for you. It works under your guidance.

 

Learner Uses

Similarly, learners can use GenAI to help them understand new topics and assist them in generating original work. The best way to think of it is that the GenAI is a private tutor or mentor that will provide assistance customized to each learner’s needs.

Ethan Mollick wrote the excellent article Assigning AI: Seven Ways of Using AI in Class, which provides an excellent (and highly recommended) overview of seven ways in which a GenAI can be used by students in class. This includes not only the use that students make of the GenAI (e.g., to receive feedback on work, to coach them through steps), but also the wording of the prompts to use to get the best result.

Consider the following student uses of GenAI to assist them in their learning:

• UNDERSTANDING. Students can ask the GenAI (like ChatGPT) to reword the description of a new concept (in a language of their choice), to draw up the concept or concept map or flowchart, provide examples, and keep doing this under they understand. The GenAI is infinitely patient and can adapt to a student’s preferred mode of understanding (e.g., by using analogies to help the student understand).
• PRACTICE. The GenAI can create practice problem sets and adapt them to the learner’s understanding, adding more practice in areas where the student’s grasp appears to need more practice, and stepping up the difficulty as the student gains strength.
• FEEDBACK. As will be discussed in the last part of this course, the GenAI can be used to provide personalized feedback to learners on their work. Using a prompt that specifies what criteria to focus on, the GenAI can review each student’s work and make suggestions about where to continue working to improve the work.
• OPEN MINDSETS. Ask students to consult a GenAI to present an alternative view to their position or to poke holes in their argument. Humans tend to become entrenched in a position, and the GenAI can provide a wider perspective that helps learners see other points of view. (see for example Ethan Mollick’s article Thinking companion, companion for thinking)
  
• COACH. The GenAI can guide a learner to do a complex task by providing instructions and taking students through these instructions, one step at a time. For example, the GenAI can help students create an outline and then work from that outline one paragraph at a time to write an essay.
• CRITICAL THINKING. Current GenAI output are far from perfect. Though impressive, they are often imprecise (they offer general responses), often inaccurate, and they reproduce the bias in the data upon which it was trained (e.g., if you ask a GenAI to draw a leader, they will tend to draw men). Asking students to critically evaluate the GenAI’s output, making them aware of these limitations, and exercising their critical thinking skills, can ensure that they use GenAI as a tool, rather than depending on it to do the work for them. They stay in control of the GenAI when they use it.
• DIGITAL LITERACY. Most experts agree that GenAI is here to stay. The fact that Microsoft is working to integrate it into its software means that it will soon become ubiquitous. It is a new tool that will be integrated into many industries. Students need to learn how to use it effectively and responsibly. This means that teaching them to use GenAI is important, as are the metacognitive skills to continue developing these skills (e.g., asking them to reflect on the effectiveness of their prompt and how they could formulate it better).