Introduction to Generative AI in Assessments

Definition

The recent interest in AI in education comes from a specific type of AI called Generative Ai (GenAI). GenAI can create new content, such as written text, code, images, music, and videos.

 

How it Works
(for those who like to understand the nuts and bolts)

There are different types of GenAI (e.g., generative adversarial network (GAN), generative pre-trained transformer (GPT), and variational autoencoder (VAE)) – but differentiating between these isn’t important to our purposes. For this introduction to GenAI, it is sufficient to think of it in the following way.

GenAI is a piece of software that is designed to identify and note down connections between “things” that it encounters. People who design GenAI systems give large amounts of data to these programs and ask them to form a model of that data set. The large amount of data can be, for example, many, many, many documents and webpages from the internet. Through that training, the GenAI recognizes patterns, things that are often associated together. For example, it might identify what whenever people talk about dogs, they often also talk about barking. Bit by bit, the GenAI learns to associate these concepts. In time, it generates a hugely complex model of the data. You can think of it as a complex concept map. The GenAI keeps tabs on the probability that any two concepts in this concept map are associated.

Once you have that model, companies or researchers create an interface to allow us (humans) to interact with the GenAI model. Different companies might develop different interfaces that each tap into the same GenAI model on the back end. Each interface might look a bit different, or might be able to interact with the GenAI in a slightly different way. But ultimately, these interfaces all tap into the same GenAI.

These interfaces allow us to ask the GenAI using typed queries. We might use the interface to ask the GenAI, for example, to tell us about dogs. When we do this, the GenAI on the backend consults its model to tell us what it knows about dogs – what it has recognized to be associated with dogs. It does this by examining the likelihood that something is connected to the concept of “dog” and formulating a response that seems to have a high chance of being linked. The interface translates “what the model knows” into a format that we can interpret, like written text on a screen.

The probabilistic nature of the model means that it will not give the exact same response every time it encounters a query. In addition, a GenAI is not a static system. It learns about the accuracy of its responses from every interactions. Thus, using the same prompt, a user will obtain different responses – this is an issue that we will talk about more in terms of detecting plagiarism or of citing the work…

This is a hugely simplified way to think about it, but this is roughly how the GPT variant of GenAI (one of the three flavours of GenAI) works.

ChatGPT

As an example, the tool that has gained the most attention in education is ChatGPT. ChatGPT was launched to the public in November 2022. It is a fancy chatbot – a conversational tool that allows users to search the internet in an interactive manner. Users engage with the GenAI in the form of a written conversation. A user asks questions and can ask follow up questions and fine-tune the GenAI’s responses until they obtain something that meets their needs.

ChatGPT draws from a GenAI model that was trained on a huge number of documents from the internet to identify how language works (how to create sentences). Based on this, the GPT model learned the probability that one word should follow another, keeping in mind the topic of the conversation. From this, it is able to generate outputs that seem surprisingly “human” – conversational and coherent.

Even though ChatGPT “sounds” human, it is not. It is not sentient, it does not have consciousness. All it does is look at its model and predict the likelihood that the next word in a sentence should follow the last word it picked.

GPT is a large language model. There are different versions of it. Currently (as of Augst 2023), the two most commonly used models are GPT-3.5 and GPT-4.0. These two GenAI models can be accessed using different interfaces. For example, users can go to the OpenAI website and use the interface called ChatGPT to access GPT-3.5 free of charge, or GPT-4 with a paid subscription. Users can also use the Edge browser and consult the Bing search engine in Creative mode to access GPT-4.0 free of charge.

Note: All decorative images on this page were created using Bing in Creative mode in August 2023.