Previous

13. Evaluating AODs

Assigning grades to online discussions is the biggest predictor of their success. If no grade is assigned, students are less likely to participate. It’s recommended that participation in online discussions counts for 10% to 20% of the course grade; research shows that no additional benefits result when the grade is increased above 20% (deNoyelles, Zydney, & Chen, 2014). 

How can educators provide feedback to learners about the quality and quantity of their contributions to an AOD? There seems to be three ways to do it.

Holistic Participation

Some courses consider discussion as a component of class participation. As such, each contribution is not evaluated and noted, but rather, a student's whole contributions over the course of the semester become part of their participation grade.

Point System

Points can be used for different things (quality or quantity) and awarded by different stakeholders.

Automated

Perhaps the easiest way to do it is to automate the process in the LMS and allow the system to grant 2 points for every original post and 1 point for each reply, up to a maximum of 10 points (or whatever works in your grading scheme). While this doesn't provide much feedback to learners about the quality of a good post, it can allow instructors to include some AOD in large classes in a way that is automated.

Distributed

The points can also serve as a way to distribute participation. In one class, learners could earn up to 10 points for AOD. They had to participate in at least 3 of the 5 course AODs. They had to make at least 2 original posts, each worth 2 points. And they had to make at least 2 replies, each worth 1 point. How a student choose to allocate their 10 points was up to them, and the system gave students flexibility in where and when they devoted effort to the discussion.

Selective

Another way to go, which can provide more in-depth feedback to learners, is to generally let discussions go as holistic participation grades (see above), but each learner could flag one of their post for grading (for points). This would ensure that each student would really do their best in posting one post or reply, and the educator could then provide feedback on the quality of that response. This system makes the "grading/feedback" work for the educator manageable, while still providing feedback on the quality of the work.

Kudo Points Awarded by Other Learners

Another way to do it is to allow learners to award "kudo points" to their peers. Each student is given 10 kudo point to award. You may want to impose rules, like the fact that students cannot award kudo points to themselves, that they cannot award more than 1 kudo point to any given prompt, etc. The educator uses the kudo points to give "bonus marks" to the learner who was awarded kudos, or to select a grade for that student. This system works best when learners are less likely to use the kudo points to help a friend, so it is better used with more mature learners.

Rubrics

Rubrics are perhaps the best option for evaluating AOD contributions. They make explicit the educator's expectations for a quality contribution and the frequency of contributions. They allow learners to rate themselves and develop meta-cognitive skills. And they make grading more transparent.

To help learners understand your expectations, consider sharing a few examples of contributions to an AOD. Then, ask students to use the rubric to rate the quality of each contribution. You can discuss the students' ratings, how they came to these ratings, and disclose your own. You can also start with examples, and work as a class to identify criteria that make for a good contribution and build your rubric together. This will give students buy-in.

Here are a few examples of rubrics to evaluate students' contributions to an AOD:

In designing your own rubric, consider that Pelz (2004, cited in Cranney et al., 2011) recommends that students ask these questions to themselves as they post:

  1. Is the information accurate? 
  2. Is your post relevant to the topic under discussion?
  3. Does your post answer the questions required? 
  4. Does your post teach something new or apply a concept in a new way?
  5. Have you added to the academic atmosphere of this course? 

Another source of inspiration is a paper by Bernstein & Isaac (2018) that examined criteria that could be used to evaluate critical thinking in an AOD.

References

Bernstein, A. G., & Isaac, C. (2018). Critical thinking criteria for evaluating online discussion. International Journal for the Scholarship of Teaching and Learning, 12(2), 11. 

Cranney, M., Alexander, J. L., Wallace, W., & Alfano, L. (2011). Instructor’s discussion forum effort: Is it worth it? MERLOT Journal of Online Learning and Teaching, 7(3), 337-348. http://jolt.merlot.org/vol7no3/cranney_0911.pdf

DeNoyelles, A., Zydney, J. M., & Chen, B. (2014). Strategies for creating a community of inquiry through online asynchronous discussions. Journal of online learning and teaching, 10(1), 153-165.

Attributions

Some sections of this page were adapted from:

Center for Distributed Learning (2022). Create Discussion Rubrics. University of Central Florida. https://topr.online.ucf.edu/discussion-rubrics/

Centre for Teaching Excellence (n.a.). Online Discussions: Tips for Instructors. University of Waterloo. https://uwaterloo.ca/centre-for-teaching-excellence/catalogs/tip-sheets/online-discussions-tips-instructors-0

Previous