Design and Deliver a Rubric

4. Step 4 - Discuss Quality

How many delineations of quality can you reliably distinguish for each of your criteria? It's important to note that instrument reliability decreases with number of delineations. Also, you actually start using norm-referencing to make your judgements if you have too many, which leads to conversations with yourself that go something like this: "I don't know what makes it a 6/10 essay introduction, except for the fact it's not quite as good as the one I gave a 7/10".

Not all criteria need to have the same number of delineations. Many criteria are best measured as a yes/no, whereas some criteria could be performed at an expert, excellent, satisfactory, or not-yet level. 

If your institution assigns letter grades, you might think about some criteria (or the entire performance) along an A-F grading system. Can you clearly describe the difference between an A, B, C, D, and F essay, oral presentation, reflection, or portfolio?

Many criteria cannot be described as discrete levels of performance and are more like a sliding scale of developing competency. This is important to think about before creating your rubric. For example, do you know when a "Satisfactory" becomes a "Proficient"?

Write out some notes about each criteria that include descriptions of the range of quality you might see, considerations, and rationale. See below for an example of some notes I've made to myself in advance of building a rubric for an oral presentation.