I enjoyed today's presentation, but it left me with a nagging question about reliance on data mining with regard to going "off-task" and "gaming."
The tutor database is necessarily limited to observations of behavior, but my concern lies in accounting, for instance, for Gardner's multiple intelligences or even learning styles.
Doesn't the upshot of research on "off-task" and "gaming" drivers suggest rewriting questions and interfaces for the many when there may be other forces at work for a few such as learning styles that are in no way accessible as an overlay from the data set?
I understand your concerns -- the researches and ed. software developers need to work together carefully to decide how much data they want to see before deciding there's a design flaw in a particular course module.
Also, instead of completely rewriting a module where multiple learners go off-task or game the system, developers could built the system to adapt, e.g., change up the questions/presentation when such behavior is detected.
Also, instead of completely rewriting a module where multiple learners go off-task or game the system, developers could built the system to adapt, e.g., change up the questions/presentation when such behavior is detected.
David, I agree that it is important to understand which factors are driving student behavior; ideally, we would model the full range of student behaviors and attributes, including things like multiple intelligences and learning styles. I believe that these kinds of latent student attributes can be inferred -- at least to some degree -- from student behavior, although it's much harder than just detecting student behaviors! See Ivon Arroyo's work, for instance (such as Arroyo & Woolf, 2005). Also, some online course designers, such as Neil Heffernan, are embedding questionnaires standardly into their courses -- that's another nice way to get at this data.
Thanks Ryan,
We have recenlt been talking about the use of (simple) standard questionnaires in all of our elearning. Although there are some limits to self-reporting, having some information is certainly better than having none (which is what we most often have now!) It's good to hear of others moving in th same direction. I'll be sure to look up Heffernan's work.
We have recenlt been talking about the use of (simple) standard questionnaires in all of our elearning. Although there are some limits to self-reporting, having some information is certainly better than having none (which is what we most often have now!) It's good to hear of others moving in th same direction. I'll be sure to look up Heffernan's work.
I'd be interested to see what folks have used in terms of standard questionnaires they felt were useful in online courses (and also to hear what you felt you learned from the responses and whether/how you subsequently changed your course).