Over the past two weeks, while I've been participating in this course I have also been having discussions F2F with a group of five instructors, one student work and an instructional designer on what we would like analytics to show us.
So we've talked and argued a lot. We listened to John Fritz's Elluminate session while writing our ideas up on whiteboard in a classroom. We debated if analytics were even required and we've also sent emails back and forth.
I found that just asking an instructor what analytics they want isn't as effective as showing them possibilities, letting them think on it and then discussing it.
We looked at analytics available in Desire2Learn, Google Analytics and Assessment Applets http://analysis.ikit.org/w/index.php/Assessment_Applets
So on an individual course level (for an online course) here's what we came up with:
Visual Design - Need a Dashboard that easily allows instructors to see data
The look of Google Analytics rather than the small font blue text on white that is provided by the current LMS- Desire2Learn.
View of analytics for the course and individual students
Warning system based on individual student participation
·Green line/ Yellow line / Red line graphic
·Could be based on the following variables
o Logins during the week
o Read discussions
o Read content
o Post discussions
o Various Criteria of Discussion quality
Motion Charts so instructors can look deeper at trends
Score cards that show site average data for the course
The two applets that impressed everyone were the vocabulary growth http://analysis.ikit.org/w/index.php/Vocabulary_Growth and writing measures http://analysis.ikit.org/w/index.php/Writing_Measures
Would like to see this type of assessment in all courses.
And the most grandiose request - an analytic that could help the instructor (prior to the essay assignment or test) determine whether the student was truly having problems with the content rather than just not putting in the time and effort required.
This is a great list and distillation. Thanks.
However, can I push you just a bit? You said "I have also been having discussions F2F with a group of five instructors, one student work and an instructional designer on what we would like analytics to show us."
Just curious: what did the one student worker say he or she understood or wanted in terms of analytics? Either for students, or for instructors -- on behalf of students?
I'm curious what impact, if any, analytics can have directly on students' self-awareness, motivation and ultimately their behavior. Ultimately, students bear at least some responsibility for their own learning, if only because each institution likely has more students who need help than instructors or advisors who can provide it one-to-one. I'm not saying we shouldn't provide instructors with the list of tools you articulate. But I think our analytics strategy should also consider the student perspective, and the benefits of self-awareness our learning environments can afford to them directly.
While I didn't post it here, we're looking at analytics from both the instructor and the student perspectives.
One of our debates was on how and if a student dashboard would be effective in helping students learn how to be "online learners." We're looking at analytics from the perspective of online learning at a community college. We have 17,000 students collegewide and our online FTE ranges anywhere from 3000 - 5000.
The questions that our student brought up are:
Would a learning dashboard motivate students?
What type of student would it motivate?
Would all of our students know how to use it?
And if students were divided into the categories of A,B, C, D, and F students would a learning dashboard make a difference to the C, D and F students?
How could you design a learning dashboard that would make a difference to the C, D and F students?
He did also express that for certain students checking the learning dashboard especially if it compared you to other students in the course may be motivating.
So what we're starting to look at is - could a learner dashboard be developed for an intrinsically motivated student that would show (or teach) them what they should be doing to be successful in the online course...
that would also serve the the student who needs extrinsic motivation in the form of their ranking in class, progress bars and maybe even more elements of social media.
We're really in the beginning stages of seeing how analytics may help us. And this course has been so helpful.