Thanks to Sten Govaerts and Erik Duval from the research unit on hypermedia and databases at Katholieke Universiteit Leuven we have a "monitor widget" for LAK11 (Moodle). To get around the access challenges of installing the script where the SCoPE site is hosted, Sten grabbed a backup of the course with logs and is giving us access through their site:
http://ariadne.cs.kuleuven.be/monitorwidget-lak11/
- enter your first and last name to view
To make the line chart more "airy", as Sten put it, he removed the facilitators from the participant list. Sten will continue to update the data every couple of days from the backup files.
I'm not always 100% sure what I'm looking at, especially under recommendations, but it's been fascinating to explore the various views and think about how I might use this information! The help document is useful.
Have a look and report back!
This is very cool - but I have to say I am disappointed on the amount of time I have devoted to the course - hoping to see my line take an upward trend over the next week!
Hi,
If you want some more explanations of the visualizations, here are some videos. The video of the parallel coordinates explains an older version of the visualization. The latest version supports re-arrangment of the axes by using drag and drop and you can visualize a histogram on top of the axes to get a better idea of the real distributions.
Sorry, but I do not seem to be able to embed the videos in the forum, so I just post the urls to the videos...
Explaining the statistics window:
http://www.screencast.com/t/MTZhMWU1
Discussing the line chart:
http://www.screencast.com/t/YTQ0NThh
...and the parallel coordinates:
http://www.screencast.com/t/ZTNkZWIwO
Finishing off with the bar chart:
http://www.screencast.com/t/ZDdjY2UyYm
Both the "coverflow" at the bottom and the tag cloud in the recommendations panel show the same data. Namely the resources on which the most time was spent. In the future we want to improve this and provide some different way of visualizing the top resources together with some added visual things, e.g. sparklines.
Any feedback, ideas and questions are of course very welcome!
Sten
If you want some more explanations of the visualizations, here are some videos. The video of the parallel coordinates explains an older version of the visualization. The latest version supports re-arrangment of the axes by using drag and drop and you can visualize a histogram on top of the axes to get a better idea of the real distributions.
Sorry, but I do not seem to be able to embed the videos in the forum, so I just post the urls to the videos...
Explaining the statistics window:
http://www.screencast.com/t/MTZhMWU1
Discussing the line chart:
http://www.screencast.com/t/YTQ0NThh
...and the parallel coordinates:
http://www.screencast.com/t/ZTNkZWIwO
Finishing off with the bar chart:
http://www.screencast.com/t/ZDdjY2UyYm
Both the "coverflow" at the bottom and the tag cloud in the recommendations panel show the same data. Namely the resources on which the most time was spent. In the future we want to improve this and provide some different way of visualizing the top resources together with some added visual things, e.g. sparklines.
Any feedback, ideas and questions are of course very welcome!
Sten
Thanks. I like this widget and would use it in some circumstances. I would like it even more if you could add time spent elsewhere - a bit like the Wii-fit gives you an option to count exercise not done on the Wii. I've grabbed a lot of files for this course and then worked offline, blogged, added chunks to VUE-map etc. I guess my working style is not even remotely unusual so to use the widget the teacher/student/administrator needs the 'health warnings' and the wider context.
Sten, thanks for your help and the technical support.
I usually do not sign into the moodle forum. The guest status is faster and more appropriate for me. It corresponds to the just-in-time-learning.
I usually do not sign into the moodle forum. The guest status is faster and more appropriate for me. It corresponds to the just-in-time-learning.
Sten Govaerts wrote,
videos.
Via Vanessa Vaile, who noted,
videos are not optimum choice for those on dial-up. ironically, open access courses like this underscore the digital divide,
You mean digital divide or gap in participation?
both ~ intensive use of apps that are bandwidth hogs is bound to contribute to participation gaps among users with access limitations
Sten,
Thanks for the cool data visualizing! I wonder how useful will Prof. Siemens find from these visual diagram.
Su-Tuan
Thanks for the cool data visualizing! I wonder how useful will Prof. Siemens find from these visual diagram.
Su-Tuan
Hi!
It is very interesting. Sten and Sylvia Thanks for providing this post. I have the impression that the more time I spend, but still do not know how to read the chart. So I will do so.
Greetings
It is very interesting. Sten and Sylvia Thanks for providing this post. I have the impression that the more time I spend, but still do not know how to read the chart. So I will do so.
Greetings
Which of the charts do you not understand?
Basically what you see in the line chart is time you spent on the course over time. On the vertical axis, you have the total time spent on the course and on the horizontal axis you have the time and dates on which you were active in the course.
So,in short: if your line goes up it means that you have been active. If it stays flat, it means that you have been idle.
Sten.
Basically what you see in the line chart is time you spent on the course over time. On the vertical axis, you have the total time spent on the course and on the horizontal axis you have the time and dates on which you were active in the course.
So,in short: if your line goes up it means that you have been active. If it stays flat, it means that you have been idle.
Sten.
This is really great and it works with Moodle!
A quick question - does it capture activity on the LAK11 site by people that are not logged in? I just spent about 10 minutes working through the visualization tools and then when I came to comment had to login. In a normal course environment, this anomaly would not occur as login is routine.
I think the combination of the personalized graphic display and the statistics display make this a powerful feedback mechanism for individual participants. The Recommendations provide a quick guide to the group's view of the most important resources.
I would use this widget on my courses if it were available.
A quick question - does it capture activity on the LAK11 site by people that are not logged in? I just spent about 10 minutes working through the visualization tools and then when I came to comment had to login. In a normal course environment, this anomaly would not occur as login is routine.
I think the combination of the personalized graphic display and the statistics display make this a powerful feedback mechanism for individual participants. The Recommendations provide a quick guide to the group's view of the most important resources.
I would use this widget on my courses if it were available.
Thank you!
It only shows the actvities of when you were logged in to moodle. When you are not logged in all your actions are tracked as the "guest" user. I removed the guest user from the visualization because it will not make any sense to see a actions from a random set of people. This problem would not exist if the course was not available for guest users. But than it would not be an "Open" course...
I can provide the tool if you are interested!
It only shows the actvities of when you were logged in to moodle. When you are not logged in all your actions are tracked as the "guest" user. I removed the guest user from the visualization because it will not make any sense to see a actions from a random set of people. This problem would not exist if the course was not available for guest users. But than it would not be an "Open" course...
I can provide the tool if you are interested!
Thanks for the offer of the tool. I will need to talk to our Moodle technical folks and try to persuade them that this is something we should try.
"Open" is an evolving notion in my mind. I struggle with what parts of a course to conduct outside the Moodle shell with all its attendant privacy and identity protections. At the same time, I'm trying to make sure that my students become proficient in using Web 2.0 and social networking tools. Also, I'm trying to develop a learning community around the topics that will exist beyond the time limits of the course.
It seems to me we are trying to define an "open" space embedded in the intersection/overlap of formal courses and personal learning environments.
"Open" is an evolving notion in my mind. I struggle with what parts of a course to conduct outside the Moodle shell with all its attendant privacy and identity protections. At the same time, I'm trying to make sure that my students become proficient in using Web 2.0 and social networking tools. Also, I'm trying to develop a learning community around the topics that will exist beyond the time limits of the course.
It seems to me we are trying to define an "open" space embedded in the intersection/overlap of formal courses and personal learning environments.
I'm interested in this tool too! 
I find it really useful to understand the students' learning style. I think it would be very useful also to build reports about the students' activities.
Can you provide it to me, please?
Vincenzo
I find it really useful to understand the students' learning style. I think it would be very useful also to build reports about the students' activities.
Can you provide it to me, please?
Vincenzo
Thanks Sylvia
When the tool counts the documents, Does this refer to the links in the forums?,
can have other activities such as lessons or SCORM,
When the tool counts the documents, Does this refer to the links in the forums?,
can have other activities such as lessons or SCORM,
Yes, lessons and scorm is also included if they are available in the course and used by the students.
The documents are indeed very wide, for example wiki pages, blog posts and comments, forum posts, files, calendars, actions in scorm packages, etc.
Maybe it would make more sense to rename it to resources, which might be perceived as a wider term than documents...
The documents are indeed very wide, for example wiki pages, blog posts and comments, forum posts, files, calendars, actions in scorm packages, etc.
Maybe it would make more sense to rename it to resources, which might be perceived as a wider term than documents...
Resources covers the range of "things" possible that extends well beyond just documents. I vote resources.
I just replaced "documents" with "resources" everywhere in the visualization. And I updated the data. So, now you can have a look at the data from the last few days.
Have a nice weekend!
Have a nice weekend!
Thank you Sten and Erik. I'm thinking even i can make some sense of this. I like that.
I may be missing the point completly, but is this data more a reflection of seat time than active learning?
I sense some anxiety re: time spent at the course hub, especially amongst those who favour distributed participation?
I'm certainly suprised at what the data suggests for my own 'participation'. Maybe it reflects my log ins where I leave moodle open without doing anything active towards course learning? Or maybe randomly clicking docs genuinely searching for learning, which would have the same data effect of gaming activity? I'm not sure. The user intent is uncertain.
Whatever this data is really telling me, I guess it makes me think about how cautiously humans should inform themselves rather than drive themselves, or student and self learning, with stuff they don't really get, yet.
The data could also reflect slower connections, which make everything take longer.
I wonder what other variables are not being taken in account.
I wonder what other variables are not being taken in account.
In my case I sometimes spend too much time reading and translating texts into Spanish. But not only from here, in the reader and email too.
Greetings
Greetings
I agree that the data can be very subjective to your learning style. Getting good data is often a hard job to build your visualization on. Especially so in an activity like learning, which takes place in many different spots and in many different contexts and often offline as well as online.
Tracking what happens after the students leave the online learning management system is of course very hard to do automatically (unless we could mount some kind of smart device on their heads that detect what they are doing and looking at ;-). We did an experiment for an evaluation of the tool, where we asked students to tweet what they were doing for the course and for how long. We used twitter because students might like it more, but most of them stopped tweeting very soon.
Next to that, I use the moodle logs to drive the visualizations. The moodle logs contain timestamps of the user interactions. The problem is that there are only events for when something starts and not when it finishes. So I know when the user opened the forum for example, but I do not know exactly when the user closed it. Most of the times this is not a problem, when the user does another action in a short period after the last action. So I use a threshold to determine the duration for longer periods between 2 actions.
I hope this makes things a bit more clear...
Tracking what happens after the students leave the online learning management system is of course very hard to do automatically (unless we could mount some kind of smart device on their heads that detect what they are doing and looking at ;-). We did an experiment for an evaluation of the tool, where we asked students to tweet what they were doing for the course and for how long. We used twitter because students might like it more, but most of them stopped tweeting very soon.
Next to that, I use the moodle logs to drive the visualizations. The moodle logs contain timestamps of the user interactions. The problem is that there are only events for when something starts and not when it finishes. So I know when the user opened the forum for example, but I do not know exactly when the user closed it. Most of the times this is not a problem, when the user does another action in a short period after the last action. So I use a threshold to determine the duration for longer periods between 2 actions.
I hope this makes things a bit more clear...
It's been interesting exploring the range of analytical tools and reading the key blog entries George shared lately. Like some I am learning about this movement and don't feel equipped to share insights or recap my understanding at this point. Being the analytical type, I hold back until I completely understand and have read all, which might take a few years! Smile. As a result, I am grateful for those who are sharing their thoughts and resources.
One thing that is concerning me about analysis of this magnitude is the validity and authenticity of the data. I am impressed with the volume of data the tools can manage but wonder about the compatibility, origin and validity of it.
I don't have the time to review the data presented so far but do plan to attend the LAK11 conference (I live in Calgary) where maybe I can explore this concern deeper.
Anyone else feel the same?
Hi Kelly. I'm nowhere near Calgary but would be delighted if you could mine a bit more on the validity question. LMSs tend to give huge amounts of data - like number of posts viewed by a particular student - but when you know full well that there aren't that many posts in the whole course and they view that number (or thereabouts) every week, something isn't right. Time on task is another big problem. Staying in a course to read material is just not me (or many others) so I download 'stuff' as fast as I can and then sort it out later. That distorts not just 'time on task' but also the through-clicks and work done there. I have no answers to getting it right - just observations :(
Gillian, thanks for your comments. I think we need to determine what learning is. Yes, an old question, but if we don't know what we are looking for we can't measure it. To me learning is not about the number of posts but what students do with it. I think we need to start with the end in mind - assessments - and rich assessments such as reflections, creations, arguments, etc. Going backwards, we can research the tools used by students to arrive at their understanding and work. If situated in the cloud and accessible, we can review the types of interactions, materials explored, and tools used.
It seems to me most people in here are saying the same thing. Student action in a LMS is only a part of their learning. With a diverse and wide spread networked world, we need to reconfigure how we capture and analyze data about student learning.
I think Tony makes an important point here and I think this is still the biggest challenge for learning analytics. We're not really measuring things that are most critical. I do think that the visualizations are interesting and thought-provoking though.
Hi everybody!
I set up a short evaluation survey for my tool. If you used the tool during the course. It would be great if you would have some time to fill out the survey.
You can find it here: http://bit.ly/laksurv
Thank you very much!!!
I set up a short evaluation survey for my tool. If you used the tool during the course. It would be great if you would have some time to fill out the survey.
You can find it here: http://bit.ly/laksurv
Thank you very much!!!