Week 1: Introduction to Learning and Knowledge Analytics

LAK11 Moodle Forum by the Numbers

LAK11 Moodle Forum by the Numbers

by Tanya Elias -
Number of replies: 9

I am reposting this info here because apparently the pics weren't visible in its previous location. (Have I mentioned that I'm technologically challenged?)

John Fritz did an excellent job yesterday starting off the conversation about learning analytics. One of the things he talked about was the need to better use existing LMS data and to share that data with others.

His talk prompted me to do some digging into the data currently available from the LAK11 Moodle site and to re-present it back to the larger group. I found out that there are:

233 members of the Moodle course representing 42 countries. The most heavily represented countries are the US, Canada and Australia. There are also two participants from Iran and 1 from Guyana.

121 of the 233 registered members have visited the Week 1 Forums. The most views by a single participant in that section? 54. It also told me that there were more views by guests than logged in members. 53 people have posted to the Week 1 forum in the last day, and only 19 registered members didn't post at least once to the introductions forum.

The overall activity thus far in the course looks like this:

week 1 overal activity

Looks impressive doesn't it? It looks to me like there were over 3200 views on the site, and the number of views is continuing to climb. That's good right? But if we take a look at the next one, the number of posts, it is starting to fall...Is that bad? Are all types of activity created equal or is posting a "better" indicator of involvement?

week 1 posts

The Moodle stats also showed me a pretty graph showing that I was really active on the site between 8 and 9 pm (while I was compiling these numbers).

It also told me that Dave Cormier has yet to view the Week 1 forums as a logged in user.

week 1 moderator activity

But anyone who checks out Dave's blog will quickly see that he has completed both the readings and the acitvities for week 1. He either accessed the information another way, or as a Guest. In this case, although the numbers make it appear that I've been more active, much of my time has been spent playing around while Dave has been interacting with both the content and other participants via his blog and email.

Although interesting, numbers (data) have their limits. Figuring out how to interpret them and use them effectively, that's where the fun stuff really begins.

(Edited by Sylvia Currie - original submission Friday, 14 January 2011, 01:16 PM -- added images)

In reply to Tanya Elias

Learning Analytics on Open Systems?

by John Fritz -
Hi Tanya,

I just wanted to comment on your following statement:

"Although interesting, numbers (data) have their limits. Figuring out how to interpret them and use them effectively, that's where the fun stuff really begins."

I was at the Educause Mid-Atlantic Regional Conference (MARC) today and heard a fascinating closing keynote talk by Cole Camplese of Penn State University.

In terms of activity being an indicator of student engagement, he reported similar findings to what we have done: but he's using open systems such as Moveable Type blogs, wikis and even Google Docs. The trick? They have integrated these systems with their single signon kerberos ID. He showed a graph that basically confirmed students who earn higher grades are more active in these systems.

I said this to Cole in the Q & A, but I've NEVER seen anything like this in what I would call more "open" systems, let alone such a wide variety of them. Combined with the cloud-based, online gradebook that Jon Mott of BYU has proposed, this could be a path to provisioning flexible, decentralized systems but with an identity management strategy that correlates student activity and grades contained in institutional data warehouses. True, correlation is not causation, but without an analytics infrastructure that accommodates the trend toward open, personal learning environments, we may be taking a step backward in terms of evidence-based prediction OR intervention.

BTW: Cole will be presenting at the Educause Learning Initiative (ELI) annual meeting in Washington, DC (Feb. 14-16). So will I. :-)


In reply to John Fritz

Re: Learning Analytics on Open Systems?

by Cris Crissman -
That's really encouraging, John, to hear of Camplese's analytics using the open Web. Though "single signon kerberos ID" sounds like Klingon.

I wonder what kind of analytic tools there are for looking at social presence, particularly real-time interaction.

Thanks for any tips --

In reply to Tanya Elias

Re: LAK11 Moodle Forum by the Numbers

by Terry Colton -
Says files are not available
In reply to Terry Colton

Re: LAK11 Moodle Forum by the Numbers

by Sylvia Currie -
Thanks Terry. We'll fix it! Tanya had some problems making those files public on an ELGG site. Tanya, if send the images on over to me via email and I'll upload them here.
In reply to Tanya Elias

Re: LAK11 Moodle Forum by the Numbers

by ines cambiasso -
Thank you, Tanya. A great overview but as you say we may be engaged in activities that are not visible for Moodle, like reading all posts in your own mail.
In reply to Tanya Elias

Re: LAK11 Moodle Forum by the Numbers

by Tony Ratcliffe -

Hi Tanya,

I am still not seeing the images. I tried in both IE8 and Firefox 3.6.13.

Responding to the greater number of guests than logged-in users reading, it is so easy to not log in. The system makes us relog each time, but we are not prompted to do so unless we want to read a profile, reply, etc. We can easily read the posts without a login. That very issue is obviously affecting the stats.


In reply to Tony Ratcliffe

Re: LAK11 Moodle Forum by the Numbers

by Sylvia Currie -
I meant to pop back in here earlier to say that Tanya's images are showing up in her week 1 Moodle Forum by the Numbers post.

Tony, your observation about being able to read posts without logging in is a topic that comes up often in my work in online communities. SCoPE, where this course is hosted, is intentionally designed to be wide open to allow people to participate on the periphery. We want readers to see the value before creating an account/joining in a discussion. If we took the opposite approach the member numbers would be huge, but many of those accounts would be associated with individuals who simply wondered what was behind the closed door. It's a bit of a dilemma, because large numbers can sometimes impress the right people mixed And, as you said, at the course level it's difficult to get a handle on how many are "participating", much less who they are. Technically I could be reading all Moodle forum posts through email or RSS. There's always a lot of explaining to do each time I present data about SCoPE activities!
In reply to Tanya Elias

Re: LAK11 Moodle Forum by the Numbers

by Marielle Lange -

There is a range of metrics that you can use to measure social participation. For instance Top 7 Deadly Metrics for Social Games

Or for website engagement. For instance Unleashing the Power of Website Analytics

For more commercial analytics (mobile analytics to measure the return of advertising banners), conversion rate is calculated by total number of visitors completing a target actions.

As with any research and data analysis, you need to know what your goal is before you start looking at your data ;-)

In reply to Tanya Elias

Re: LAK11 Moodle Forum by the Numbers

by Marielle Lange -

Beware about using All Roles. If this includes Guest and includes the landing page, you could count in Robots (search engine crawlers) instead of actual users.

Filtering for Spiders and Robots

That's another reason why starting with goals and a predefined strategy for data analysis matters. It is important to ensure that the numbers that you see really are what you think they are.