Week 1: Introduction to Learning and Knowledge Analytics

What analytics would help you with this course?

What analytics would help you with this course?

by Richard Olsen -
Number of replies: 43
Would having access to any analytics help you as a learner in this course? Feel free to dream about tools that don't exist.
In reply to Richard Olsen

Re: What analytics would help you with this course?

by Bradley Shoebottom -
A tool that autmatically reminded me to make posts instead of George (thanks George!) This smae analytics could measure my particilation and with some more complex semantics, how well I replied.
In reply to Richard Olsen

Re: What analytics would help you with this course?

by Mark Gbur -
Having access to analytics would be a huge benefit. Especially in a course with so many voices and so much content.

I think a feedback poll for each reading assignment could help gauge the helpfulness of content for readers of various experience. Additionally, it can help the facilitators weed out less useful content and scaffold weak areas.

The feedback mechanism would be simple: http://thestaticfrost.posterous.com/injecting-a-feedback-mechanism-into-the-learn

Cheers,
-Mark
In reply to Mark Gbur

Re: What analytics would help you with this course?

by Asif Devji -
Nice -- could add to the poll how many people actually read the article (fully).

In terms of analytics, one thing I'd like to know is how many of my total posts got responses, and how many responses they got.

Then I'd wanna mine down into which posts got responses, what kinds of responses they were, and whether they furthered the overall discussion.

In reply to Asif Devji

Re: What analytics would help you with this course?

by Kami Mutan -
"Then I'd wanna mine down into which posts got responses, what kinds of responses they were, and whether they furthered the overall discussion"

I'd like this too. I would add some form of tagging users.
Don't get me wrong, I do not mean to make a statistical or automatic profile of users, but you could add custom tags to each user ... Of course, to identify users whom you connect to the system you described is needed first.

Me gustaría esto también. Yo le agregaría alguna forma de taggear a los usuarios.
Que nos se confundan, no me refiero a realizar un perfil estadístico o automático, sino que uno pudiese agregar tags personalizados a cada usuario... Por supuesto que para identificar a los usuarios con los que más te conectas necesitarías primero el sistema que describiste.
In reply to Asif Devji

Re: What analytics would help you with this course?

by Angela van Barneveld -
@Asif - And then what would you do with that information? How would it influence any decision-making?
In reply to Angela van Barneveld

Re: What analytics would help you with this course?

by Asif Devji -
Think I'd use the info in a fuzzy non-systematic way to get a sense of things like:

- how valid an idea it was I put forward in a post, based on (critical) feedback
- how relevant the post was to the conversation, based on # of responses
- how much of what I was thinking resonated with others
- what kinds of communications work best in a forum context (ie. is provocation more effective than diplomacy in generating responses)
- who else in the forum got what I was saying and took it further
- who could offer thoughtful alternative perspectives

In terms of tangible influence, I think I'd see a discernible (though I can't say how I'd measure it) change in my communication style in discussion forums.
In reply to Asif Devji

Re: What analytics would help you with this course?

by Gillian Palmer -
Hmmm. Reading the post 'fully'. There's a challenge. Skimming to the bottom of the article or reading the whole thing? As for eyes and brain connecting and making understanding.... That's a massive area for analytics but not one to ignore.
Gillian
In reply to Richard Olsen

Re: What analytics would help you with this course?

by Claudia Guerrero -
Cómo educador para ejemplificar las categorías de clasificación del uso del concepto, por ejemplo en estadística descritiva si se está ejemplificando la graficación la búsqueda de estadística y de tipos de gráfica resulta la siguiente clasificación de uso.

Las preguntas propuestas son:

What would you like to show?

    Composition: the components that pieces make to a whole.
    Distribution: How one variable's values are distributed.
    Comparison: How do values change over time or in relation to one another?
    Relationship: How one or more variables changes as a direct function of another variable.
    Not sure yet.

De ahí se pueden identificar diferentes casos de uso.


As an educator for the classification categories exemplify the use of the concept. Descriptive statistics for example if you are exemplifying the search for statistical graphing and chart types is the following use classification.

The proposed questions are:

What Would You like to show?

     Composition: That the components to make a whole pieces.
     Distribution: How one variable's values Are Distributed.
     Comparison: How do values change over time or in relation to One Another?
     Relationship: How Changes one or more variables as a direct function of Another variable.
     Not sure yet.

From there you can identify different use cases.
In reply to Richard Olsen

Re: What analytics would help you with this course?

by Christopher Teplovs -
I'd like to have an automatically-generated network graph or map that showed me various representations of all the posts in the moodle forums. I'd like to be able to arbitrarily reorganize the postings according to various criteria: authorship, chronology, theme, keywords, latent semantic content, relatedness, and so on. I'd like to be able to overlay additional information, like whether I had read the postings, atop the two- or three-dimensional display of the organizational variables. I'd like to be able to identify criteria that make certain notes or postings "rise up" and then be able to manipulate those postings.

You did say to dream ;)
In reply to Richard Olsen

Re: What analytics would help you with this course?

by Adam Weisblatt -
A tag cloud would help. Does Netvibes have one? The reason is that if you can't read everything generated by a MOOC you need some way of being able to focus on content that interests you.
In reply to Adam Weisblatt

Re: What analytics would help you with this course?

by Kami Mutan -
Yes. I am definitely missing the option of being able to custom tag posts and, as I already said, users :)

I also miss the "I like" buttom that Facebook has. I guess I'm too accustom to "liking" posts instead of actually replying to them. But it's been a good exercise. Facebook has made me a little lazy :O
In reply to Kami Mutan

Re: What analytics would help you with this course?

by Sylvia Currie -

Kami Mutan wrote,

I am definitely missing the option of being able to custom tag posts

This is possible here in the forum using Marginalia. This quick video shows how to make basic annotations (this includes tags) to posts. A second video shows more advanced annotation features. What I like most about Marginalia is that it is so flexible, and wide open for different uses.

I've been thinking about the best way to annotate forum posts in this course in order to generate a summary documents. So far I've only tagged a few posts containing suggestions or issues related to participation in the course ('user experience' and 'course considerations'). So I see one category as being course administration and facilitation. A second category would be around emerging themes and recurring questions. Another possibly the suggested tools.... endless!


In reply to Sylvia Currie

Re: What analytics would help you with this course?

by Mary Rearick -
Kami,
Thanks for sharing the link to the quick video that shows how to make basic annotations. The presenter was suggesting the annotations could be used to take notes. The annotations might be a useful way of coding comments. The annotations might contribute to knowledge building of the group if they are made public. An individual might use them to track her thinking, whilst exploring the resources.
In reply to Adam Weisblatt

Re: What analytics would help you with this course?

by Apostolos Koutropoulos -
You do raise a good point!
This being my first MOOC, I am somewhat surprised by the volume of messages generated. I created a filter in my inbox and direct messages to a LAK folder; but 2 days in I find that I am mostly skimming, unless of course I want to respond to something. I've also found that even though I might nit want to respond to someone's post, what they write is so compeling that I'd like to connect with them on LinkedIn or Academia.edu for future discussions.
In reply to Apostolos Koutropoulos

Re: What analytics would help you with this course?

by Kami Mutan -
We are connected through Linkedin now. Thanks for the invitation, but I never use that.
My blog is http://kamiresearching.blogspot.com (it's in Spanish)
In reply to Kami Mutan

Re: What analytics would help you with this course?

by Asif Devji -
I do use LinkedIn -- in fact I learned about LAK11 via my blog feed in there & then Tweeted it out via LinkedIn to my network once I learned about it.

Was thinking that LinkedIn could provide one model of a PLE-like space into which one could pull (learning) content, contacts, discussions, feeds, etc -- all ripe for analytics tools.
In reply to Richard Olsen

Re: What analytics would help you with this course?

by Chris Luchs -
Yes, I think having access to analytics in this course would be helpful. I'd like to see things like:

Inside of Class analytics:
  • Number of views for my posts
  • Top ten most viewed posts (other than the instructors :D)
  • Country and City Location of those who view and comment on my posts (to view geographic closeness and identify cluster areas)
  • Length of reader stay on my post
  • Maybe something like the adaptive database that Amazon or Barnes & Nobles online, where you get an automatic prompt like, "People who liked this post also liked these".
  • A way to see whether any link I posted was clicked on and visited.
  • Some form of performance assessment/ credibility ranking as far as quality of posts
  • At a glance graphic of participant active useage based on MST :D So I can see when most of us are on
  • A buzz indicator (something that shows what the most popular instructor and participant posts are).
  • Class statistics: average length of stay in site, which components were viewed, number of posts, etc.

In reply to Chris Luchs

Re: What analytics would help you with this course?

by Kami Mutan -
Number of views for my posts
AGREE

Top ten most viewed posts (other than the instructors :D)
AGREE

Country and City Location of those who view and comment on my posts (to view geographic closeness and identify cluster areas)
AGREE

Length of reader stay on my post
AGREE

Maybe something like the adaptive database that Amazon or Barnes & Nobles online, where you get an automatic prompt like, "People who liked this post also liked these".
:) GREAT IDEA!!

A way to see whether any link I posted was clicked on and visited.
AGREE

Some form of performance assessment/credibility ranking as far as quality of posts

DISAGREE, I think that's too subjective. What is important for me is not important for everyone here. Maybe the statistics of the user will suffice.



At a glance graphic of participant active usage based on MST :D So I can see when most of us are on
WHAT IS MST?

A buzz indicator (something that shows what the most popular instructor and participant posts are).
AGREE, just to go against the current and visit the least popular posts XD

Class statistics: average length of stay in site, which components were viewed, number of posts, etc.
It would be great if that statistics could be linked to our personal sites, blogs, etc. I mean to have an exact idea of how many people signed and visited other people's blogs, made facebook friends, etc. A way to measure the online connection generated by he course :)

Excuse my English
In reply to Kami Mutan

Re: What analytics would help you with this course?

by Apostolos Koutropoulos -
Id like to see dig/bury type buttons for each post (not just like) - admittedly this might not give you info a to WHY it was promoted or demoted but it is another analytic at hand.

I would also like to see some sort of "priority inbox" like google that takes data from forum posts and prioritizes then for the learner
In reply to Kami Mutan

Re: What analytics would help you with this course?

by Chris Luchs -
No worries, Kami. MST is Mountain Standard Time for those of us in the US. Sorry about that. Basically, I would like the active times to be translated in my local time.

While I agree with your comment about the subjectiveness for the performance measure. I was thinking about some metric that would appeal to the competitive nature of everyone and measure their contribution to the course. You could have something as simple as a "like" button common in social media or a start system that readers could assign as they felt like it. :D

One of the things I've noticed about the MOOC setup is the feeling of being lost with so many other voices. As a participant, I'd like to see who is driving the discussion and be able to identify them quickly and easily.

As an instructor, I'd love to see confirmation of who my class influencers are and the peer groups that get established within a course.
In reply to Chris Luchs

Re: What analytics would help you with this course?

by Kami Mutan -
Thanks for the clarification.

I understand both of your points regarding the 'metric' for quality/credibility

And if we can personalize it... Who's more credible for me... like the Hunch results were supposed to be I'd vote for it too.

The same for the instructor... different students will be influential in different ways so different criteria will turn different leaders.

In reply to Kami Mutan

Re: What analytics would help you with this course?

by Mary Rearick -
Kami,

I liked the way you framed the conversation.


Like you, I believe that it would be difficult to develop a performance assessment or credibility rating for determining the quality of a post. People who are enrolled in the course are diverse and we have very different reasons for studying analytics and for engaging in a MOOC.

I am not sure how to track class statistics...

Class statistics: Any effort to track class statistics would be 'partial'. Perhaps it would be possible to see how long someone took to type a response, but what does that really mean? A course and learning management system can track the time participants spend on the site. Semester after semester my students personalize their learning. Final papers, presentations and group projects reveal little correlation between time spent in the shell and actual learning. Since the LAK11 course is distributed, thinking, reading, etc. will occur both on and offline and in multiple sites. Wouldn't it be nearly impossible to actually track and individual's activity?

In reply to Chris Luchs

Re: What analytics would help you with this course?

by Sylvia Currie -
Ranking forum posts could serve as a step toward "people who liked this post also liked this" type of feedback.

It might be interesting to try out different rating options in the forums in this course. As an example, in the open for anything forum I added the Separate and Connected ways of knowing scale. You can use the drop-down menu for each forum post to flag that contribution. (I hope I've flicked all the right switches so everyone can use this tool -- Let me know!)

This is just an example. I can set up scales here in Moodle with whatever words we'd like to use. We can also implement a rating system (one star, two stars, etc) although I'm not sure if that kind of feedback is as valuable as using language. For example, maybe a rating that indicate how individuals relate to the post would be useful:

1. need clarification
2. helpful now
3. advanced, need to revisit later

I'm making this up as I go along so I hope it's making sense!

What are your ideas for scales to use in this course?

In reply to Richard Olsen

Re: What analytics would help you with this course?

by Fred Haas -
There are two features in Quora that are kind of soft analytic elements, but I think would be great in discussion forums, especially like the ones in a MOOC, that generate so many posts,

The voting up or down and summarizing features, that can be created to appear as a kind of intro to the thread, would also be really helpful for people to be able to scan the forums with greater prioritization and filtering. The voting is a little like the FB "I Like" but tracks the voters and raises the level of the post answers in the order visible. The summary feature is a great way to kind of keep an ongoing editable abstract of the posts.

Those two analytic features might serve as great ways to reduce the kind of MOOC fatigue and overwhelmed feeling that so many people can suffer from after about the third or fourth week. There might be some added complexity with the threaded nature of discussion forums, but I know I would appreciate something like that a whole lot in any of the MOOCs I have experienced.
In reply to Fred Haas

Re: What analytics would help you with this course?

by Barbara Dieu -

Fred Haas wrote,

The voting up or down and summarizing features, that can be created to appear as a kind of intro to the thread, would also be really helpful for people to be able to scan the forums with greater prioritization and filtering. The voting is a little like the FB "I Like" but tracks the voters and raises the level of the post answers in the order visible.

While I agree with you that this may help, how does the system deal with people who play it to their advantage?
In reply to Barbara Dieu

Re: What analytics would help you with this course?

by Jeanette Delgado -

The communication and discussion show us how the platforms use for

E- learning, need to change, we live in word where we learn to just (like) something, (vote) and follow the post with the most comments. The E-learning platforms should evolve to.  I am not an analytics fan but I'm learning so much, with all your points of view. I may twitter this :-)

Jeanette

In reply to Jeanette Delgado

Re: What analytics would help you with this course?

by Apostolos Koutropoulos -
The one thing that I would be cautious about is the group think that might develop from only following popular threads. It's important to not just put the blinders on and ignore potentially insightful threads and comments that aren't popular :-)
In reply to Apostolos Koutropoulos

Re: What analytics would help you with this course?

by Sylvia Currie -

Apostolos Koutropoulos wrote,

The one thing that I would be cautious about is the group think that might develop from only following popular threads.

This statement really bounced out at me. I wonder about this with applications like twitter which show number of followers. A decision to follow might be based on how popular that individual appears to be, rather than the quality of what they heave to offer, or, more importantly, now connected you might be to their ideas so that you can build on them.

I spend a lot of time communicating online and meet (and seek out) new people all the time. I notice that some of the most thought-provoking dialogue can be with individuals who are finding their way in a new domain, asking questions, challenging ideas that others skip over because they're so used to hearing them.

I do see value in somehow showcasing individuals who contribute to and advance collaborative dialogue. But as Apostolos suggests it might have a negative impact if we begin to use popularity ratings as an indicator that those are the conversations we should be paying attention to.

In reply to Sylvia Currie

Re: What analytics would help you with this course?

by Fred Haas -
While I am no Quora expert, by any means and have only been tinkering with it for a couple of weeks, the voting up or down is connected to answers to questions not necessarily individuals. This is not to say that who composes the answer is a non factor in the voting. Yet, what is rather interesting is this space seems to have a stronger sense of culture than a lot of other online "communities." Of course that is not likely to last long, but for now the features I mentioned are pretty interesting and unlike most of what is available in a lot of other online spaces.
In reply to Richard Olsen

Re: What analytics would help you with this course?

by merce galan -
Hello,
So I'm reading, I should probably change the habit of reading, as I read every post from my email and not from the forum. If I read the email I will not count in the statistics but I suppose that "my life will be more visible" if read in the forum, right?
Greetings
mercè
In reply to merce galan

Re: What analytics would help you with this course?

by George Siemens -
Mercè,
Good point - i.e. reading online vs. reading in the forum. So, yes, your life is more visible if you read it on the moodle forum. I don't think moodle offers a widget yet that allows email tracking the way that services like Constant Contact do.

As others have stated in this discussion - analytics are only as good as our data. It's a good reminder that all data related to learning analytics is somewhat incomplete - especially if confined to learning that occurs in a classroom or an LMS. We need multi-silo data access. But, that then raises Total Information Awareness concerns: http://en.wikipedia.org/wiki/Information_Awareness_Office
In reply to George Siemens

Re: What analytics would help you with this course?

by Wendy Kraglund-Gauthier -

Agreed. This is also related to the "lurkers" (I prefer the label "auditors") of courses, listservs, presentations. They are there, participating in their own way, yet not captured adequately or even accurately in the data.

Wendy

In reply to Richard Olsen

Re: What analytics would help you with this course?

by Angela van Barneveld -

Is there a possibility that we are confusing data/information, data management, and metrics with analytics?

In reply to Richard Olsen

Re: What analytics would help you with this course?

by Dianne Rees -
I guess I'm less interested in posts that are the most commented on because I don't think this necessarily correlates with how interesting/valuable the question is to me as an individual learner (though popular questions/answers do give you a broader range of information to explore). The voting up and down allowed by Quora does allow you to register your impressions of how well a post answers a specific question (and that's a valuable input) although we don't use Moodle in quite that way and I wonder if voting up or down would discourage some of the more free range discussions going on.

Personally, I would be interested in tagging trends and how "connected" a bit of information is (does it include links to outside sources other networks?) so that I could also expand my range of connections. As a way of considering how much this course is benefiting me and whether I am using resources effectively, I'd want to understand where the buckets of information are and how I'm actually tapping into those buckets to achieve something...so I'd want to tie the analytics to my own learning contract (after having identified goals I want to accomplish).
In reply to Richard Olsen

Re: What analytics would help you with this course?

by Mary McEwen -
Oh! I would love A VUE (or similar) map of more specific interests in LAK (of participants) e.g. corporate sector, LMS/CMS, Institutional, higher ed, K-12, adaptive guidance, etc.

The interests should/could be mapped with their semantic relationships, and with the size of node representing numbers of participants with that specific interest. Nodes could be zoomed in and out. (Zooming in to the most detailed level would reveal usernames of participants listing that interest, and if we're dreaming - hotlinks to their presence on the web - blogs, twitter, fb, linkedIn, etc.)

This would be helpful for a number of reasons:
1) finding a subgroup with similar interests/emphasis in this MOOC
2) viewing the breadth and proportions of interests represented by LAK participants
3) It would be most enlightening to see how (or if?) this interest/emphasis map evolved over time. An animation of such would be very insightful, especially if somehow it could be tracked before and after LAK11 in Banff, to measure impact the course, and then the conference had on interests.
In reply to Mary McEwen

Re: What analytics would help you with this course?

by Nicola Avery -
Hi, this is a bit late and most covered above but have started to try and put together a list of analytics for general online collaboration and thought it was kind of relevant.
Not sure if I think it would be useful for either a learner/user or facilitator or administrator at this point in terms of asking questions:

Collaboration Analytics
Technologies Types Other considerations
Server logs
Tracking code + widgets e.g.
Google Analytics
Statcounter
web behaviour
  • views
  • clicks / hovers - nos & comparison of areas (e.g. heatmaps)
  • error views
  • paths - comparison
  • search/keywords/adwords
  • referrals, clickthroughs, countries, IP addresses, browsers, OS etc
  • entries/exits
  • usage per time/date incl traffic rates, session length, visitor returns & bounce rates
  • downloads & abandoned rates
  • other ‘transactions’ & abandoned rates
What technologies are they using
Where they come from
What they do when they are here
Where do they go next
Why do they return to something
Why do they leave something
Do they provide any info about what they are interested in
Do they provide any info about what they are looking for
Do they provide any info about who else they might be interacting with (see below)
What problems might they be having
Why they start something and stop halfway through
Use of translation
Any other accessibility issues
Twitter, FB, Google Friend Connect,
Additional analytic apps for the above
Disqus, IntenseDebate, Backtype
social behaviour
  • language used in post/wiki page/tweet etc
  • views, shares
  • tags
  • likes, dislikes, rates, votes
  • comments / edits (incl language)
  • comments / edits & abandoned
  • changes over time (e.g. more or less views, comments, edits)
  • who connected to (via their ID if provided e.g Twitter, ...Connect, Disqus etc)
  • who else is online at same time
  • use of notifications of updated content
  • links to related / similar content / increased connections / relationships density
Needs to be linked to referrals i.e.
Did they come via a recommendation from someone
Do they reply to a particular person/people
Do they not reply to particular person/people
Do they increase / decrease their number of contributions over time
Where are exit points in a discussion and who leaves
Who else are they already connected to / social tools already using
Are likes/dislikes/votes/rates about particular topics / people
Are there any signs of people being influenced by others existing likes/votes etc
Changes in style, language, tone in comments over a discussion, over period of time
What is tagged and where


General
  • Who would benefit from viewing this information?
  • Who can already view this information?
  • What is a successful experience?
  • Are people are bored/satisfied/ecstatic about their experience?


Some (partially) unknowns
  • How much is private messaging being used?
  • Why anonymous users are choosing to be anonymous?



In reply to Nicola Avery

Re: What analytics would help you with this course?

by Francisco Reis -
I find that table interesting and I would take them to what is my field of present work.
(Sorry for always insisting on developing better tools).

Let's take these three items of the social behaviour (see attached image):
  • tags (I call topics)
  • likes, dislikes, rates, votes (I call UDRIVE)
  • who else is online at same time (I propose virtual presence)
As Fora seams to be the most used paradigm in our LAK11 MOOC, why
can't we have the three items above in each Forum (perhaps only one Forum is then needed).

TAGS
When adding and entry to a Forum it would be very easy to tag it as belonging to one or more topics, for instance EDM, Privacy, K12 or even Spanish.
Any user should add a topic if he feels the need (or the Forum manager can set a predefined set of topics if he wishes to).

NOTE 1: The operation to choose to which topics an Entry belong must be easy like using checkboxes!
NOTE 2: The filtering of entries by topics should be available!
NOTE 3: If you are only seeing entries of certain topics it should be seamless to make our reply also belong to those topics (enabling, of course, to add or remove any of those topics)!

LIKES, DISLIKES
Any Forum Entry can be marked by readers for Instance as UDRIVE (Uread, Deleted, Read, Interesting, Very Interesting, Exceptional).
NOTE 4: FIltering by the marks we gave should also be possible.
NOTE 5: If all this information, likes and topics, can be repesented in various forms in table and charts then we could have insight on what is going on.

WHO ELSE IS ONLINE AT THE SAME TIME

Virtual Worlds are very good at it. If the Forum has a 3D representation then we should see other avatars that are close to it.

All the above is already integrated in my www.umniverse.com system and soon you will be able to test it if you want (I join an image to open up your appetite).

My main message is: tools should get better! Integrated, easier to use and with instant analytics (
which for common users I call them Statistics ;-) ).



Attachment SuperVirtualForum6.jpg
In reply to Francisco Reis

Re: What analytics would help you with this course?

by Nicola Avery -
Wow Franciso the umniverse looks very interesting. What would also be useful where you suggest about the tags is to do along the lines that Many Eyes do with TagClouds, which is when you hover over the tag, the related entries are previewed in a short list in context of the sentence of where they are written.

There should also be an option to view as maybe a cloud or maybe something else - a tagcloud of related tags to that tag which are clickable or have previews as above.
In reply to Nicola Avery

Re: What analytics would help you with this course?

by Francisco Reis -
Thanks a lot, Nicola, your wow means the world to me for motivation is the basis of long term work!

I'll take a look at other tag systems, namely those that you mentioned, and improve Umniverse Forum object accordingly.

Here I join another view of current graphics/analytical possibilities hoping it meets some of your ideas.

Best Regards

   Francisco
Attachment MooWheel_in_Umniverse.jpg
In reply to Francisco Reis

Re: What analytics would help you with this course?

by Nicola Avery -
Hi Francisco, sorry for the extremely late reply. It looks very interesting. So the tags could expand if linked to similar / related tags potentially in your visualization which might provide some possibilities about why users have tagged something with a chosen tag.

Wishing you the very best with umniverse development,
Nicola :)
In reply to Nicola Avery

Re: What analytics would help you with this course?

by Francisco Reis -
Thanks a lot for your comments and interest.

Umniverse development will continue and it gained a lot from our LAK11 experience.

Let's see how learning can be improved with Virtual Worlds and analytics.

All the best for you,

Francisco