Critiques of learning analytics?

Re: Critiques of learning analytics?

by Marielle Lange -
Number of replies: 0
It may help to define terms before engaging in critique.

In the business field, "Analytics often involves studying past historical data to research potential trends, to analyze the effects of certain decisions or events, or to evaluate the performance of a given tool or scenario. The goal of analytics is to improve the business by gaining knowledge which can be used to make improvements or changes." (source: http://www.businessdictionary.com/definition/analytics.html)

In the web-design field, "Measurement tool that analyzes user behavior based on logs of activity on a website. Includes information such as entry and exit pages, most popular pages, paths through the site, links from other sites, and search terms." (source: http://nform.com/tradingcards/web-analytics)

What is not too helpful is that you mix two different notions in your in your post. The ones of evaluation and analytics. Your list of concerns have to do with "analytics" used as ONLY and exclusive means of evaluation. Which it rarely is.

It is important to be aware that analytics should be part of a strategy that should be carefully managed. Project management is a well established practice in many fields. It is typically characterized by these different stages:
* analysis-phase – discovery
* design-phase – problem definition
* development-phase – write solution
* implementation-phase - deliver and manage → delivery
* evaluation-phase – evaluate how well you did
* revision – made as necessary.

Instructional designers may have come across the ADDIE acronym that captures these different steps, though in simplified manner.

Analytics are typically used during the evaluation stage. Evaluation can be defined as "The process of gathering information in order to make good decisions. It is broader than testing, and includes both subjective (opinion) input and objective (fact) input. Evaluation can take many forms including memorization tests, portfolio assessment, and self-reflection." (source: http://www.nwlink.com/%7Edonclark/hrd/glossary/e.html)

A huge range of tools are available to perform evaluations. One of them is analytics. It is just one tool in the toolkit. You pick up the analytics tools whenever you think that it is more adequate than other tools you have in the kit. Whether you pick it or not depends on your goals and strategy.

However, and most importantly, you don't select that tool very late in the process. Both your evaluation and assessment strategy should be defined early, at the design stage. Evaluation will be useless unless you have clearly defined & tangible objectives.

This is unambiguously recognized in the field of web-design, where analytics is extensively used:

"Site Strategy: Defining your own goals for the site can be surprisingly tricky. Arriving at a common understanding of the site's purpose for your organization, how you'll prioritize the site's various goals, and the means by which you'll measure the site's success are all matters of site strategy."
(source: http://www.jjg.net/ia/files/pillars.pdf)