Today at the very interesting talk of Ryan Baker a question arise about the differences (and similarities) between Educational Data Mining, Learning and Knowledge Analytics and the traditional field of Educational Research. I think that this question deserves further exploration in this course.
The definitions that those terms have in Wikipedia are very similar:
Educational Research:
Educational research refers to a variety of methods in which individuals evaluate different aspects of education including but not limited to: “student learning, teaching methods, teacher training, and classroom dynamics”.
Educational Data Mining:
Educational Data Mining (called EDM) is an emerging discipline, concerned with developing methods for exploring the unique types of data that come from educational settings, and using those methods to better understand students, and the settings which they learn in.
Learning (and Knowledge) Analytics:
Learning analytics is the use of intelligent data, learner-produced data, and analysis models to discover information and social connections for predicting and advising people's learning.
Is there a difference? Do you think that EDM and LAK are just an evolution of Educational Research, but with better tools and data, or there is something radically different and unique in the new approaches?
Do you think that EDM and LAK are synonymous or there is a meaningful difference between the two fields? Should we merge or we should keep them separated?
Let's discuss.
I don't know what Learning and Knowledge Analytics is, which is why I'm doing the course. Hopefully over the period of the course, things will start to make more sense.
Some ideas, about what I think about these terms........
Looking at the threads in this course so far. What makes a good thread? The number of responses? The quality of the conversation? Evidence of new knowledge being created? Diversity of replies?
How can I start a "good thread"
I could use analytics to see what was the best time of day to start a thread (does it matter?), look at the title (is a question more likely to lead to enagement?), look at the author (will a course leader get more engagement, maybe I could get George to start the thread instead?). Does the time in which takes for questions to be answered effect the "success" of the thread? If it does, I could monitor the thread closely to maximize the success of my thread.
I could also use data mining techniques, I could look at Introductory thread to get an idea of the backgrounds and interests of the course participants and identify possible threads. I could also mine data more widely looking at the feeds (twitter, blog, delicious, ...) that course participants and also by monitoring the LAK11 tag on those and other services.
Alternatively, I could use data mining techniques to look at similar courses to learn about what makes a "good thread"?
Having identified the qualities of a "good thread" using analytics and data mining I would then start my thread, and use analytics (number of replies, content of the replies, number of unique replies) to see whether my predictions were accurate.
Oh, and no I haven't done this :)
Some ideas, about what I think about these terms........
Looking at the threads in this course so far. What makes a good thread? The number of responses? The quality of the conversation? Evidence of new knowledge being created? Diversity of replies?
How can I start a "good thread"
I could use analytics to see what was the best time of day to start a thread (does it matter?), look at the title (is a question more likely to lead to enagement?), look at the author (will a course leader get more engagement, maybe I could get George to start the thread instead?). Does the time in which takes for questions to be answered effect the "success" of the thread? If it does, I could monitor the thread closely to maximize the success of my thread.
I could also use data mining techniques, I could look at Introductory thread to get an idea of the backgrounds and interests of the course participants and identify possible threads. I could also mine data more widely looking at the feeds (twitter, blog, delicious, ...) that course participants and also by monitoring the LAK11 tag on those and other services.
Alternatively, I could use data mining techniques to look at similar courses to learn about what makes a "good thread"?
Having identified the qualities of a "good thread" using analytics and data mining I would then start my thread, and use analytics (number of replies, content of the replies, number of unique replies) to see whether my predictions were accurate.
Oh, and no I haven't done this :)
Given the definitions, educational data mining would appear to be a subset of educational research. Educational researchers have always collected data (classroom observation) or mined data (retention rates, final marks, mid term marks etc.). The ability to produce and collect large quantities of data is a recent development. As well, large quantities of raw data are/may being generated and recorded without purpose and are available for synthesis and refinement at a later date. This would imply that educational data mining will be a growing field.
Analytics would also appear to be a subset of educational research and would refer to the breakdown or synthesis of the data into identifiable learning-related abstractions. Educational researchers have always been gathering and analyzing data (I am assuming that the data has been learner produced, but abstracted by a different means) which would imply this isn't a new field, but a growing subset of educational research.
Data mining and analytics aren't synonymous, but are closely related (as are measuring and analyzing are in any science).
So to summarize, EDM and LAK are existing subsets of educational research, but technology has enabled a rapid expansion and evolution of the fields that will require new tools and will enable more sophisticated analysis.
Analytics would also appear to be a subset of educational research and would refer to the breakdown or synthesis of the data into identifiable learning-related abstractions. Educational researchers have always been gathering and analyzing data (I am assuming that the data has been learner produced, but abstracted by a different means) which would imply this isn't a new field, but a growing subset of educational research.
Data mining and analytics aren't synonymous, but are closely related (as are measuring and analyzing are in any science).
So to summarize, EDM and LAK are existing subsets of educational research, but technology has enabled a rapid expansion and evolution of the fields that will require new tools and will enable more sophisticated analysis.
I think a useful additional distinction to make is that learning analytics implies transactional data, as opposed to assessment data (during rather than after learning and a census rather than a sample of learning). LA becomes possible with the advent of digital traces of learning behavior. This kind of data would be possible to gather manually from non-virtual environments--classrooms, but is too time consuming to be practical beyond the level of a single classroom and the processing abilities of the teacher.
Also -- this is a less rigid distinction -- to me analytics implies an exploratory or descriptive phase of data analysis. It precedes any kind of modeling or predictive statistics. I have learned from the week 1 readings that some folks are taking LA to the next level, but in general, "analytics" has been a term used in business to provide larger amounts of descriptive data clustered and classified in different ways made possible by larger processing capacity (again, the sample v. census, or population, distinction).
Also -- this is a less rigid distinction -- to me analytics implies an exploratory or descriptive phase of data analysis. It precedes any kind of modeling or predictive statistics. I have learned from the week 1 readings that some folks are taking LA to the next level, but in general, "analytics" has been a term used in business to provide larger amounts of descriptive data clustered and classified in different ways made possible by larger processing capacity (again, the sample v. census, or population, distinction).
Hi Andrew,
You say: "to me analytics implies an exploratory or descriptive phase of data analysis. It precedes any kind of modeling or predictive statistics."
I would respond: theoretically and ideally, yes -- however, when we choose which analytics tools to use (and for that matter, when analytics tools themselves are built) doesn't the logic we use to make those choices already begin to move us towards 'modeling and prediction'?
Put simply -- you wouldn't know what/how to measure unless you already had some kind of model or prediction in your head that you wanted to prove or disprove, no?
Or maybe I'm just splitting hairs here(?)
You say: "to me analytics implies an exploratory or descriptive phase of data analysis. It precedes any kind of modeling or predictive statistics."
I would respond: theoretically and ideally, yes -- however, when we choose which analytics tools to use (and for that matter, when analytics tools themselves are built) doesn't the logic we use to make those choices already begin to move us towards 'modeling and prediction'?
Put simply -- you wouldn't know what/how to measure unless you already had some kind of model or prediction in your head that you wanted to prove or disprove, no?
Or maybe I'm just splitting hairs here(?)
It's an interesting question indeed!
I think that the underlying methods/tools that are used in EDM and LAK are probably similar. You develop models, programs and queries to mine through a large corpus of data to get some result.
I think that where EDM and LAK are different is the intended audience of the data output. It seem to me that LAK is geared toward the educator and the team that the educator works with (principally instructional designers and educational specialists I'd say). LAK seems more classroom oriented.
EDM on the other hand seems more like understanding students at the macro level (how they take courses, in what sequence, which courses are generally taken together, etc). I'd say that EDM is more like the Google Latitude of learning analytics
I think that the underlying methods/tools that are used in EDM and LAK are probably similar. You develop models, programs and queries to mine through a large corpus of data to get some result.
I think that where EDM and LAK are different is the intended audience of the data output. It seem to me that LAK is geared toward the educator and the team that the educator works with (principally instructional designers and educational specialists I'd say). LAK seems more classroom oriented.
EDM on the other hand seems more like understanding students at the macro level (how they take courses, in what sequence, which courses are generally taken together, etc). I'd say that EDM is more like the Google Latitude of learning analytics
One surprising thing across these comments (to me) is the claim that EDM is about understanding students at the macro level, and looks at assessment data rather than transaction-level data. I'd say that the opposite is true of most (80%, say?) EDM research. EDM research is typically quite micro (as with the models of gaming the system and off-task behavior I talked about), and all about fine-grained data.
However, I definitely agree that EDM's focus has not historically been on the educator. It has been on the curriculum developer, and the learning scientist. There are definitely exceptions (like Neil Heffernan's work). So that is a potential difference on focus.
As for whether EDM and LAK should be the same community (a comment I saw in another post). I think that they two communities have emerged separately -- so far -- for plausible reasons (different backgrounds of researchers and different focuses, despite different purviews). I do think there's value in multiple venues -- conferences, journals -- for different perspectives to emerge. I think education research is plenty big for both communities, especially with all of the potential coming out of "big data" in education.
Cheers
Ryan
However, I definitely agree that EDM's focus has not historically been on the educator. It has been on the curriculum developer, and the learning scientist. There are definitely exceptions (like Neil Heffernan's work). So that is a potential difference on focus.
As for whether EDM and LAK should be the same community (a comment I saw in another post). I think that they two communities have emerged separately -- so far -- for plausible reasons (different backgrounds of researchers and different focuses, despite different purviews). I do think there's value in multiple venues -- conferences, journals -- for different perspectives to emerge. I think education research is plenty big for both communities, especially with all of the potential coming out of "big data" in education.
Cheers
Ryan
It is good to question what is in a name.
From casual observations it seems to me that "educational data mining" is trying to establish itself as a specific discipline (with its own journal) but is competing with a number of sub-specializations.
In the U.S. the term "academic analytics" is also in play (Educause and WCET) but seems to be overtaken with commercial tones from the business intelligence vendors. On the other hand it may appeal to a slightly more popular audience as it does resonate with the popularization of quants working in other areas and may suggest more dimensions than data mining might imply.
Learning analytics seems to be slightly less married to either particular tools and techniques but may not have longevity. There is plenty of room for cynicism if this is only about applying existing methodologies to particular types of data sets or if the observations generated are tautological.
From casual observations it seems to me that "educational data mining" is trying to establish itself as a specific discipline (with its own journal) but is competing with a number of sub-specializations.
In the U.S. the term "academic analytics" is also in play (Educause and WCET) but seems to be overtaken with commercial tones from the business intelligence vendors. On the other hand it may appeal to a slightly more popular audience as it does resonate with the popularization of quants working in other areas and may suggest more dimensions than data mining might imply.
Learning analytics seems to be slightly less married to either particular tools and techniques but may not have longevity. There is plenty of room for cynicism if this is only about applying existing methodologies to particular types of data sets or if the observations generated are tautological.
I do believe that both LAK and EDM are from the areas of Educational Research even though not directly; more on technicalities in terms of data/information, methods of analysis. They are the shared areas between Educational Research and other technological research, such as artificial intelligence and of course data mining.
So far, this is what I understand from the 'research areas' that makes LAK (as shown in attached diagram).
- Shazz
Kuala Lumpur 10:30PM
So far, this is what I understand from the 'research areas' that makes LAK (as shown in attached diagram).
- Shazz
Kuala Lumpur 10:30PM
I am still learning about the different methods of collecting and analyzing data, but LA seems to present the possiblity of studying connections between unlikely events, such as on task and off task activities.
Maybe the student is learning more through an off task event that is helping them understand or even give them time out. For instance, I remember reading a study that showed tweeting or surfing momentarily helps relieve tension from intense studying, giving some white space to the mind. That notion is me to a tee.
I also like the idea that LA teases out other possibilities, such is finding relationships between learning and logistical events (comparing downloading online library resources to the depth of online posts).