We've been focusing on learning analytics in higher education. Corporations are making extensive use of analytics and business intelligence. IBM in particular has been a leader on this front (see the book Numerati for an overview of their extensive and at times disturbing use of analytics in improving individual/team effectiveness).
So - for those of you that represent or have interest in the corporate space, what are your interests/concerns/thoughts around the use of analytics to improve learning and/or collaboration?
In reply to George Siemens
Re: What about learning analytics in the corporate sector?
by Adam Weisblatt -
It has been interesting to listen to the higher ed learning analytics discussions because there are some key differences. I believe in the end there will be more similarities but these differences really struck me:
- Corporate training rarely has grades. Instead, performance reviews and business outcomes are the measure of success. In a regulatory environment, compliance is the significant measure.
- Analytics is most important for decision making at the senior level. Senior management wants to see business results for their learning spend. In a regulatory environment senior management needs summaries of compliance levels.
- Prediction would be most helpful in creating development plans, but that information would come from the Talent Management System, not the LMS.
- Corporations are interested in the value of social media but they want to be able to track the advantage of its use because they pay for it in one way or another (exposure to outside tools, cost of internal tools, time spent).
I would be very interested in how to measure business impact from learning activity.
- Corporate training rarely has grades. Instead, performance reviews and business outcomes are the measure of success. In a regulatory environment, compliance is the significant measure.
- Analytics is most important for decision making at the senior level. Senior management wants to see business results for their learning spend. In a regulatory environment senior management needs summaries of compliance levels.
- Prediction would be most helpful in creating development plans, but that information would come from the Talent Management System, not the LMS.
- Corporations are interested in the value of social media but they want to be able to track the advantage of its use because they pay for it in one way or another (exposure to outside tools, cost of internal tools, time spent).
I would be very interested in how to measure business impact from learning activity.
In reply to Adam Weisblatt
Re: What about learning analytics in the corporate sector?
by Tanya Elias -
Hi Adam,
Where corporate training may be less likely to have grades (although I did have to get over 85% on my a recent elearning module at work before the system would mark me as completed), they do/should/could meaure performance in a way that I think is difficult in the academic world (No work management systems there).
Businesses with clearly defined KPIs and competencies should be able to determine: What is happening? What do we need to improve?
I should then "just" of figuring out how to foster and measure that improvement.
I think I mentioned we are working a project that is crunching agent performance data pre and post the implementation of new training, and preliminary results are quite positive (it is nice to know the training you spend months building is actually making a difference).
And, of course, because I see the process a cyclical, training outcomes should both be a product of strategic planning and performance management and seek to inform/improve those high-level decisions moving forward in a process of continuous improvement.
In short, I envision a world where the LMS and WMS (work management system) play nice with one another without the need to generate and interpret enormous Excel spreadsheets, all in the name of doing whatever it is we are doing better.
I do also have some siginificant concerns about building a system thats only goal is to increase ROI may actually in the long term cause more harm than good, but I'll save those thoughts for another time...
Where corporate training may be less likely to have grades (although I did have to get over 85% on my a recent elearning module at work before the system would mark me as completed), they do/should/could meaure performance in a way that I think is difficult in the academic world (No work management systems there).
Businesses with clearly defined KPIs and competencies should be able to determine: What is happening? What do we need to improve?
I should then "just" of figuring out how to foster and measure that improvement.
I think I mentioned we are working a project that is crunching agent performance data pre and post the implementation of new training, and preliminary results are quite positive (it is nice to know the training you spend months building is actually making a difference).
And, of course, because I see the process a cyclical, training outcomes should both be a product of strategic planning and performance management and seek to inform/improve those high-level decisions moving forward in a process of continuous improvement.
In short, I envision a world where the LMS and WMS (work management system) play nice with one another without the need to generate and interpret enormous Excel spreadsheets, all in the name of doing whatever it is we are doing better.
I do also have some siginificant concerns about building a system thats only goal is to increase ROI may actually in the long term cause more harm than good, but I'll save those thoughts for another time...
In reply to Tanya Elias
Re: What about learning analytics in the corporate sector?
From conversations that I've had with former classmates that work in corporate training (Instructional Designers), corporate does care about what grade you get, but perhaps they don't care how you use the knowledge you've gained. The main thing that comes up are compliance workshops that are mandatory. You need a grade of 70% or above to pass (just naming a figure, actual figure may vary). All corporate is concerned with (from the use cases I've heard) is that their people pass the mandatory compliance. No evaluation past the L1 or L2 (if you are concerned with compliance) evaluation is ever done.
In reply to Apostolos Koutropoulos
Re: What about learning analytics in the corporate sector?
by Peter Condon -
It seems to me that industry has to set targets for the completion of courses to satisfy accreditation (or mandatory compliance as Apostolos says). Once past the basics, many industrialists that I have dealt with in the past have been more interested in the outcomes of the learning, for example: What will the learner be able to do/ do better/ contribute more, etc., what benefits will the comapny gain?
In addition the cost/ benefit has to be justified by the provider/ HR department, to ensure more of the same (if that is what you want to encourage).
They (HR/ Provider) then need to persuade the learner (if the course is not to be compulsory) why they need to undertake the course: The immediate benefits: future prospects: relevance to current role, all apply here but there may also be an element of who else is taking/ has taken the course and the personal interests of the learner.
In this way I think that a 'Hunch' based analysis might be beneficial in influencing course design, subject matter presentation and may well be better placed at the end of a course rather than at the beginning to relate to uptake decisions / success rate and even completion rates. As long as the results are used. So much analysis is quickly reviewed and the filed away - never to be used again or not looked at in the first place.
How many of those who looked at Hunch would return to it again in a years time to performa a comparison. In addition it may be difficult to get learners to complete an in depth analysis. How many have answered all of Hunch's questions - how many got bored before they finished?
I think the questions I would like to ask is 'Who are the analytics for - the learning provider or the learner?' Which leads to 'Should the analytics be up-front or hidden?'
In reply to Peter Condon
Re: What about learning analytics in the corporate sector?
by Gillian Palmer -
Peter, my own view from organising corporate education/training is that if the analytics are hidden, they can cause major loss of trust between the employee and the organisation. The employee may be quite happy for the manager to have data, or their learning mentor, or the HR person or the union rep but people in non-compliance programmes are, in my experience, not usually happy for all - or even any - of them to have the information without their knowledge. There's a psychological contract in place that runs something along the lines of, "If I get the right learning outcomes, how I get there is my business as a professional responsible for my own work and you have no more right to micro-manage that than you do any other aspect of my work." Of course, if you draft the learning outcomes properly and share them, then the analytics can be built-in for the benefit of all rather than tacked on as a back-room service. For more on my unease about non-disclosure see my blog where I'm working slowly through the doubts to try and get to a practical and learner-centric solution. Latest entry here: http://learningandqualifications.wordpress.com/2011/01/12/learners-using-their-lms-data/
In reply to Peter Condon
Re: What about learning analytics in the corporate sector?
by Dolors Capdet -
The corporate training, unlike academic training, can not be measured during the training process because it is linked to its practical development.
It is only possible to measure its effectiveness based on its practical application in the real working environment, where does not measure the specific result of knowledge adquired but if you measure the degree of achievement of individual goals and group obtained.
It is only possible to measure its effectiveness based on its practical application in the real working environment, where does not measure the specific result of knowledge adquired but if you measure the degree of achievement of individual goals and group obtained.
In reply to Dolors Capdet
Re: What about learning analytics in the corporate sector?
by Francisco Reis -
It can be measured during the training process if we allow trainees to use their real work during the course. Isn't that what life long learning is all about?
In reply to Francisco Reis
Re: What about learning analytics in the corporate sector?
by Dolors Capdet -
No, no. Corporate training is not what is meant by lifelong learning, although there are coincident points.
You may you be formed throughout life in different topics, not necesarily only of your current job.
The business training especially for new and old workers is usually provided in a simulation environment and only at the last moment can access on the real environment - to avoid mistakes.
You may you be formed throughout life in different topics, not necesarily only of your current job.
The business training especially for new and old workers is usually provided in a simulation environment and only at the last moment can access on the real environment - to avoid mistakes.
In reply to Dolors Capdet
Re: What about learning analytics in the corporate sector?
by Peter Condon -
Dolors: If, as you say, corporate training can only be measured by the results of applying the learning in the learner’s work, I would struggle to see the difference between corporate training and lifelong learning.
In my experience it is the knowledge that is gained that is assessed by corporate training and often courses are written that do not allow the learner to progress unless they prove their knowledge of the current study. In fact this is my argument against much of the current corporate training; that it only teaches knowledge, to be able to do something by rote, not the application of the knowledge or the development of skills. Of course there are courses that are skills based, but I do not know of many such online courses in industry. To write a course that can assess skills costs more than the general (off the shelf) courses that teach and assess the knowledge since the skills will be relevant to a particular company’s systems and the course applicable only to that company.
Francisco: I think that companies would be reluctant to allow trainees to use their real work during a course because of the risks involved. For example; in the Timber Industry (with which I am familiar) trainers often take on trainees from different companies but have to develop their own scenarios because of the secrecy many companies have about their processes.
In the UK, the term ‘Lifelong Learning’ is applied to any learning that is undertaken after leaving formal education. In theory online corporate training would fall into this category but, to me, it does not fit well due to the simplistic methods used (in order to assure assessment) and because it does not address the acquisition of the ability and skills needed to apply the knowledge. Lifelong learning is more about knowledge based upon experience (both one’s own and that of colleagues) and therefore bound up with skills and ability. The trick is, understanding how to capture this lifelong learning in a way that is meaningful to employers (current and future). I am hoping that learning analytics may supply some of the answers.
In reply to Peter Condon
Re: What about learning analytics in the corporate sector?
by Dolors Capdet -
We agree Peter.
Indeed, one thing is the learning and other is the skills training. The first is permanent while the second is set to a particular work situation
And it is true, which the training have controls (tests) that measure the understanding of the basic concepts learned. However, I think this does not really measure learning but justifies the payment of fees of training.
Indeed, one thing is the learning and other is the skills training. The first is permanent while the second is set to a particular work situation
And it is true, which the training have controls (tests) that measure the understanding of the basic concepts learned. However, I think this does not really measure learning but justifies the payment of fees of training.
In reply to Peter Condon
Re: What about learning analytics in the corporate sector?
by Francisco Reis -
Of course, Peter, there are many diffent situations and no rule.
But my experience is that even in traditional higher education you have students doing internships in companies whose work has to be kept secret (my students are from software engineering). All goes well not even involving lawyers!
Joining this with John Fritz post on students being the ones to act on their own data I think all makes even more sense: we learn our way with our personal and/or professional data and interests and we share just what we want.
Is this utopia? I do think in many cases we can achieve it.
But my experience is that even in traditional higher education you have students doing internships in companies whose work has to be kept secret (my students are from software engineering). All goes well not even involving lawyers!
Joining this with John Fritz post on students being the ones to act on their own data I think all makes even more sense: we learn our way with our personal and/or professional data and interests and we share just what we want.
Is this utopia? I do think in many cases we can achieve it.
In reply to Dolors Capdet
Re: What about learning analytics in the corporate sector?
From all the training sessions that I've had, I can say that some things can be measured during training, but more substantive things (like L3 and L4 evaluations under Kirkpatrick) cannot.
I also tend to think that some training does require more than the time one puts into completing the eLearning module (or classroom session). I think that post-training interventions and assistance really do make it much more likely that the information gained in training can be put to use.
I also tend to think that some training does require more than the time one puts into completing the eLearning module (or classroom session). I think that post-training interventions and assistance really do make it much more likely that the information gained in training can be put to use.
I think you hit the nail on the head here Peter with this:
What will the learner be able to do/ do better/ contribute more, etc., what benefits will the comapny gain?
In the corporate world learning analytics has to tie to the return on investment for training - why would a commercial entity do it otherwise?
What will the learner be able to do/ do better/ contribute more, etc., what benefits will the comapny gain?
In the corporate world learning analytics has to tie to the return on investment for training - why would a commercial entity do it otherwise?
In reply to George Siemens
Re: What about learning analytics in the corporate sector?
by Hans de Zwart -
There are a couple of questions in the corporate area that I hope to have better answers for at the end of this course:
1. I am very involved with innovation. It would be interesting to think about how learning analytics can help me in creating business cases voor learning innovation opportunities. Can it help in validating the business value of certain innovations? I am sure it can, but it requires the competences of asking the right questions and successfully navigating the maze of data ownership. This could allow us to use metric and data (instead of opinion) to prioritise investment.
2. I have a complete fascination for methods like social network analysis or semantic similarity checking and how they could be used to make individuals work smarter. I firmly believe we are at the cusp of many practical implementations of these kinds of methodologies and it is good to be at the forefront of that.
The concern that I have is very much regulatory. Most attempts at doing social network analysis for example seem to be blocked by legal departments with privacy concerns. This needs to change in some way before it can start becoming more useful.
1. I am very involved with innovation. It would be interesting to think about how learning analytics can help me in creating business cases voor learning innovation opportunities. Can it help in validating the business value of certain innovations? I am sure it can, but it requires the competences of asking the right questions and successfully navigating the maze of data ownership. This could allow us to use metric and data (instead of opinion) to prioritise investment.
2. I have a complete fascination for methods like social network analysis or semantic similarity checking and how they could be used to make individuals work smarter. I firmly believe we are at the cusp of many practical implementations of these kinds of methodologies and it is good to be at the forefront of that.
The concern that I have is very much regulatory. Most attempts at doing social network analysis for example seem to be blocked by legal departments with privacy concerns. This needs to change in some way before it can start becoming more useful.
In reply to Hans de Zwart
Re: What about learning analytics in the corporate sector?
by Bert De Coutere -
I can indeed testify that some experiments with social network analysis or even capturing and using peer evaluations (how people rate or think about the expertise of their peers) is flirting with privacy legistlation, mainly or even exclusively in Europe.
In my company, some of these tools under the umbrella 'Technology Adoptation Program' are for this reason not available in selected European countries.
In my company, some of these tools under the umbrella 'Technology Adoptation Program' are for this reason not available in selected European countries.
In reply to Hans de Zwart
Re: What about learning analytics in the corporate sector?
by Dianne Rees -
Your first topic is an interesting one and challenging because innovation won't necessarily have a predictable ROI. It might be worth building a case by benchmarking against companies that have set aside a separate part of the company as a "Lab" in which there's less pressure to achieve immediately commercial endeavors and noting how many valuable commercial products came out of such an environment. You could also note the value arising from innovation workshops (in terms of IP which has something of a measurable ROI).
In reply to Dianne Rees
Re: What about learning analytics in the corporate sector?
by Hans de Zwart -
You are right Dianne in that it is hard to predict the ROI of innovations, but after you have done a pilot it should be possible to use data to strengthen your intuitions about whether somethings should work.
In reply to Dianne Rees
Re: What about learning analytics in the corporate sector?
by Bert De Coutere -
I still remains very hard for proper ROI calculation, because people participating in innovation are usually far from a 'random' set of people. So any comparisions you would typically make between the 'early adopters' versus 'the control group' is biased because the randomness assumption usually doesn't hold. (Unless you can force employees to participate in innovative initiatives, which you usually can't)
In reply to George Siemens
Re: What about learning analytics in the corporate sector?
by Bert De Coutere -
(taken from post on my blog where I try to make sense of week 1)
The finality of education and corporate sectors is different, and that has its consequences for the analytics. The finality of education (from what I've read this week) is 'the grade', and thus learning and academic analytics will be about dashboards on how students are doing on their journey towards 'the grade' and how they match with the rest of the group, and signaling and remediation of students that might not get to 'the grade'.. It will be about the Key Performance Indicators the government cares about: dropouts and people failing. The finality of the corporate sector is not an educational grade. It is not even an accreditation. (That is an semi/false finality that got into corporations when they adopted the university model in their corporate 'universities'. Only within the operational aspects of corporate 'universities' can you apply much of the same that education analytics would.) The finality of corporations is valuable performance of their employees. Therefore, in my humble opinion, you cannot properly introduce learning analytics in a corporation without also getting your act together on performance analytics. They go hand in hand. (Who said that learning IS the work again?)
The finality of education and corporate sectors is different, and that has its consequences for the analytics. The finality of education (from what I've read this week) is 'the grade', and thus learning and academic analytics will be about dashboards on how students are doing on their journey towards 'the grade' and how they match with the rest of the group, and signaling and remediation of students that might not get to 'the grade'.. It will be about the Key Performance Indicators the government cares about: dropouts and people failing. The finality of the corporate sector is not an educational grade. It is not even an accreditation. (That is an semi/false finality that got into corporations when they adopted the university model in their corporate 'universities'. Only within the operational aspects of corporate 'universities' can you apply much of the same that education analytics would.) The finality of corporations is valuable performance of their employees. Therefore, in my humble opinion, you cannot properly introduce learning analytics in a corporation without also getting your act together on performance analytics. They go hand in hand. (Who said that learning IS the work again?)
In reply to Bert De Coutere
Re: What about learning analytics in the corporate sector?
by Linda Burns -
I have work both in higher education and the corporate world. Learning is learning. Learning is process the same way in the brain. It does not depend on where the learning is taking place in a university or a corporation.
We need to break out of this old idea the training and learning is different. And we need to change education/learning in both worlds. Students are getting out of school without the skills needed for the very specific skills needed for jobs today. Schools need to change and teach students how to learn. Corporations need to change and start educating their employees. Both need to create a much more open and creative and trusting environment.
We need to break out of this old idea the training and learning is different. And we need to change education/learning in both worlds. Students are getting out of school without the skills needed for the very specific skills needed for jobs today. Schools need to change and teach students how to learn. Corporations need to change and start educating their employees. Both need to create a much more open and creative and trusting environment.
I tend to agree with Linda that there should be less distinction between approaches to (good) corporate training and (good) higher education. Both should be problem-based and the problems should reflect real-world issues.
Analytics relating to training tends to be focused on Kirpatrick's approach, which isn't the only evaluation method out there. As noted by Brinkerhoff, "Achieving performance results from training is a whole-organization challenge. It cannot be accomplished by the training function alone. … Virtually all evaluation models are construed conceptually as if training were the object of the evaluation.” (Brinkerhoff, http://aetcnec.ucsf.edu/evaluation/Brinkerhoff.impactassess1.pdf) (as cited by Jane Bozarth in Learning Solutions Magazine). I think the concept of training as an isolated function is an issue that goes beyond the analytics challenges. It's why training has received quite a bit of backlash lately (in favor of non-formal learning, for example). Just as training needs to be integrated into the work environment more seamlessly, analytics related to workplace learning also needs to be integrated into the overall work environment.
Analytics relating to training tends to be focused on Kirpatrick's approach, which isn't the only evaluation method out there. As noted by Brinkerhoff, "Achieving performance results from training is a whole-organization challenge. It cannot be accomplished by the training function alone. … Virtually all evaluation models are construed conceptually as if training were the object of the evaluation.” (Brinkerhoff, http://aetcnec.ucsf.edu/evaluation/Brinkerhoff.impactassess1.pdf) (as cited by Jane Bozarth in Learning Solutions Magazine). I think the concept of training as an isolated function is an issue that goes beyond the analytics challenges. It's why training has received quite a bit of backlash lately (in favor of non-formal learning, for example). Just as training needs to be integrated into the work environment more seamlessly, analytics related to workplace learning also needs to be integrated into the overall work environment.
Hi LInda,
I couldn't agree more. I've recently been talking with my boss at work in the business world about the importance of moving towards creating efficient problem-solvers and innovators in the workplace (educating). Simultaneously, I see a shift in universities towards focusing on skills of importance in the workplace (training).
Both sides have a lot to learn from the other.
I couldn't agree more. I've recently been talking with my boss at work in the business world about the importance of moving towards creating efficient problem-solvers and innovators in the workplace (educating). Simultaneously, I see a shift in universities towards focusing on skills of importance in the workplace (training).
Both sides have a lot to learn from the other.
In reply to Linda Burns
Re: What about learning analytics in the corporate sector?
by Bert De Coutere -
You are right that we should avoid keeping the silos of education and work in an interconnected world. You might be interested in a paper from IBM that lists the idea of 'the education continuum'. It not only talks about a continuum across the silos in education (primary, secondary, university) and work, but also alignment with government and society's priorities and needs.
So let me try to better explain my point why there are differences: a lot of discussions in this course so far seem to talk about using learning analytics within the boundaries of learning/training. It's about operational productivity to do things better (predict failure and remediate, better allocation of time and resources, analyse what works best, etc). If we reduce learning analytics to that, there might be little difference within education and corporate training (that is often just mimicking the educational system). In corporations, learning is not a goal on its own. It only becomes relevant if that helps to achieve the real goals that are valuable performance. (same goes for most corporate functions, they only matter as much as how they contribute to the overall corporate performance). So if learning analytics really wants to make the day in corporations, it needs to see beyond its own box, and stretch into performance analytics, align with trends detection in business and workforce, etc. Corporate training is often critised for its low integration within the business, insufficient alignment with business priorities and weak evidence of impact. Learning analytics have potential to address these issues, as well as the operational sanitation checks within your learning box.
So let me try to better explain my point why there are differences: a lot of discussions in this course so far seem to talk about using learning analytics within the boundaries of learning/training. It's about operational productivity to do things better (predict failure and remediate, better allocation of time and resources, analyse what works best, etc). If we reduce learning analytics to that, there might be little difference within education and corporate training (that is often just mimicking the educational system). In corporations, learning is not a goal on its own. It only becomes relevant if that helps to achieve the real goals that are valuable performance. (same goes for most corporate functions, they only matter as much as how they contribute to the overall corporate performance). So if learning analytics really wants to make the day in corporations, it needs to see beyond its own box, and stretch into performance analytics, align with trends detection in business and workforce, etc. Corporate training is often critised for its low integration within the business, insufficient alignment with business priorities and weak evidence of impact. Learning analytics have potential to address these issues, as well as the operational sanitation checks within your learning box.
In reply to Bert De Coutere
Re: What about learning analytics in the corporate sector?
by Peter Meerman -
Hi Bert and all,
I'm very in line with Bert's comments on the scope of learning analytics to be wider than the learning (development) process and/or intervention only, and must include components that are relevant for the business.
Also agree with Hans that we haven't really (yet?) talked about the 'how' in LA for corporations.
Working only in the corporate setting myself, I have no clear answer nor examples of how to use/implement it. Key reason for that is the strong link you need with business intelligence & analytics (BIA) and the fact that so many corporations (still) have little to no strategy/approach/system in place.
So my question would be, are we at all able to deliver LA that are valuable for the business if and when the business have no clear understanding of what is 'valuable' at all??
(I know, i'm being a bit provocative here)
Using the 'hunch' tool did triggered a thought:
1. (corporate) learning changes more and more into enabling employees to work smarter
2. One of the key trends today is the move to 'personal learning paths' (thanks Bert for sharing the paper).
3. Employees (especially 'knowledge workers') most likely know best a) what competencies/skills/knowledge the have, b) what the need for the job, and c) what they are missing (if not, ask yourself if you are a through knowledge worker!)
4. Looking at myself, I have increased challenges finding the knowledge & learning that would really benefit (especially as there is so much available) myself, and for sure would grasp reliable data analysis that can help me out here.
Why don't we combine these statements and look at LA from the perspective of the individual, rather than the corporation and/or senior/line management? (on a side note, this could also contribute to stimulating a learning culture etc...)
In that case, the key question would be: how can we design/develop/implement LA to optimal support the individual 'learner' in their effort to become a smarter worker...?
We could do this by enabling the individual to build their own business case based on for example comparising based analysis of present data, like peer references a la 'hunch', combined with models/analytics that predict or compare the potential result/outcome?
I'm very in line with Bert's comments on the scope of learning analytics to be wider than the learning (development) process and/or intervention only, and must include components that are relevant for the business.
Also agree with Hans that we haven't really (yet?) talked about the 'how' in LA for corporations.
Working only in the corporate setting myself, I have no clear answer nor examples of how to use/implement it. Key reason for that is the strong link you need with business intelligence & analytics (BIA) and the fact that so many corporations (still) have little to no strategy/approach/system in place.
So my question would be, are we at all able to deliver LA that are valuable for the business if and when the business have no clear understanding of what is 'valuable' at all??
(I know, i'm being a bit provocative here)
Using the 'hunch' tool did triggered a thought:
1. (corporate) learning changes more and more into enabling employees to work smarter
2. One of the key trends today is the move to 'personal learning paths' (thanks Bert for sharing the paper).
3. Employees (especially 'knowledge workers') most likely know best a) what competencies/skills/knowledge the have, b) what the need for the job, and c) what they are missing (if not, ask yourself if you are a through knowledge worker!)
4. Looking at myself, I have increased challenges finding the knowledge & learning that would really benefit (especially as there is so much available) myself, and for sure would grasp reliable data analysis that can help me out here.
Why don't we combine these statements and look at LA from the perspective of the individual, rather than the corporation and/or senior/line management? (on a side note, this could also contribute to stimulating a learning culture etc...)
In that case, the key question would be: how can we design/develop/implement LA to optimal support the individual 'learner' in their effort to become a smarter worker...?
We could do this by enabling the individual to build their own business case based on for example comparising based analysis of present data, like peer references a la 'hunch', combined with models/analytics that predict or compare the potential result/outcome?
In reply to Peter Meerman
Re: What about learning analytics in the corporate sector?
by Dianne Rees -
I'd just add that the learner should have a thorough understanding of business issues, where the business fits in the competitive landscape, and business intelligence impacting that fit. But I don't separate learning from that larger understanding. And, for that mattter, learning developers in organizations should be intimately involved in these issues as well since these are the critical elements of the performance and task analysis that form the heart of instructional design. Training and informal learning should always be about that bigger picture; they're not separate except as currently implemented.
I think Peter's point is important, to keep learners hooked into that bigger picture,the learner should be responsible for his/her own talent management (though this can be nurtured and supported by management). The ideal of LAK is to keep the worker/learner a self-directed participant in the work culture in order to give the learner a sense that he/she has some ownership in the business (and no, LAK is not currently implemented that way, either in education or in organizations).
I think Peter's point is important, to keep learners hooked into that bigger picture,the learner should be responsible for his/her own talent management (though this can be nurtured and supported by management). The ideal of LAK is to keep the worker/learner a self-directed participant in the work culture in order to give the learner a sense that he/she has some ownership in the business (and no, LAK is not currently implemented that way, either in education or in organizations).
In reply to Bert De Coutere
Re: What about learning analytics in the corporate sector?
by Gillian Palmer -
Hi Bert, Peter and all,
I definitely agree that the education continuum (otherwise known as professional lifelong learning?) needs learning analytics to be seen in a much broader context than mere course statistics. While so much political input is made into the availability (or not) of education and training that is hardly avoidable and the challenge, as others have commented, is to stop the analytics becoming gameplay. My stance remains that at the grassroots level there should be a focus on making learning analytics useful for learners but that does not stop them being useful to others as well. In a corporate setting, Dan Snowden has done some very useful work under the 'knowledge management' umbrella (and his blog is 'Cognitive edge'). The diversity of academic labels used by different people does not, in this case, matter: it is the utility of the end-result that matters; along with the spread of that utility, its ethics and affordability.
I definitely agree that the education continuum (otherwise known as professional lifelong learning?) needs learning analytics to be seen in a much broader context than mere course statistics. While so much political input is made into the availability (or not) of education and training that is hardly avoidable and the challenge, as others have commented, is to stop the analytics becoming gameplay. My stance remains that at the grassroots level there should be a focus on making learning analytics useful for learners but that does not stop them being useful to others as well. In a corporate setting, Dan Snowden has done some very useful work under the 'knowledge management' umbrella (and his blog is 'Cognitive edge'). The diversity of academic labels used by different people does not, in this case, matter: it is the utility of the end-result that matters; along with the spread of that utility, its ethics and affordability.
In reply to George Siemens
Re: What about learning analytics in the corporate sector?
by Viplav Baxi -
One of the areas that greatly interest me is simulations for corporate training needs. I have done work in the IT and Financial sectors that show the power of simulations to bring together some complex information about how a learner navigated a simulated job situation. Scores are too simplistic to do justice to such complex tracking of learner progress and competence. Consequently, learning & knowledge analytics become more complex as well.
For example, let us consider a scenario that has multiple decision points (connected like in a graph) and multiple paths to the correct outcome. Let us assume that there is an ideal path (not hard to imagine in a highly disciplined process training). A learner's decision making trail or actions trail could be compared to the ideal path/trail and analytics could be programmed to infer from deviations to get a better and more comprehensive picture of learner performance. (also not unlike the notion of knowledge analytics being used to compare competency levels in a discipline).
If you know cricket, you would be familiar with a graph that shows runs scored vs overs for both teams with circles denoting fall of wickets, resulting in what are popularly called "worms" that deviate from each other on the graph as the match progresses.
Point is, these analytics move from being comparisions between numbers (Peter spent 5 more minutes than Pan on the google group), to being comparisons and analytics based on patterns and paths.
Viplav
In reply to Viplav Baxi
Re: What about learning analytics in the corporate sector?
by Shahrinaz Ismail -
Thanks, Viplav... Your explanation really put my understanding in words, quite clearly (about the patterns and paths versus numbers).
Tailing this, I wonder:
- How can this pattern be presented to senior management (assuming they are not at 'our level of understanding') in a form that they appreciate?
- How can we be certain that the paths the employee takes in getting to the final point are something they learnt and contributing to the final result, or merely a waste of time until they found the right 'nodes'?
- How should the evaluation be designed to make it fair for every employee (because some people do take time to understand after a long-winded paths, etc...)?
- Does this mean an experienced 'wanderer' could achieve the KPI better than newbies? I don't think so too.
There's still further way to go after this, and I believe corporate sector is very particular about 'measurement' in evaluation.
Hope I'm in track with my points,
- Shazz
Kuala Lumpur 1:33AM
Tailing this, I wonder:
- How can this pattern be presented to senior management (assuming they are not at 'our level of understanding') in a form that they appreciate?
- How can we be certain that the paths the employee takes in getting to the final point are something they learnt and contributing to the final result, or merely a waste of time until they found the right 'nodes'?
- How should the evaluation be designed to make it fair for every employee (because some people do take time to understand after a long-winded paths, etc...)?
- Does this mean an experienced 'wanderer' could achieve the KPI better than newbies? I don't think so too.
There's still further way to go after this, and I believe corporate sector is very particular about 'measurement' in evaluation.
Hope I'm in track with my points,
- Shazz
Kuala Lumpur 1:33AM
In reply to Shahrinaz Ismail
Re: What about learning analytics in the corporate sector?
by Viplav Baxi -
Shazz,
There could be many visualization techniques culled from daily intelligible life that we could use e.g. directional maps, heat maps, graphs etc. that could be presented to senior management.
How to be sure they are learning? The system could record dead-ends. Simulated scenarios, for example, often have pedagogical dead-ends only a couple of levels deep, post which the learner is forced to retrace her steps and try again. Again, what works in a simulation may not work in an open ended scenario unless there are checkpoints that enable the system to understand and measure progress. In some cases, the wandering around gathering accidental knowledge, may be a desired feature of the learning experience.
How to keep evaluation fair? No definite answer for that one. It depends on the metrics you are using for the evaluation.If time taken is a metric then it would be uniformly used for all learners. Having said that simulations could have guide modes or collaborative environments would have peers or facilitators or instructors who provide the guidance. There is also often some value in letting an orientation happen if it is the first time the learner has encountered the simulation or the format.
Experienced wanderer = expert learner/performer? Maybe. Some people learn much better if they wander around getting a general sense before they settle down to a decided path.
You are absolutely on track :)
Viplav