Chat log for session with Kimberly Arnold
January 26, 2011
Andreas Link: hi
Moderator (George Siemens): http://www.itap.purdue.edu/people/detail.cfm?alias=arnold0
Gillian: Hello from UK
merc√® gal√°n: Hello again to all from Spain
Moderator (George Siemens): perfect
Moderator (Sylvia Currie): sounds good
ines cambiasso: yes, sound is good
Moderator (George Siemens):
Moderator (Sylvia Currie): Nice, always like to have a photo of presenter
Moderator (Sylvia Currie): learning analytics, knowledge analytics... now academic analytics
Moderator (George Siemens): yes
SuzGupta: Hello from sunny Reno, NV and hello again to CCK folks. Quite the Elluminate party today
schawn t: how about just analytics
Vince Jansen: 2 of 3 !
schawn t: we can throw any number of adjectives in front of it we want. Really it becomes the context in which we are using analytics
ViplavBaxi: 2 AM and a second cup of coffee..Hi All
Gillian: @KIm What length is the cycle?
SuzGupta: @Viplav -- where are you?
ViplavBaxi: New Delhi
SuzGupta: oh! thanks for staying up to learn with us!
ViplavBaxi: wouldn't miss it...
SuzGupta: interesting that two here are huge for-profit online u's
SuzGupta: @Kim -- do you know of any wide uses of this in K-12?
schawn t: Does anyone know of any Corporate or Government programs using learning analytics
Moderator (George Siemens): @schawn - i'm not that familiar with corporate use of analytics in learning
Moderator (George Siemens): DoD is using learning analytics in their teaching/training
Gillian: @schawn t I can put you in touch with corps if you want.
Saleha: Has there been any work done in associating student success indicators with student learning?
schawn t: just seems as though most of the analytics (learning, knowledge) is happening more in the academic sector than anyother - true statement
Moderator (George Siemens): @schawn - corporations are, however, driving data use for decision making...
ViplavBaxi: Project TinCan and LETSI are also re-looking at stuff on the standards side
Moderator (George Siemens): @schawn - companies like IBM, in particular, are very active in data analysis and gaining insight and contributing to decision making process
schawn t: @Gillian, yes thanks. I am just curious of some examples
SuzGupta: do you have some data on that (preservation of marketing/recruiting doillars)?
Phillip Long: Pearson is one of the corporate leaders in this space, along with Sunguard
schawn t: @george - are companies using more learning analytics or knowledge analytics (maybe both) to help drive decision? I guess I am trying to see if there is a line that can be drawn between learning and knowledge analytics
SuzGupta: I would think there would be savings too if tutoring can be offered automatically online v. live tutors/centers
Moderator (George Siemens): @schawn - I think companies would be more interested in the concept of knowledge analytics
Moderator (Tanya Elias): we are working at integrating business and leanring analytics
schawn t: @george, I would agree.
Moderator (George Siemens): learning often doesn't always get high funding priority
Adam Weisblatt: Yest they'll spend money on scantrons which seems to me to be a waste
ViplavBaxi: @schawn t: anyone using Human Capital Management software such as those provided by SABA, or enterprise social platforms like MZinga regularly employ learning analytics with a high use of BI work
Moderator (George Siemens): Verna allee's work on "value networks" has made a fari bit of impact
ViplavBaxi: millions of variables?
Saleha: Has there been any work done in associating student success indicators with student learning?
Saleha: associating student success indicators with student learning indicators
Phillip Long: What were the principal variables in this model (independent variables)?
SuzGupta: what does "SIS" mean here?
Mark Gbur: Student Information System
Adam Weisblatt: Who on their team actually did this statistical analysis work?
SuzGupta: student information systems?
Phillip Long: That's another argument for an institutional academic data warehouse!
ViplavBaxi: comparative dynamics - testing the model over periods and samples - would be interesting to do
Mark Gbur: It is interesting because there are multiple CMS's and other networks at Purdue, Krannert School of Management had money and they were on their own networks so I imagine quite a bit of effort went into funneling all the data
Moderator (George Siemens): did you want to pause for questions here?
Phillip Long: Need more clarity on the difference between the larger CMS model and the course-specific algorithms
Mary 1: Good question.
Phillip Long to George Siemens, Kim Arnold, Tanya Elias, Sylvia Currie: Ironically many institutions have administrative data warehouses...
Phillip Long: It reveals what matters in terms of dollars!
Adam Weisblatt: Why is there a privacy concern is the data is being used for statistical analysis?
Moderator (George Siemens): @Phillip - agree!
schawn t: I vote for the latter
Adam Weisblatt: Why is there a privacy concern IF the data is being used for statistical analysis?
ViplavBaxi: In corporates, one of the interesting uses of analytics is to judge the performance of Learning and Development Managers
Phillip Long: Learning is a diffcult output measure - how much you spend for parking is much more direct and easier to measure and has significant annual costs
Phillip Long: (where parking is just an example
Gillian: I can see a bigger problem with privacy if a university is heavily commercial (sponsored...)
schawn t: is it because measuring performance improvement is often very complex and situational
Phillip Long: There is a significant alternate view here that you can inadvertently lead to a 'tracking' system if individual data is in any way identified
Mark Gbur: It is interesting George, you're somewhat against utilization of LMS/CMS's ie (Blackboard), but much of the data pulled here is a result of that structure and the ability to pull data from a standardized, rigorous system and databases. Without structured systems, the data would not be there and a SIGNALS system would not be possible. Where do you stand on utility v. value of the data?
Mary 1: The accrediting agencies require alignment with 'standards' and 'benchmarks', which require the creation of performance assessments and rubrics. The development of these takes an enormous amount of effort and time. Faculty have concerns about this approach to learning. Faculty are concerned about the use of student data in ways that do not take into account the conditions and the constraints.
Moderator (George Siemens): @Mark - v. good ques
Moderator (George Siemens): we are working with distributed systems for learning at AU
Moderator (George Siemens): I'm interested in how we can collect data in distributed networks (social learning)
Phillip Long: Mark - that's exactly the concern I think - contextualising performance can be easily lost in the roll-up of statistical analysis
Moderator (George Siemens): I find some of the monitoring tools (klout, radian6) to be an interesting model in that regard
Mary 1: Great point! Who wants to know. Why? So what...
Adam Weisblatt: Does anyone use this approach to measure the effectiveness of assessments?
Mark Gbur: Klout is definitely an interesting tool, I haven't seen radian6 though. I think it would be hard to itemize something from klout for performance/assessment in a course though, (at least at this time)
Richard Olsen: may have missed it but how is success measured? grades, other?
SuzGupta: @Richard -- grades I think
Phillip Long: Interesting that a stoplight is interpreted as my instructor cares about me ...
ViplavBaxi: If I remember correctly, Ryan Baker also showed us similar conclusions - students benefit from getting feedback about how they are doing...
Adam Weisblatt: Problem with corp data is that is is big (5 million transactions per year in Pfizer's LMS) but no depth (Required date, completed date)
Moderator (George Siemens): @Viplav - yes. On one level, that seems obvious, but....
Mark Gbur: (apologies @ work and I think maybe I missed something), is Signals a plugin for Blackboard or is it an open source plugin, how exactly is it being implemented and delivered to the students (stand alone v. integrated)
Phillip Long: Does this mean tha NOT enrolling in the next semester = dropping out?
Phillip Long: or just delaying
SuzGupta: would be great to see this from a cost standpoint = money gained by increased retention v cost of project
SuzGupta: or whatever the right way is to analyze the cost side
ViplavBaxi: so one application of BIG Data analytics in education would be to develop a system that could inform and counsel students in a bid to improve performance...
SuzGupta: and thus increase retention and reduce loss of marketing/recruitment dollars
Phillip Long: That system is Bb ?
SuzGupta: (lakh, whatever ;))
Moderator (George Siemens): ok
Moderator (George Siemens):
Mark Gbur: perfect, thanks
SuzGupta: It's Cris! <Waves>
Adam Weisblatt: The corp equivalent would be edn of year performance review grades.
Adam Weisblatt: I wonder if the approach is transferrable form acedemic to corporate
Cris: Hi SuzGupta --shhhh trying to sneak in
ViplavBaxi: Corporates would perhaps just hold up the salary raise as part of the carrot and stick implementation of elearning
Phillip Long: Generic question: there are concerns about using linear regression (traditional parametric statistics) for these kinds of predictive models. This is in contract to Bayesian modeling as the intention of the modeling is intervention into the same population, and, the variables involved don't often follow normal disributions. Comment?
Mike: Can you provide some more examples on how you collaborate with the student interventions amongst multiple department/stake holders. Does Signals support this?
Cris: Ethics --- "obligation of knowing" like that
Adam Weisblatt: In corp, LMS and Perf mgment system woiuld have to be connected
Gillian: Automation with adult distance learners could be a nightmare for teachers if they have to spend a lot of time online explaining the 'automaticness' and assuring students that flexibility is there really ?
ViplavBaxi: @Phillip: I am almost certain they would be using non-linear and time series techniques...good question
Phillip Long: Obligation of knowiing is a double-edged sword. Do we believe the model enough to suggest the student
Phillip Long: oops - continuing " at risk" should be give other options (withdrawal) or ...?
Mary McEwen to George Siemens, Kim Arnold, Tanya Elias, Sylvia Currie: What about ‚Äúweed out‚Äù classes? [courses requiring at least a B grade before acceptance into highly competitive programs (e.g. computer science, nursing, etc.)] How might something like Signals be perceived for these courses?
Mark Gbur: I believe the perception is that SunGard is the 'benchmark' SIS on the market, (portal, billing, grades, course schedules, marketing, recruiting - "all but CMS") - having Signals would certainly further their reputation, I find that rather intriguing given the results presented here.
Feedback is often the most underutilized component in courses and most LMS/CMS are not even close to figuring it out.
Adam Weisblatt: Is the juice worth the squeeze
ViplavBaxi: @Phillip I think Cointegration techniques would also be very useful here
SuzGupta: @Adam -- catchily put
Adam Weisblatt: What is the difference between a LMS and a SIS?
Cris: @Mark -- so true on feedback and LMS connection or lack thereof
Moderator (George Siemens): @Phil - reflecting on your regression vs. baysian ques - good point
Mark Gbur: LMS - Learning Management System, - used for courses
Moderator (George Siemens): but not sure how to answer that....
Mark Gbur: Student Information System - Data of students, ect
Moderator (George Siemens): statistics are great for broad level (probability) insights
Mark Gbur: Moodle is an LMS, Blackboard, ect
Moderator (George Siemens): baysian models are effective, as you noted, for in context interventions
Moderator (George Siemens): @phil - please send
ViplavBaxi: Bayesian + Cointegration : http://villani.files.wordpress.com/2009/08/bayesrefcointfinalet.pdf
Gillian: Very interesting. Thank you. Got to go...
Moderator (George Siemens): sounds like eportfolios
Moderator (George Siemens): i.e. with different security levels
Phillip Long: Oh god - please let's not go there
Cris: Can you see access across universitites for students and guidance counselors, parents, etc. can select universities for study
Cris: okay, yes
Moderator (George Siemens): hehe - but, it is a similar function though, isn't it? treating our data as portfolios with variable access rights
Mark Gbur: It is interesting, Signals now becomes kproprietary with SunGard and not stand alone but requiring integration with an LMS, and presently only integrates with Bboard? That definitely narrows the playing field for potential customers. Most Universities never want to change their SIS given all of the work. Especially given the number of individuals affected.
Phillip Long: Indeed @Mark it does.
Moderator (Sylvia Currie): Excellent questions and comments in the chat. I'll be sure to save + post in Moodle
merc√® gal√°n: can you put the last slide, please?
Adam Weisblatt: But academia has the funds for the statistical analysis
Mark Gbur: Lol but not the time or patience for change management
Adam Weisblatt: I can't see corporate investing in this without clear ROI
khopkins: Do you think this type of analytics will trickle down to high school and elementary classrooms?
Cris: Are entrepreneurs moving into K-12/
Phillip Long: Thanks
merc√® gal√°n: Thanks
Moderator (George Siemens): @Cris - yes
ViplavBaxi: Wireless Generation
Mark Gbur: Feedback and performance increases are a clear ROI, but Universities are slow moving creatures and proprietary implementations completely box out the new players
ViplavBaxi: NY Charter schools
Cris: Is anyone doing any work with analytics in virtual immersive learning environments?
ViplavBaxi: News Corp is buying 90% of it
Francisco Reis: Good question, Cris!
SuzGupta: @Cris -- like SecondLife?
Mark Gbur: Many thanks Kim - greatly appreciate your time and experience
Cris: @Francisco: Thx.
Adam Weisblatt: Thanks
Phillip Long: In the notion that this is related to ePortfolios it's important to remember point of view - i.e., who 'owns' the data and how it's presented. The Signals approach is an institutional view not the learner's view.
Moderator (Tanya Elias): Thanks so much!
Phillip Long: Thanks everyone
Barbara Truman: Thanks very much
Francisco Reis: Thanks!
SuzGupta: Thank you!
Cris: @SuzGupta -- yes and other environments
Moderator (Sylvia Currie): Thanks everyone! See you in the forum and elsewhere
merc√® gal√°n: Thank you
Phillip Long: ciao
Moderator (Kim Arnold): thanks all looking forward to future discussion!