Chat log for session with Kimberly Arnold

January 26, 2011

Andreas Link: hi

Moderator (George Siemens): http://www.itap.purdue.edu/people/detail.cfm?alias=arnold0

Gillian: Hello from UK

mercè galán: Hello again to all from Spain

Moderator (George Siemens): perfect

Moderator (Sylvia Currie): sounds good

ines cambiasso: yes, sound is good

Moderator (George Siemens): :)

Moderator (Sylvia Currie): Nice, always like to have a photo of presenter :-)

Moderator (Sylvia Currie): learning analytics, knowledge analytics... now academic analytics

Moderator (George Siemens): yes

SuzGupta: Hello from sunny Reno, NV and hello again to CCK folks. Quite the Elluminate party today :)

schawn t: how about just analytics

Vince Jansen: 2 of 3 !

schawn t: we can throw any number of adjectives in front of it we want. Really it becomes the context in which we are using analytics

ViplavBaxi: 2 AM and a second cup of coffee..Hi All

Gillian: @KIm What length is the cycle?

SuzGupta: @Viplav -- where are you?

ViplavBaxi: New Delhi :)

SuzGupta: oh! thanks for staying up to learn with us!

ViplavBaxi: wouldn't miss it...:)

SuzGupta: interesting that two here are huge for-profit online u's

SuzGupta: @Kim -- do you know of any wide uses of this in K-12?

schawn t: Does anyone know of any Corporate or Government programs using learning analytics

Moderator (George Siemens): @schawn - i'm not that familiar with corporate use of analytics in learning

Moderator (George Siemens): DoD is using learning analytics in their teaching/training

Gillian: @schawn t I can put you in touch with corps if you want.

Saleha: Has there been any work done in associating student success indicators with student learning?

schawn t: just seems as though most of the analytics (learning, knowledge) is happening more in the academic sector than anyother - true statement

Moderator (George Siemens): @schawn - corporations are, however, driving data use for decision making...

ViplavBaxi: Project TinCan and LETSI are also re-looking at stuff on the standards side

Moderator (George Siemens): @schawn - companies like IBM, in particular, are very active in data analysis and gaining insight and contributing to decision making process

schawn t: @Gillian, yes thanks. I am just curious of some examples

SuzGupta: do you have some data on that (preservation of marketing/recruiting doillars)?

Phillip Long: Pearson is one of the corporate leaders in this space, along with Sunguard ;-)

schawn t: @george - are companies using more learning analytics or knowledge analytics (maybe both) to help drive decision? I guess I am trying to see if there is a line that can be drawn between learning and knowledge analytics

SuzGupta: I would think there would be savings too if tutoring can be offered automatically online v. live tutors/centers

Moderator (George Siemens): @schawn - I think companies would be more interested in the concept of knowledge analytics

Moderator (Tanya Elias): we are working at integrating business and leanring analytics

schawn t: @george, I would agree.

Moderator (George Siemens): learning often doesn't always get high funding priority :)

Adam Weisblatt: Yest they'll spend money on scantrons which seems to me to be a waste

ViplavBaxi: @schawn t: anyone using Human Capital Management software such as those provided by SABA, or enterprise social platforms like MZinga regularly employ learning analytics with a high use of BI work

Moderator (George Siemens): Verna allee's work on "value networks" has made a fari bit of impact

ViplavBaxi: millions of variables?

Saleha: Has there been any work done in associating student success indicators with student learning?

Saleha: associating student success indicators with student learning indicators

Saleha: ?

Phillip Long: What were the principal variables in this model (independent variables)?

SuzGupta: what does "SIS" mean here?

Mark Gbur: Student Information System

Adam Weisblatt: Who on their team actually did this statistical analysis work?

SuzGupta: student information systems?

SuzGupta: thx

Phillip Long: That's another argument for an institutional academic data warehouse!

ViplavBaxi: comparative dynamics - testing the model over periods and samples - would be interesting to do

Mark Gbur: It is interesting because there are multiple CMS's and other networks at Purdue, Krannert School of Management had money and they were on their own networks so I imagine quite a bit of effort went into funneling all the data

Moderator (George Siemens): did you want to pause for questions here?

Phillip Long: Need more clarity on the difference between the larger CMS model and the course-specific algorithms

Mary 1: Good question.

Phillip Long to George Siemens, Kim Arnold, Tanya Elias, Sylvia Currie: Ironically many institutions have administrative data warehouses...

Phillip Long: It reveals what matters in terms of dollars!

Adam Weisblatt: Why is there a privacy concern is the data is being used for statistical analysis?

Moderator (George Siemens): @Phillip - agree!

schawn t: I vote for the latter

Adam Weisblatt: Why is there a privacy concern IF the data is being used for statistical analysis?

ViplavBaxi: In corporates, one of the interesting uses of analytics is to judge the performance of Learning and Development Managers

Phillip Long: Learning is a diffcult output measure - how much you spend for parking is much more direct and easier to measure and has significant annual costs

Phillip Long: (where parking is just an example ;-)

Gillian: I can see a bigger problem with privacy if a university is heavily commercial (sponsored...)

schawn t: is it because measuring performance improvement is often very complex and situational

Phillip Long: There is a significant alternate view here that you can inadvertently lead to a 'tracking' system if individual data is in any way identified

Mark Gbur: It is interesting George, you're somewhat against utilization of LMS/CMS's ie (Blackboard), but much of the data pulled here is a result of that structure and the ability to pull data from a standardized, rigorous system and databases. Without structured systems, the data would not be there and a SIGNALS system would not be possible. Where do you stand on utility v. value of the data?

Mary 1: The accrediting agencies require alignment with 'standards' and 'benchmarks', which require the creation of performance assessments and rubrics. The development of these takes an enormous amount of effort and time. Faculty have concerns about this approach to learning. Faculty are concerned about the use of student data in ways that do not take into account the conditions and the constraints.

Moderator (George Siemens): @Mark - v. good ques

Moderator (George Siemens): we are working with distributed systems for learning at AU

Moderator (George Siemens): I'm interested in how we can collect data in distributed networks (social learning)

Phillip Long: Mark - that's exactly the concern I think - contextualising performance can be easily lost in the roll-up of statistical analysis

Moderator (George Siemens): I find some of the monitoring tools (klout, radian6) to be an interesting model in that regard

Mary 1: Great point! Who wants to know. Why? So what...

Adam Weisblatt: Does anyone use this approach to measure the effectiveness of assessments?

Mark Gbur: Klout is definitely an interesting tool, I haven't seen radian6 though. I think it would be hard to itemize something from klout for performance/assessment in a course though, (at least at this time)

Richard Olsen: may have missed it but how is success measured? grades, other?

SuzGupta: @Richard -- grades I think

Phillip Long: Interesting that a stoplight is interpreted as my instructor cares about me ...

ViplavBaxi: If I remember correctly, Ryan Baker also showed us similar conclusions - students benefit from getting feedback about how they are doing...

Adam Weisblatt: Problem with corp data is that is is big (5 million transactions per year in Pfizer's LMS) but no depth (Required date, completed date)

Moderator (George Siemens): @Viplav - yes. On one level, that seems obvious, but....

Mark Gbur: (apologies @ work and I think maybe I missed something), is Signals a plugin for Blackboard or is it an open source plugin, how exactly is it being implemented and delivered to the students (stand alone v. integrated)

Phillip Long: Does this mean tha NOT enrolling in the next semester = dropping out?

Phillip Long: or just delaying

SuzGupta: would be great to see this from a cost standpoint = money gained by increased retention v cost of project

SuzGupta: or whatever the right way is to analyze the cost side

ViplavBaxi: so one application of BIG Data analytics in education would be to develop a system that could inform and counsel students in a bid to improve performance...

SuzGupta: and thus increase retention and reduce loss of marketing/recruitment dollars

Phillip Long: That system is Bb ?

SuzGupta: (lakh, whatever ;))

ViplavBaxi: :)

Moderator (George Siemens): ok

Moderator (George Siemens): :)

Mark Gbur: perfect, thanks

SuzGupta: It's Cris! <Waves>

Adam Weisblatt: The corp equivalent would be edn of year performance review grades.

Adam Weisblatt: I wonder if the approach is transferrable form acedemic to corporate

Cris: Hi SuzGupta --shhhh trying to sneak in ;-)

ViplavBaxi: Corporates would perhaps just hold up the salary raise as part of the carrot and stick implementation of elearning :)

Phillip Long: Generic question: there are concerns about using linear regression (traditional parametric statistics) for these kinds of predictive models. This is in contract to Bayesian modeling as the intention of the modeling is intervention into the same population, and, the variables involved don't often follow normal disributions. Comment?

Mike: Can you provide some more examples on how you collaborate with the student interventions amongst multiple department/stake holders. Does Signals support this?

Cris: Ethics --- "obligation of knowing" like that

Adam Weisblatt: In corp, LMS and Perf mgment system woiuld have to be connected

Gillian: Automation with adult distance learners could be a nightmare for teachers if they have to spend a lot of time online explaining the 'automaticness' and assuring students that flexibility is there really ? :(

ViplavBaxi: @Phillip: I am almost certain they would be using non-linear and time series techniques...good question

Phillip Long: Obligation of knowiing is a double-edged sword. Do we believe the model enough to suggest the student

Phillip Long: oops - continuing " at risk" should be give other options (withdrawal) or ...?

Mary McEwen to George Siemens, Kim Arnold, Tanya Elias, Sylvia Currie: What about “weed out” classes? [courses requiring at least a B grade before acceptance into highly competitive programs (e.g. computer science, nursing, etc.)] How might something like Signals be perceived for these courses?

Mark Gbur: I believe the perception is that SunGard is the 'benchmark' SIS on the market, (portal, billing, grades, course schedules, marketing, recruiting - "all but CMS") - having Signals would certainly further their reputation, I find that rather intriguing given the results presented here.
Feedback is often the most underutilized component in courses and most LMS/CMS are not even close to figuring it out.

Adam Weisblatt: Is the juice worth the squeeze

ViplavBaxi: @Phillip I think Cointegration techniques would also be very useful here

SuzGupta: @Adam -- catchily put

Adam Weisblatt: What is the difference between a LMS and a SIS?

Cris: @Mark -- so true on feedback and LMS connection or lack thereof

Moderator (George Siemens): @Phil - reflecting on your regression vs. baysian ques - good point

Mark Gbur: LMS - Learning Management System, - used for courses

Moderator (George Siemens): but not sure how to answer that....

Mark Gbur: Student Information System - Data of students, ect

Moderator (George Siemens): statistics are great for broad level (probability) insights

Mark Gbur: Moodle is an LMS, Blackboard, ect

Moderator (George Siemens): baysian models are effective, as you noted, for in context interventions

Moderator (George Siemens): @phil - please send

ViplavBaxi: Bayesian + Cointegration : http://villani.files.wordpress.com/2009/08/bayesrefcointfinalet.pdf

Gillian: Very interesting. Thank you. Got to go...

Moderator (George Siemens): sounds like eportfolios

Moderator (George Siemens): i.e. with different security levels

Phillip Long: Oh god - please let's not go there ;-)

Cris: Can you see access across universitites for students and guidance counselors, parents, etc. can select universities for study

Cris: okay, yes

Moderator (George Siemens): hehe - but, it is a similar function though, isn't it? treating our data as portfolios with variable access rights

Mark Gbur: It is interesting, Signals now becomes kproprietary with SunGard and not stand alone but requiring integration with an LMS, and presently only integrates with Bboard? That definitely narrows the playing field for potential customers. Most Universities never want to change their SIS given all of the work. Especially given the number of individuals affected.

Phillip Long: Indeed @Mark it does.

Moderator (Sylvia Currie): Excellent questions and comments in the chat. I'll be sure to save + post in Moodle

mercè galán: can you put the last slide, please?

Adam Weisblatt: But academia has the funds for the statistical analysis

Mark Gbur: Lol but not the time or patience for change management

Adam Weisblatt: I can't see corporate investing in this without clear ROI

khopkins: Do you think this type of analytics will trickle down to high school and elementary classrooms?

Cris: Are entrepreneurs moving into K-12/

Phillip Long: Thanks

mercè galán: Thanks

Moderator (George Siemens): @Cris - yes

ViplavBaxi: Wireless Generation

Mark Gbur: Feedback and performance increases are a clear ROI, but Universities are slow moving creatures and proprietary implementations completely box out the new players

ViplavBaxi: NY Charter schools

Cris: Is anyone doing any work with analytics in virtual immersive learning environments?

ViplavBaxi: News Corp is buying 90% of it

Francisco Reis: Good question, Cris!

SuzGupta: @Cris -- like SecondLife?

Mark Gbur: Many thanks Kim - greatly appreciate your time and experience

Cris: @Francisco: Thx.

Adam Weisblatt: Thanks

Phillip Long: In the notion that this is related to ePortfolios it's important to remember point of view - i.e., who 'owns' the data and how it's presented. The Signals approach is an institutional view not the learner's view.

Moderator (Tanya Elias): Thanks so much!

ViplavBaxi: thanks!

Phillip Long: Thanks everyone

Barbara Truman: Thanks very much

Francisco Reis: Thanks!

SuzGupta: Thank you!

Cris: @SuzGupta -- yes and other environments

Moderator (Sylvia Currie): Thanks everyone! See you in the forum and elsewhere :-)

mercè galán: Thank you

Phillip Long: ciao

Moderator (Kim Arnold): thanks all looking forward to future discussion!