Week 5: Organizational Implementation

Hi Kae -

Getting a sense of what kind of data is available is an important start. Thanks for listing the outcome of your exploration.

Most universities/colleges focus on institutional statistics (ages of learners, where they live, previous academic experience, drop out rates, etc).

In an educause presentation, I suggested that analytics need to be considered at five levels:

1. Course-level: how are students doing, attendance rates, are they conducting the readings, frequency of log in, use of clickers in classroom.

2. Aggregate-level: this is where we draw most of our patterns of performance. What are the actions of students that succeed? What are early warning signs that we need to intervene? Which readings and learning resources are effective? Which aren't? These results need to be fed back into the course design and teaching processes.

3. Institutional: Three things are important at this level: 1. knowing our students - where they are from, why they are here, what their needs are, etc. 2. The performance of faculty is also a consideration at this level (research impact, publications, productivity in contrast to other universities). 3. Finally, how does information flow within an organization? What are the social and value networks that contribute to effective collaboration and information sharing? This is focus on the administration of the university - understanding how work gets done and ensuring that barriers to creativity and innovation are removed.

4. Regional: state/provincial/institutional comparisons. Analytics at this level assist funders/politicians/decision makers/bureaucrats to understand how different universities are performing in relation to others. Where are costs high? Productivity low? Why do some universities produce significant research while others languish? Who is using public funds "well" and who is not making an impact, etc.

5. National and international: league tables and aggregate comparisons of universities/colleges occur at this level. Obviously ranking highly is a huge benefit to universities for attracting students...but these comparisons are also important for determining which countries and regions are transitioning to a knowledge economy based on how universities perform internationally by research, patents, etc. These comparisons are controversial...but are increasingly being used by decision makers in setting goals for "world class universities".

I've thrown out many subjective terms (productivity, teaching well, creativity, and so on). Each of these is worth debate. For example, what does it mean to be a good teacher? Or a productive researcher? These are not easy questions. But answering them isn't my main concern in this post. Instead, I'm trying to emphasize the different levels of analytics use in higher education.

What have I missed?