Footprints of Emergence: Nov 18-29, 2013

Assessing Emergent Learning

Assessing Emergent Learning

by Jenny Mackness -
Number of replies: 48

The second question which might be interesting to discuss is:

Is it possible to assess emergent learning? How do you 'capture' learning that is not expected? How do you measure or value it? Are these the right questions or are they flawed?

We would welcome your thoughts about these and questions in the other forums.

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Dilip Barad -
These are quite relevant and interesting questions.
In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Jaap Bosman -

Student and teacher and peers need to look out for Emergent Learning artefacts. When someone produces an unexpected answer or learning product we mostly do think it a mistake or a false answer. We need to be carefull to call unexpected learning artefacts false or wrong. Artefacts = products of the student as a result of work in a course. 
Once we do recognize an artefact as a product of EL we could assess it. 
To assess these EL outcomes we need to use very broad objectives. The assesor needs to be an expert or a forum of experts in the field to recognize the EL outcome as useful and valuable. The expert is needed, because courses mostly do not use these very broad objectives and goals. 

Would it be possible to ask for emergent learning as a result of a course? Would EL be expetected when a student is doing synchronously  two courses in different fields? 

 

In reply to Jaap Bosman

Re: Assessing Emergent Learning

by Jenny Mackness -

Hello Jaap - great to see you here. I like the idea of Emergent Learning Artefacts, which could then be assessed. It makes me wonder if the arts subjects lend themselves more to this than other subjects.

I am thinking here of one of my sons who has just completed a Masters in Music Technology. I think there was lots of emergent learning in that course because the students decided on their own research projects, which ultimately, in their final performance, had to demonstrate the knowledge and skills they had acquired over the course.

I went to see the final performances and they were all completely different. The tutors couldn't have known before hand what to expect - but presumably they had some sort of rubric to assess these performances.

In reply to Jaap Bosman

Re: Assessing Emergent Learning

by Kathleen Zarubin -

YES .... artefacts or 'evidence'  of something 

on which some kind of assessment judgement can be made -

& then compared to 'something else' (benchmarks / standards / required outcomes ????)  

 

This is the concept and process of 'assessment'  that I find absolutely FASCINATING  and really really important & valid & valueable for some reason ... 

In reply to Jaap Bosman

Re: Assessing Emergent Learning

by Scott Johnson -

Hi Jaap,

Does emergent learning have to produce something unique or odd? If the artifact appeared out of place we might ask for an explanation that would show a new twist in logic or viewpoint. Even the obvious needs to account for itself but the oddity might be a richer source of explanations.

In reply to Scott Johnson

Re: Assessing Emergent Learning

by Jaap Bosman -

Hi Scott, Good Question. The difficult question is What is New or what is Unique. The teacher should know what is new for the student, for that is what is important. New and unique are personal subjective terms. 

So if a little boy is gaming about the Roman Empire and while playing is  learning  historecal and cultural knowledage in the game than for this little boy  it is New Knowledge. 

 

In reply to Jaap Bosman

Re: Assessing Emergent Learning

by Scott Johnson -

Jaap,

I would think all learning is personal subjective to the person who learns. The boy playing the game is operating (in his mind) the conditions of play through a series of understandings of what is expected next. His brain is also constructing new or deeper understandings of the complexity of the subject along with the enactment / expression of more mature strategy of play. Something is developing here that is embodied within the boy, not the game or his teacher.

Better play patterns demonstrate change in strategy and maybe New Knowledge is included and I'd say the teacher wasn't involved it this beyond providing access to the game. School directs material judged useful by the culture it springs from to a student in a form determined to be most easiy metabolized into a thing called learning. All this occures at the surface.

How do we get to the person as product of themselves over person as "product" of education. Before we claim that something we did caused learning we need evidence they were listening to us. Would this be an Artifact?

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Peter Rawsthorne -

Is it possible to assess emergent learning? I would thinks so... but why? What is the purpose of the assessment? To provide meaningfull feedback to the learner? or to become a part of an assessment for coursework or progress toward a learning goal / objective. I think with emergent learning this is an impoartant distinction which will assist in answering the subsequent questions.

How do you 'capture' learning that is not expected? Capture?... if the person has learned it... they have 'captured' it. How is unexpected learning recognized by both the learner and others... I believe through reflective activities and writing or creative works that externalize the learning. If an artifact of some kind can be created that includes, and identifies, the unexpected learning will have been 'captured'. I do believe having the ability to be mindful and self-reflective is very impostant.

How do you measure or value it? Why the need to measure? This goes back to my first question about why assess. In could be measured via the person shows increasing mastery of the subject domain, peers and others also recognize the mastery. But if it was completely out-of-the-blue emergent learning the measure would come from the depth of self-reflection and subsequent artifacts. The value comes in a progressive or transformative way... if learning has an impact and increases a persons knowledge, or progress toward a learning goal, or helps them feel more aware in a new subject domain, etc... it has value. Again, why the need to measure? A more important question IMO.

Are they the right questions or ar they flawed? absolutely! ;) ...

Not flawed and are a 'right' question. I think we need to get better at honouring all learning. emergent, transformative, formal, informal, etc... I don't think the assessment is nearly as important. People learn many things in many ways, and just because it can't be measured doesnt mean it didn't happen. The need to assess is flawed.

My son who was in K6 school on west coast of Canada was always approaching expectations according to thier approach, assessment techniques, curriculum. (he was losing confidence) Now on the east coast of Canada he has become star student according to a different set of expectations... the child didn't change, the assessment did. Assessment is flawed, we need to honour everyones learning, and how they learn, their pace, their depth... we need to encourage people to self-direct their learning... When we stop assessing to a framework or curriculum everyone becomes a star, become confident in themselves, a beautiful thing to see. Assessment ruins lives!

Don't get me wrong we need assessment when training surgeons. But most of the time we should do away with assessment.

In reply to Peter Rawsthorne

Re: Assessing Emergent Learning

by tony cairns -

I agree with Peter Rawsthorne


Is it possible to assess emergent learning?
 In science the testing fixes and creates the product, process and phenomena. So testing actually makes it so. Do testing to see if the circumstances, teaching learning is effective if only to learn how to teach and learn better but it’s a snapshot imperfect tool that excludes more data than it captures. The effect of testing may create the phenomena, will influence the phenomena and may destroy the phenomena rather than reinforce it. it certainly closes out other options.

I think self-assessment, peer assessment and digital tracking without intervention may be valid but testing may not tell you what you think it is telling you.

How do you 'capture' learning that is not expected?

I think all learning, discovery and original science is unexpected - if we expected it - it wouldn’t be unexpected. I think it’s hard to know what students and teachers, know, learn and think. Memories are made up of dream fragments that we assemble on the fly so there is no future or past just a present becoming and making both through making meaningful stories. If there is a past then there is no reason why it isn’t the future and vice versa i.e. they both plot, progress and proceed the same. i think it’s all down to choices we make on a moment by moment basis at a nanoparticle, micro and cellular level - most of these cells are not ours but bacterial or fungal or even viruses. So we are a sort of a multispecies hive or ecology lurching into other ecosystems in both time and space. Capture is the wrong word, track, discover, manifest, measure, assess, 

How do you measure or value it? 

If you measure it then start with self-assessment through reflection, journaling and meta data analysis, move to more interventionist approaches but avoid detection of assessment, videoing simulating and drama. Finally if you must invent a totally artificial paper based time resource support and location limited test based on comprehension, writing, numeracy and scientific literacy skills and inflict that on all students of a certain age based cohort on an annual basis and tie careers, lives and income to such spurious measure at your and their peril. But don’t fool yourselves, students, managers, bosses assessors and professional peers and colleagues that it is anything more than your ability to create a newer swifter more attractive mouse trap – ie mouse dead - cheese safe

Are they the right questions or are they flawed? 

Assessment is more for the teacher’s managers to rank and negate/reward teachers. They interfere with learning and become the purpose of the process. Teachers teach to the test and to the assessment questions – they forgo learning and teaching and replace with cramming, revising, reiterating, repeating, replicating rather than discovering, exploring and learning. Learning is replaced with grading. It is part of but not the reason for education it seeks to measure but there is no transferability, objectivity or point to it apart from ranking students and teachers and making rewards and disincentives easier to administer

The problem with not assessing brain surgery is the dead bodies of those that did not adopt the strategy that resulted in the best outcomes for the patient, family community and professions – both surgeons and patients – but few things are as extreme as this – most are incremental and not mission or survival crucial. Also brain surgeon’s and surgery changes over time from shamans to trepanning, to laser surgery and so should assessments and processes – robotic surgery and assessment simulations are now coming into focus

In reply to tony cairns

Re: Assessing Emergent Learning

by Jenny Mackness -

Wow - this is quite a forum thread - such a lot to think about and digest. Thank you Peter and Tony for your thoughful responses to the questions I posted, which seem to not only align with each other but also to the posts of others.

I think the key points that have come out of all these responses for me are:

- we should honour all learning, not only what can be assessed,

- capture - is probably the wrong word to use in relation to emergent learning.

- assessment that is aligned with emergent learning will probably need to focus on self-assessment and reflection

- and finally - a thought that has come through this for me - and which has cropped up before from time to time - is - does the act of 'measuring/assessing' destroy what it is trying to 'measure/assess'?

So, by considering whether to assess emergent learning, are we in danger of destroying possibilities for emergent learning? Is this what you were getting at Peter when you asked why we should assess emergent learning?

And if we don't try to assess it - how can we ensure, in our current educational climate - that it is valued and encouraged? I think some of the answers are in the posts already made, but I am just marking this question.

In reply to tony cairns

Re: Assessing Emergent Learning

by Phillip Rutherford -

First up I must say that I am excited to find this group. I first came across the concept of emergent (and transformative) learning while researching for my Ph.D thesis. I have progressed my thinking and experience a long way since then but have found very few thinkers who really understood the question of emergent learning (much less understood the answer).

Nevertheless, I am responding to this point so had better move along.

One of the issues I find is that so much of learning has attempts made to measure and evaluate it by the training/teaching which went into it. Unlike measuring the speed of a car by how much pressure is placed on the accelerator, learning cannot be measured by how much or what is taught. It is a phenomenon which occurs within a context and environment, much - most - of which cannot be influenced by the teacher, or even by the learner. So attempting to measure learning by what a teacher or trainer does is fraught.

Emergent and transformative (if it is to be real learning and not simply responding to a stimulus) learning finds its roots in the theories behind complexity and chaos. It is these concepts which, when underpinned by the studies into neuroplasticity, gives us a clue as to how the environment can be shaped in order to generate emergent learning.

If the learning is truly emergent and transformative it cannot be by definition be measured and evaluated. Therefore, it is my contention - drawn from a lot of years actual experience - that what should be assessed is not the learning but the learner's reaction to the environment and how his/her knowledge is shaped as a response.

 

In reply to Phillip Rutherford

Re: Assessing Emergent Learning

by Jenny Mackness -

Hi Phillip - many thanks for all these ideas. I am interested that you mention studies in neuroplasticity. Do you have any more information about this you could share?

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Phillip Rutherford -

Jenny,

Try the work of Dr Norman Doidge (author of The Brain That Changes Itself) or Barbara Arrowsmith-Young (author of The Woman Who Changed Her Brain). Very interesting research into how the brain can be trained to allow for new ways of learning.

 

In reply to Phillip Rutherford

Re: Assessing Emergent Learning

by Nick Kearney -

It is hard to put a finger on what the "learning" is anyhow, but moving beyond that Philip what you are doing is to move the goal posts. Shame on you!!! :)

The issue about emergent learning (or whatever you want to call it) is that it escapes the prior definitions we have worked with in the field. There are three reactions, and they are present in this debate.

One tries to lassoo the coltish concept and drag it back within the fold. Once the emergent learning generates evidence it becomes manageable within a system that fails fundamentally to trust the individual (this is so ingrained we mostly dont even notice it).

Another notices emergent learning as an interesting anomaly, something worth studying, and of course measuring, and scoping and observing. Welcome of course, but I dare say, eternally marginal.

Then there is the view (not a new view) that understands emergent learning, once it is recognised as existing, as a fundamental and profound challenge to the way our society understands learning, knowledge and socialisation. If you recognise it you have to rethink education.

In this rethink as Philip puts it, it is all about the learner and her reaction to the environment, that interaction is central. Schools are a relatively limited part of it, and most of the influences that shape our society could be construed as emergent (though in many cases not accidental). There is an urgent need to address the issue.

I would be interested in building footprints of emergent learning experiences around events such as any episode of  the X factor :) or CNN news.

 

In reply to Nick Kearney

Re: Assessing Emergent Learning

by Phillip Rutherford -

Moved the goalposts or learned that they were actually somewhere else? :-)

You are so correct - emergent learning does escape the 'prior definitions'. It is, after all, emergent  :-)

Seriously, I believe that emergent learning is un-lassoo-able (if there is such a word). The concept, perhaps, but not the phenomenon. IMHO for too long educationalists and training folk have tried to box learning in so that it could be controlled. But learning is continuous and happens without us even knowing. We see directly ahead but our conscious is scanning what is happning to the left and right of us. All of that is mixed with our prior experience and learning to contextualise and create what is happening to us. This is why learning is such a fluid, individual thing.

It is also why so many students learn very quickly what they must do to satisfy the examiner, and not necessarily their own curiosity. It is also why, in my experience, there is a real need to adopt your third point - the need to recognise the truths about learning and rethink education.

 

 

 

 

In reply to Phillip Rutherford

Re: Assessing Emergent Learning

by Kathleen Zarubin -

I so so agree with you Phil re: It is also why so many students learn very quickly what they must do to satisfy the examiner ... 

in fact in the recorded webinar I saw someone write in the chat ....  (words along the lines of ..)

...  Students say ...  "just tell me what to do" 

 

I have said this myself - or even more overtly ...  'LOOK - tell me what you want &  I will give it to you..."  

Especailly when I am doing some 'learning' because I have to ..   in fact I often go straight to the assessment requirements and begin 'there'. 

I am not saying this is 'good'  (or even bad I guess) - It is what it is - especailly  when Person X holds the 'ticks'  and I NEED x amount of 'ticks'  to get Z outcome (almost always a bit of paper)  so I can (most often) give it to Authority Z  ...

So (most often) Authority Z can put a really BIG TICK against my name (and then everyone is happy :) 

I know it sounds cynical and it is, I guess.  In many ways I am sad to say Uni taught me this the most.  (but to be fair that was a long time ago & I am sure things have changed now ) 

and also - please don't get me wrong.  I was the FIRST in my extended family to even go to Uni -  complete with a number of people in the street to see me drive off on my first day .....  and I do have much 'fondness' for my degree and I really did learn how to play  the game ...  a skill which has helped me in life and one I did overtly encourage my children to learn as well ......  (AND I also did learn other things ) 

Maybe a classic case of Emergent Unplanned Learning -  unassessable (except for the fact I got 'good' grades - even from the 'hard markers' lol ) - but valuable learning for some situations nether-the-less :)    ??

In reply to Kathleen Zarubin

Re: Assessing Emergent Learning

by Joyce McKnight -

Kathleen: From becoming acquainted with you in other venues, I would say you play the game well indeed and still manage to learn a lot and connect many diverse ideas well so it is possible to play the game, get the "ticks" and still win...too bad all the "ticks" often discourage folks.    I too have gone for some big "ticks" next to my name, but mostly while I was learning the really important things in life elsewhere and sometimes actually getting to apply that "real" learning to the "tick collection"...the real trick is to somehow find ways to get both "ticks" and satisfying learning...those who do have a right to feel smug about it!  :-)

In reply to Phillip Rutherford

Re: Assessing Emergent Learning

by Pat Tymchatyn -

I would agree with you Phil - "just tell me what I need to do . . .".  This is not only the attitude of students but professors - rubrics so that they can quantify the learning in some way and they tell themselves they are moving from subjectivity to objectivity - and what happened to expert opinion?  HOw to get to the "rethinking of education?"?

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Deirdre Bonnycastle -

As long as we think of assessment as standardized testing on factual knowledge, emergent learning will fail the test. I see this issue regularly in medicine where we need a fine balance between anatomical knowledge and diagnostic reasoning yet continue to use MCQ's as our testing tool.

I agree with Peter, narrow objectives can only be assessed using highly specific tests and leave no room for moments of illumination. So was the objective for my granddaughter "to cast on 50 stitches" or was it "when faced with a new situation to create a way to participate in the process."

I've been looking at portfolios and the rubrics used to assess them lately. I always ask for a bonus area for students who demonstrate learning something new or unexpected.

In reply to Deirdre Bonnycastle

Re: Assessing Emergent Learning

by Jenny Mackness -

Hi Deidre - it's interesting that you bring up medicine as a subject. We have often discussed whether medicine, nursing, teaching (i.e. those subjects in which what people learn can have such a profound effect on other people's lives) are subjects which need to be prescriptive. They certainly seem to be subjects in which there is a lot of standardised testing.

Is diagnostic reasoning the same as emergent learning? Can we afford to have our doctors' knowledge be emergent as they practice on us? I'd be interested to hear what you think.

I like the idea of adding a bonus area to a rubric to allow for emergent learning.  

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Barbara Berry -

Hi Jenny and All,

Jumping in again (juggling meetings etc.) but really want to engage in the conversation. Forgive me if I repeat (hard to read all of the posts in a timely fashion. I clearly need more practice at managing my "emergent" learning : )

I love your questions on "assessing emergent learning" as they are provocative and thus are assisting me to consider the possibilities, my own bias and assumptions and experiences (in prescribed and emergent scenarios). I am leaning towards "no", it is not really possible to assess emergent learning but what might be possible is to assist individuals to develop their skills of self-awareness, reflection and paying attention to whatever is "emerging" on the periphery or perhaps centrally (even in the context of highly prescribed instruction). 

In the context of health sciences and medical/nursing education (my initial background), there are legal requirements for licensing and thus prescribed content-heavy curricula but to practice "safely" one has to be learning in situ (with real patients - in my day and now it's often simulations). We also were taught to "pay attention", be alert for stuff that might not be obvious but that would influence practice. This was practiced in scaffolded situations until such time as one could operate "safely" without the clinical expert being there. As I recall (this is historic for me now after all of these years) is that I learned about myself and my own capabilities (strengths and weaknesses) to think, practice safely, be empathic etc. I also learned what I liked and didn't like about this work. These two are examples of what I might consider "emergent" learnings and while there was no way of assessing these I was assessed on my ability to be "reflective" and in practicing reflection, I became aware of what I was really learning about myself in this context.

it's only one example and I might be off base.

all for now, 

Barb

In reply to Barbara Berry

Re: Assessing Emergent Learning

by Scott Johnson -

Hi Everyone,

Interesting that medical education comes up. My sense is the environment that medicine is in is essentially emergent. Constantly readjusting by responding to surprise, rotating teams with different strengths, learning through a kind of open strategy of referencing, learning, trying, failing, re-referencing, learning, trying....

Fine art training is like that and the assessing is no less genuine than a deliberate and thoughtful attempt by someone with high standards for themself. Of course we want things to work without fault but our best chance of that happening is to put someone with pride in their practice and then back them up. Too easy an answer though.

Have a quote to insert here but don't want to plug up the discussion area. There's a wiki?

Loved the medical education website and will pass it on.

In reply to Barbara Berry

Re: Assessing Emergent Learning

by Jenny Mackness -

No- not off base Barb. That's a great description of how emergent learning and prescriptive learning rub along beside each other, even in situations where you would expect prescriptive learning to dominate.  A really helpful example. Thanks

In reply to Barbara Berry

Re: Assessing Emergent Learning

by Joyce McKnight -

Barbara:  I very much agree with you.   I think that most professions (and most disciplines for that matter) begin with a common core of knowledge, skills, values, techniques, and often vocabulary that are crucial in enabling practitioners to work together and helps increase the sum of human knowledge.  Plain, old-fashioned rote learning is often the best way to begin, then the scaffolding experience that you so eloquently described takes shape as one listens, learns, and practices under experts' careful attention...then you become a practitioner yourself, a level where many people remain for their entire lives, doing useful but not especially creative work, but the person who becomes a master healer keeps reflecting, learning, trying new things, observing, bringing in ideas that may seem to come from "left field" ...but is always aware of the ongoing dictum..."first of all do no harm"...and occasionally circles back to basic principles to make sure s/he is on track.

I also liked your thoughts on the ways emerging learning is really about learning who you are (and who you are not) as a practitioner and how structured reflection as is often required in medicine and other helping professions can be useful in this process.

By the way, we seem to have shared a somewhat similar journey...although I started out as a mental health professional not as a medical person.

 

In reply to Joyce McKnight

Re: Assessing Emergent Learning

by Phillip Rutherford -

Joyce,

I think you make some very good points and with your indulgence would like to address two of them.

Regarding rote learning, neuroscience is now discovering that using the brain in a single, consistent manner (such as memorising and repeating facts in a repetitive way - eg, 'times table', movement of a limb) can increase one's neural flexibility. I know this is contrary to the position many teachers take on rote learning, but studies into neuroplasticity are revealing the positive aspects of this.

Regarding your comments about increasing the sum of human knowledge, I think that when we discuss emergent learning we cannot avoid discussing also the purpose of such learning - and that is to turn information into knowledge in order to better understand the world or various elements of it. My research and continuous observation suggests that there is three broad domains which span the continuum between the Kantian view of knowledge and that of Hegel.

At the Kantian end there are those who view knowledge as static and 'truth'. They adopt a long learning/feedback loop in order to bring together all of the information in a structured and stable way. They don't see knowledge as context-specific, more truth as its own argument, and give greater concentration to getting the information right. Their aim is for accuracy of knowledge.

At the other end are those who look at information in context in order to build a platform of knowledge from which to launch further discovery - often through trial and error. They apply a short or single learning/feedback loop and base decisions around limited input - sometimes no input whatsoever. They apply creative chaos to knowledge in order to keep it alive and dynamic, and by doing so reveal information which might not have otherwise been revealed. The ideal at this end of the continuum is to minimise risk and avoid errors.

In between the two are those who use information to make sense of either 'truth in itself' or 'truth in context', and apply medium learning/feedback loops in order to position their knowledge and understanding between the two poles. They are happiest questionning wisdom, not for its own sake but in order to better place it in either the Kantian or Hegelian camp. And having done so they apply knowledge gained at either end of the continuum to gain a better understanding (eg, applying creative chaos to 'single source of truth' in order to test this truth, or adopting a more corporate view of dynamic knowledge in order to slow down the rush of knowledge in order to determine usability.

The first group 'accepts truth', the second 'creates truth', while the third 'interprets truth'.  

This continuum could also be seen as that which stretches between stability at one end and chaos at the other. In between is complexity, and learners could be anywhere along this continuum depending on whether or not they are clear on the information and its context. Moreover, any one of these could become a master in their chosen field but, as you state, this can only occur when the learner understand who he/she is and accepts their inherent and preferred way of dealing with information as it becomes knowledge and understanding.

 

In reply to Joyce McKnight

Re: Assessing Emergent Learning

by Barbara Berry -

Hi Joyce, 

Yes, you are correct and we do share some common insights about emergence as a part of the development of both practical knowledge while undertaking work and that it must be continuous and intentional if it will amount to anything. For me there is a creative dimension in the sense that to interrogate practice one must be willing to delve for truth and this often takes another to listen, to ask questions and to work through the process of reflecting in and on action. Sometimes it happens when you least expect it, at other times, I have found this practice only when push comes to shove and one has to reflect for one reason or another. 

My undergraduate degree is in Nursing and I worked in clinical practice (intenstive care, surgery and pediatric surgery) for a total of 6 years then in public health for another 5 followed by the emergence of my consulting practice after my graduate degree (still in the health sector). 

Depending on the practice context, clinical reasoning in my experience takes place in association with the practice of dealing with the patient or client or family or community. It is a complex, reasoning process that entails inductive and deductive reasoning happening at the same time and sometimes very quickly in actue settings. What is interesting is that good clinical reasoning includes "critical thinking" and this includes rigorous habits such as analysis, inferential, evaluative, predictive, and explanatory thinking all in order to make sound clinical decisions in context.  The intent is to resolve the particular clinical problem and treat/care for the patient/client as said with their safety uppermost in mind. Clinical reasoning is highly intentional processing and done by an individual and in certain situations by a team who "reason together". There is an "emergent" dimension to some situations in clinical practice where the patient's condition is in constant flux and responds based on judgements that are being made almost simultaneously. It can be a full-body experience in the sense that after all is said and done, a person can feel emotion, physical, cognitive and social exhastion. (no wonder I used to go home and collapse!). I am sure you have seen and or experienced this in your own professional work. I think that emergent learning (as I am learning about it in this seminar) can happen at the same time as clinical reasoning but how this all "happens" together and in fact how it might be similar to but different from clinical reasoning as I experienced it, I need to give more thought to! It has been years since I have "recalled" this kind of "work" and so, now I am once again curious : ) 

cheers, 

Barb

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Deirdre Bonnycastle -

Occupational training programs are their own kettle of fish because there is a large amount of absolutely essential memorized content and skills that must be learned to a point of automaticity.

There is also an apprenticeship component where you work in real or simulated situations with an expert in order to learn to think like an expert. This is where emergent learning makes an appearance.

Diagnostic Reasoning requires taking the patient symptoms, matching this to what you know about body systems and coming up with a couple of predictions about what is wrong. Clinical Reasoning takes the predictions, and refines the diagnosis through patient history, physical examination and tests, then determines treatment. Compare, contrast, research, analyse, identify systemic problems, juggle multiple factors are important strategies that must be developed in medicine.

You learn to do this by seeing a wide range of patients over time but you also need a constant feedback system from patients, nurses, preceptors and self reflection. This is where training programs often fail their students. Poor feedback loops demoralize on one hand because of their severity and allow negative behaviour to continue on the other extreme. So formative assessment is a critical element. Summative assessment in occupational training is usually done externally and is a client safety step that confirms this person is qualified to practice.

So you have the prescriptive classrooms, emergent clinical experiences and the chaos of the unsupervised experience, all existing  and clamoring for more time.

In reply to Deirdre Bonnycastle

Re: Assessing Emergent Learning

by Scott Johnson -

Hi Deirdre,

Having been diagnosed by 5 different doctors and a specialist over a period of 5 weeks based on the simple fact that no one bothered to read further back on my record than my last hospital test, I think we can add the power of proper listening or observation to the things medical practitioners should know. Humans make mistakes and when they build simulated environments like schools they rob themselves of the juicy details of reality in trade for the convenience of rightness, prediction and further simulated performance. On the belief there are people out there so well trained they can imagine reality into existence we follow their proofs and not the reality we are presented.

To me, “qualified to practice” is a comfort (or maybe an approximation of a comfort) but as an assurance that the system behind it functions properly is not good evidence. By saying that I know there are many things to learn and many people more qualified than myself to learn them from. Yet this doesn’t diminish my ability process the world as I see it. Though of course we don't ask the receiver--we test them.  

In reply to Scott Johnson

Re: Assessing Emergent Learning

by Scott Johnson -

Correction: should had said "mis-diagnosed" though I'm sure everyone got it.

Diagnostic thinking can go beyond projecting specific chains of cause and effect onto to something to explain it. In the sense that dissonance illustrates something being out of place, we could measure novelty by noticing one part fails to be explained by those around it. That part should show as a mistake or a misinterpretation and could as well indicate emergence? Do emergent thoughts need to be novel and out on their own? If they slipped in as connections between things we knew but previously couldn't connect they would not necessarily be noticible.

In reply to Scott Johnson

Re: Assessing Emergent Learning

by Joyce McKnight -

Hi Scott:  I am sorry for your unhappy experience with the medical system.  I had a similar one in 2011 that nearly killed me and from which I am still recovering...mine had to do with the unfortunate tendency that modern medicine (at least modern Western medicine) seems to have developed to "play the numbers"...one "can't" have something because it is rare, therefore one can't be tested for it because it is "rare", and, of course, it is "rare" because it never shows up in tests!!!    My particular example is hereditary hemochromatosis...the tendency of my body to collect too much iron over the decades and the most common life threatening hereditary disorder found in people of nothern European decent.  I nearly died before a wise physician's assistant thought to check my iron levels which were 20+ times the normal level.  I tell this story to emphasize that all medical practicitioners need to observe and think, not just play the numbers...and because I feel folks need to know about hemochromatosis which manifests itself with arthritis of the hands and feet, feels a lot like fibromyalgia, kills vital organs like your pancreas, liver, and eventually your heart and is kept in check by old fashioned phlebotomies...other than the arthritis it shows up most commonly as middle age onset diabetes which by the numbers (again) is most usually called Type II diabetes and blamed on lifestyle especially being overweight.  In fact, in the US endocrinologists aren't even allowed to  order the relatively inexpensive blood test to rule out hemochromatosis because the numbers "show" it is so rare.   At any rate, Scott you are so right!  And folks with northern European ancestry...if you have some or all of the symptoms please don't be afraid to bring the possibility up to your health care providers..

In reply to Joyce McKnight

Re: Assessing Emergent Learning

by Scott Johnson -

Joyce, your situation sounds similiar to mine in it being a balance of a medical problem that can kill you combined with sloppy diagnostics. Long story short, the take away for me is that mastry of anything is useless without the willingness to listen beyond the voice of your well-trained brain.

The doctors and the specialist I saw are not unskilled but they made a decision on what to treat me for based on the first condition which satisfied their incomplete investigation. To say we train people isn't enough. We need to impress on them that neither the jolly endorphan rush of being right nor your degree makes your decision correct.

Feeling like you know is a very attractive sensation. It seems the more training a person gets the more suseptable to this distraction a person becomes. This undoes the whole concept of "mastery" for me and suggests we should look for characteristics of the search for "knowing" in the uncertain habits of the beginner. Habits that my be emergent.

In reply to Deirdre Bonnycastle

Re: Assessing Emergent Learning

by Jenny Mackness -

Thanks Deidre for explaining where you think emergent learning fits into relation to medical situations. I like the idea of emergent experiences and I'm now wondering how they differ from emergent learning. Stil thinking..... :-)

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Jaap Bosman -

Hi Jenny, Deirdre, 

In my view the Emergent Learning Artefact  (I did mention this Artefact somewhere else in this forum) is a proof or evidence of this emergent learning of this student. You (Jenny) are right I think, the Emergent Learning Artefact does fit into the bonus area of the rubric of Deirdre Bonneycastle.
Great idea of Deirdre to make a special room for the unexpected. If we do not expect that some unexpected thing could be happening, we will not notice it.

(this is acomment on my blog http://connectiv.wordpress.com/2013/11/21/emergent-learning/ 

In reply to Jaap Bosman

Re: Assessing Emergent Learning

by Nick Kearney -

what happens when the "emergent learning artefact" is a behaviour, or an attitude, or something so ingrained that the artefact is the learner?

to me the idea of evidence in relation to emergent learning is problematic

it feels like a parent saying to an adolescent "so you fell in love, prove it"

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Nick Kearney -

Assessment of learning, as the term (and the practice) is currently understood,implies a set of objectives that the teaching/learning process is designed to achieve.

Emergent learning will not be assessed, nor can it be assessed since its very nature implies an absence of clear objectives that drive the process.

Thats the easy answer. But in fact frameworks and contexts can be designed in such a way that learning or learnings can emerge, (the young chimp mentioned earlier may be a very interesting example of this, it would be useful to unpack intentionality there) and the space between emergent and planned learning can be conceived of as a slow cline. Different assessment mechanisms can be brought to bear, and instances of emergent learning can be placed under the light and examined, by the self and others.

Perhaps a useful idea here is ipsative assessment. Measurement is done against the benchmark of the self. Value emerges quite simply.

Best to all

Nick

In reply to Nick Kearney

Re: Assessing Emergent Learning

by Phillip Rutherford -

Nick - I agree. By adding intentionality to learning you are, by definition, creating a space where learning is no longer emergent.

I prefer the notion of assessing learning against self, but how could this work from a teacher/trainer's point of view? My experience has been to not focus on the learning but on the environment in which learning takes place. Others have spoken of complexity and complex learning environments so I won't reiterate them here. Instead I will say that I set the parameters for the learning - that is, the tools to be used (eg "In teams/individually, how many uses can you find for a paper cup?"), and milestones (eg "Must include at least one colour").

Undertaking a Ph.D is a good example of emergent learning. The individual sets the question (and quite often changes it as new knowledge is formed), identifies what is known, and sets out to bridge the gap between the two.

This concept has many benefits outside of the classroom. I am currently working on a major government project which, in simple terms, seeks to create knowledge as we are, er, creating knowledge. In other words, we are building the bridge as we are crossing it.

 

In reply to Phillip Rutherford

Re: Assessing Emergent Learning

by Nick Kearney -

In other words, we are building the bridge as we are crossing it. In my doctorate I felt like the coyote most of the time, running across thin air (Where's that confounded bridge!!)

I have worked in a lot of fields, everywhere everyone is driven to innovate, to build that bridge and cross it at the same time, except education.

Ipsative as an educational innovation (in the public arena) seems weird, too resource intensive in times when education is not apparently worth the candle. But it could work in education, if teachers had manageable class sizes. That works, I have seen it work.

In reply to Jenny Mackness

Re: Assessing / (meauring? reporting on?) Learning

by Kathleen Zarubin -

One thought I have is that 'learning'  overall is about some kind of change .... 

 

So maybe somehow the question is ...  Can the Change be 'described' / measured / reported .... 

just a broad thought ??? 

In reply to Kathleen Zarubin

Re: Assessing / (meauring? reporting on?) Learning

by Nick Kearney -

I would venture that all learning involves a transformation of identity. As it is mostly gradual this is not sufficiently recognised. Take a moment to think about it in your own terms.

Then think about the absurdity of assessing that change against externally imposed criteria. Who are you working for when you do that? Whose agenda?

Education should provide frameworks for emergent learning, In fact a curriculum is precisely that, it sets a framework. The problem is the belief that only one emergent result is valid. And this belief stifles identities.

Ipsative assessment approaches are arguably the only truly democratic way to go about assessment. They are the only way to respect individual freedoms and identities. However they require a lot of rigour and reflection to work (let alone to answer the critiques of the usual social engineers) and our systems are not used to that. But it is valuable to try.

 

In reply to Nick Kearney

Re: Assessing / (meauring? reporting on?) Learning

by Phillip Rutherford -

Agreed - but ipsative is 'improvement on'. It doesn't measure emergent unless you're arguing that emergent is from one level to another. Emergent learning may occur as a result of serindipity, or from an area not previously assessed or quantified. It may even progress from stability to complexity thereby implying the learning is retrograding (eg, going from knowing everything about a particular subject to knowing little about the next iteration of that subject - such as during the research phase of a Ph.D).

Personally I prefer an assessment of the use of tools and artefacts than on what is achieved through their use.

 

In reply to Nick Kearney

Re: Assessing / (meauring? reporting on?) Learning

by Roy Williams -

Nick, precisely

If ... all learning is about transformation of identity

Then ... the learner

In reply to Jenny Mackness

Re: Assessing Emergent Learning in enquiry

by Jaap Bosman -

In my country (adult) students could  ask for a kind of enquiry on their acquired knowledge and skills. This enquiry tries to measure their learning in the workplace or elsewhere. School does this enquiry and the student gets certificates for things learned when NOT in school. Students can prove their learning by telling about work experience, and all kind of proof.

Emergent learning results will be noticed by the teachers doing the inquiry. These teachers know the standards. 
Most cases school will advise the student to do some extra courses to earn a whole set of certificates.. 

In reply to Jenny Mackness

Re: Assessing Emergent Learning/ Our Questions

by Jenny Mackness -

It seems that these initial questions have sparked off a whole load more! Following Sylvia's advice (thanks Sylvia), I thought it might be helpful  if I gathered them together - so see the list below - which might help us gather our thoughts. We seem to have moved from thinking about whether we should assess emergent learning to wondering whether we should completely rethink education :-) Wonderful! Thanks to all.

A new question: Do you think we have missed any significant questions in relation to assessing emergent learning and if so what might they be? 

  • Is it possible to assess emergent learning? How do you 'capture' learning that is not expected? How do you measure or value it? Are these the right questions or are they flawed?
  • Would it be possible to ask for emergent learning as a result of a course? Would EL be expected when a student is doing synchronously two courses in different fields? 
  • What is the purpose of the assessment? To provide meaningful feedback to the learner?
  • Why the need to measure?
  • Is diagnostic reasoning the same as emergent learning? Can we afford to have our doctors' knowledge be emergent as they practice on us? 
  • Does the act of 'measuring/assessing' destroy what it is trying to 'measure/assess'?
  • One thought I have is that ‘learning’ overall is about some kind of change .... So maybe somehow the question is ...  Can the Change be 'described' / measured / reported ....? 
  • I prefer the notion of assessing learning against self, but how could this work from a teacher/trainer's point of view?
  • I would venture that all learning involves a transformation of identity. As it is mostly gradual this is not sufficiently recognised. Take a moment to think about it in your own terms.Then think about the absurdity of assessing that change against externally imposed criteria. Who are you working for when you do that? Whose agenda?
  • What happens when the "emergent learning artefact" is a behaviour, or an attitude, or something so ingrained that the artefact is the learner?
  • Does emergent learning have to produce something unique or odd?
  • The difficult question is What is New or what is Unique.
  • How do we get to the person as product of themselves over person as "product" of education. Before we claim that something we did caused learning we need evidence they were listening to us. Would this be an Artifact?
  • "just tell me what I need to do . . .".  This is not only the attitude of students but professors - rubrics so that they can quantify the learning in some way and they tell themselves they are moving from subjectivity to objectivity - and what happened to expert opinion?  How to get to the "rethinking of education?"
In reply to Jenny Mackness

Re: Assessing Emergent Learning/ Our Questions

by Scott Johnson -

Jenny,

My sense is learning defined as something you want to know is more likely to induce emergent insights. The first thing in school is to redirect the attention away from personal interest to a socially selected list of learned things with their own peculiar methods of accumulation. We unplug the person's power to reason from their own mind because we consider it unsophisticated, or whatever.

Stripped of their ability to demonstrate thinking to themselves we then recreate thinking in the form of foreign procedures that leave people dependent on pre-approved methods neither of their making or genuine discovery.

That said, we aren't slaves to this system. Instead of disallowing imagination, guessing, intuition, inference and etc. we can teach from mistakes and encourage alternate explanations as rightful attempts.

To the individual all new ways of explaining the world are emergent to them. We may already know these explanations and they aren't new to us but that doesn't make them less new. Maybe because we mistake the repeating back of what we just told someone as correct "learning" we can't see any other alternative?

In reply to Jenny Mackness

Re: Assessing Emergent Learning/ Our Questions

by Phillip Rutherford -

Jenny - very interesting and challenging questions. I have inserted my thoughts below, but I must warn that I am taking a different approach from that of the teacher/trainer. I am looking at these questions from the point of view of someone assessing whether or not emergent learning has occurred whether that be the teacher, the learner or someone/something that benefits from any learning that has emerged:

  • Is it possible to assess emergent learning? How do you      'capture' learning that is not expected? How do you measure or value it?      Are these the right questions or are they flawed?

This is an interesting question because it depends on what is being measured and valued - what it is that has been 'learned', how the learning took place, or the gap between what the individual knew before and what he/she knows now. The last two are very easy to evaluate, but it would be far more difficult to evaluate what has been 'learned', mostly because of the "I knew that!" factor. How often have we realised that we already knew something but have never articulated it before? I know I quite often read in order to more fully understand what I already know, so is this emergent learning or transformative learning? And how do we know?

  • Would it be possible to ask for emergent learning as a result of a course? Would EL be expected when a student is doing      synchronously two courses in different fields? 

Hopefully this is the main reason why we would use androgogical processes to facilitate learning rather than pedogogical. Moreover, if a student is doing two different courses one would hope that he/she learned things in one which would have application in the other - for example commerce and law, physics and medicine, business and education, and so on.

  • What is the purpose of the assessment? To provide meaningful feedback to the learner?

It depends on what the assessment is of. If we accept that we can assess emergent learning (and we don't seem to be wholly agreed on this) then, yes, assessment can be meaningful to the learner - but more particularly to those designing the processes whereby the learning is emergent.

  • Why the need to measure?

Wy indeed. Surely if we are able to measure emergent learning we are to redefine what we mean by emergent?

  • Is diagnostic reasoning the same as emergent learning? Can we afford to have our doctors' knowledge be emergent as they practice    on us? 

I would hope my doctor's knowledge is enhanced while he is practicing (is that the right word??) on me. After all, if we accept that as biological beings our health is emergent then hopefully his knowledge is capable of keeping up. I certainly don't want stock standards answers to all my ailments :-)

  • One thought I have is that ‘learning’ overall is      about some kind of change .... So maybe somehow the question is      ...  Can the Change be 'described' / measured / reported ....? 

Does learning have to be about change? I mean, can't it be about learning that, for example, something does not need to change? Or is that change in itself - that is, changing from a state of unknown to a state of known?

  • I prefer the notion of assessing learning against self, but how could this work from a teacher/trainer's point of view?

Measure self at start, measure progress, measure end result at a particular point in time. Professional sports people do it all the time.

  • I would venture that all learning involves a transformation of identity. As it is mostly gradual this is not sufficiently recognised. Take a moment to think about it in your own terms.Then think about the absurdity of assessing that change against externally imposed criteria. Who are you working for when you do that? Whose agenda?

You have just described one of the most significant failures of nearly every VET system in the world. They do not measure and value transformation but adherance to an externally imposed set of standards. The VET systems in doing so are working to a government agenda (ie, the Minister being able to stand up and tell the country how well he/she is spending our money).

  • What happens when the "emergent learning artefact" is a behaviour, or an attitude, or something so ingrained that the artefact is the learner?

Should anything happen, or do we just accept?

  • Does emergent learning have to produce something unique or odd?

I don't know about odd, but isn't all emergent learning unique? It is like the old saying that you can never step into the same river twice - ever step is unique because the river is constantly moving. Besides, if it isn't unique can it be called learning?

  • The difficult question is What is New or what is Unique.

What is the difference?

  • How do we get to the person as product of themselves over person as "product" of education. Before we claim that something we did caused learning we need evidence they were listening to us. Would this be an Artifact?

I honestly don't believe there is such a thing as a 'product of education'. I believe (as do others) that we have learned all we need to know by the age of 4-5. After that we spend the rest of our lives using this knowledge to manipulate the environment in which we exist in order to achieve the future that we desire. Your next question is an example of this. Emergent learning, therefore, is centred on becoming who you want to be - or are prepared to be given the circumstances in which you exist.

  • "just tell me what I need to do . . .". This is not only the attitude of students but professors - rubrics so that they can quantify the learning in some way and they tell themselves they are moving from subjectivity to objectivity - and what happened to expert opinion?  How to get to the "rethinking of education?"

By making sure that the professors, as well as the learners, are open to their own emergent learning. Too many are stuck to certain paradigms or knowledge and won't budge despite evidence which fails to support their contentions. If you have a teacher who is also an emergent learner you have one very excited, happy and self-actualising person.

 

In reply to Phillip Rutherford

Re: Assessing Emergent Learning/ Our Questions

by Jenny Mackness -

Phillip - thanks so much for taking the time to provide such detailed response to these questions. They are all fascinating and worthy of attention (the questions and your responses), but I'm going to select two of your responses that have jumped out at me.

1. I would venture that all learning involves a transformation of identity. I completely agree. Learning is about learning who we are and as Etienne Wenger has so eloquently written - learning, meaning and identity are all intertwined. I would be very interested to hear more about what you mean by transformation. How would we recognise this transformation? You may have noticed that one of the factors we use in drawing footprints of emergence is Identity. When I am reflecting on learning, I think I can recognise when my identity has been 'changed' in some way by the learning event, but it's difficult to pinpoint what caused the change or exactly what the nature of that change has been.

2. I believe (as do others) that we have learned all we need to know by the age of 4-5. I would love to know who the others are, because this feels so counter-intuitive to me. How does it fit with the fact that our brain cells are growing and developing until we are into our 20s? And this thought has made me wonder whether there is a difference in the way little children experience emergent learning - to the way in which adults experience it. My experience tells me that little children, in their play, experience emergent learning all the time - and I think this also relates to embodied learning.

Thanks for your thought-provoking post Phillip. Plenty for me to chew on here :-)

In reply to Jenny Mackness

Re: Assessing Emergent Learning/ Our Questions

by Phillip Rutherford -

Hi Jenny,

From my research into the impact that the complexity sciences have on training and learning I found that using knowledge to create knowledge could be termed transformative - in other words, transforming what has been learned into new learning. This may not be everybody's definition but it works for me.

Transformative learning, from my research, is not about learning more about what one already knows some or a lot about, but continuously learning what one doesn't know - including learning how the context and environment changes as new knowledge is applied. Learning at the edge of chaos if you like.

Re your second question, I would recommend reading any studies into neuroplasticity and the research conducted into how brains change from zero to 3 years of age. Some scientists believe that the greatest growth in character and brain capacity occur up until the age of 3, and from then on learning concerns how to adapt one's capacity within the environment in which one lives. As someone far more famous than I once said: "Give me the child until 7 years of age and I will give you the man".  Of course this only refers to male children :-)

Phil

 

In reply to Phillip Rutherford

Re: Assessing Emergent Learning/ Our Questions

by Jenny Mackness -

Thanks Phillip - I can see that my Christmas reading will be studies in neuroplasticity. Thanks for the book references.

Jenny