Assessing Emergent Learning

Re: Assessing Emergent Learning

by Deirdre Bonnycastle -
Number of replies: 15

As long as we think of assessment as standardized testing on factual knowledge, emergent learning will fail the test. I see this issue regularly in medicine where we need a fine balance between anatomical knowledge and diagnostic reasoning yet continue to use MCQ's as our testing tool.

I agree with Peter, narrow objectives can only be assessed using highly specific tests and leave no room for moments of illumination. So was the objective for my granddaughter "to cast on 50 stitches" or was it "when faced with a new situation to create a way to participate in the process."

I've been looking at portfolios and the rubrics used to assess them lately. I always ask for a bonus area for students who demonstrate learning something new or unexpected.

In reply to Deirdre Bonnycastle

Re: Assessing Emergent Learning

by Jenny Mackness -

Hi Deidre - it's interesting that you bring up medicine as a subject. We have often discussed whether medicine, nursing, teaching (i.e. those subjects in which what people learn can have such a profound effect on other people's lives) are subjects which need to be prescriptive. They certainly seem to be subjects in which there is a lot of standardised testing.

Is diagnostic reasoning the same as emergent learning? Can we afford to have our doctors' knowledge be emergent as they practice on us? I'd be interested to hear what you think.

I like the idea of adding a bonus area to a rubric to allow for emergent learning.  

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Barbara Berry -

Hi Jenny and All,

Jumping in again (juggling meetings etc.) but really want to engage in the conversation. Forgive me if I repeat (hard to read all of the posts in a timely fashion. I clearly need more practice at managing my "emergent" learning : )

I love your questions on "assessing emergent learning" as they are provocative and thus are assisting me to consider the possibilities, my own bias and assumptions and experiences (in prescribed and emergent scenarios). I am leaning towards "no", it is not really possible to assess emergent learning but what might be possible is to assist individuals to develop their skills of self-awareness, reflection and paying attention to whatever is "emerging" on the periphery or perhaps centrally (even in the context of highly prescribed instruction). 

In the context of health sciences and medical/nursing education (my initial background), there are legal requirements for licensing and thus prescribed content-heavy curricula but to practice "safely" one has to be learning in situ (with real patients - in my day and now it's often simulations). We also were taught to "pay attention", be alert for stuff that might not be obvious but that would influence practice. This was practiced in scaffolded situations until such time as one could operate "safely" without the clinical expert being there. As I recall (this is historic for me now after all of these years) is that I learned about myself and my own capabilities (strengths and weaknesses) to think, practice safely, be empathic etc. I also learned what I liked and didn't like about this work. These two are examples of what I might consider "emergent" learnings and while there was no way of assessing these I was assessed on my ability to be "reflective" and in practicing reflection, I became aware of what I was really learning about myself in this context.

it's only one example and I might be off base.

all for now, 

Barb

In reply to Barbara Berry

Re: Assessing Emergent Learning

by Scott Johnson -

Hi Everyone,

Interesting that medical education comes up. My sense is the environment that medicine is in is essentially emergent. Constantly readjusting by responding to surprise, rotating teams with different strengths, learning through a kind of open strategy of referencing, learning, trying, failing, re-referencing, learning, trying....

Fine art training is like that and the assessing is no less genuine than a deliberate and thoughtful attempt by someone with high standards for themself. Of course we want things to work without fault but our best chance of that happening is to put someone with pride in their practice and then back them up. Too easy an answer though.

Have a quote to insert here but don't want to plug up the discussion area. There's a wiki?

Loved the medical education website and will pass it on.

In reply to Barbara Berry

Re: Assessing Emergent Learning

by Jenny Mackness -

No- not off base Barb. That's a great description of how emergent learning and prescriptive learning rub along beside each other, even in situations where you would expect prescriptive learning to dominate.  A really helpful example. Thanks

In reply to Barbara Berry

Re: Assessing Emergent Learning

by Joyce McKnight -

Barbara:  I very much agree with you.   I think that most professions (and most disciplines for that matter) begin with a common core of knowledge, skills, values, techniques, and often vocabulary that are crucial in enabling practitioners to work together and helps increase the sum of human knowledge.  Plain, old-fashioned rote learning is often the best way to begin, then the scaffolding experience that you so eloquently described takes shape as one listens, learns, and practices under experts' careful attention...then you become a practitioner yourself, a level where many people remain for their entire lives, doing useful but not especially creative work, but the person who becomes a master healer keeps reflecting, learning, trying new things, observing, bringing in ideas that may seem to come from "left field" ...but is always aware of the ongoing dictum..."first of all do no harm"...and occasionally circles back to basic principles to make sure s/he is on track.

I also liked your thoughts on the ways emerging learning is really about learning who you are (and who you are not) as a practitioner and how structured reflection as is often required in medicine and other helping professions can be useful in this process.

By the way, we seem to have shared a somewhat similar journey...although I started out as a mental health professional not as a medical person.

 

In reply to Joyce McKnight

Re: Assessing Emergent Learning

by Phillip Rutherford -

Joyce,

I think you make some very good points and with your indulgence would like to address two of them.

Regarding rote learning, neuroscience is now discovering that using the brain in a single, consistent manner (such as memorising and repeating facts in a repetitive way - eg, 'times table', movement of a limb) can increase one's neural flexibility. I know this is contrary to the position many teachers take on rote learning, but studies into neuroplasticity are revealing the positive aspects of this.

Regarding your comments about increasing the sum of human knowledge, I think that when we discuss emergent learning we cannot avoid discussing also the purpose of such learning - and that is to turn information into knowledge in order to better understand the world or various elements of it. My research and continuous observation suggests that there is three broad domains which span the continuum between the Kantian view of knowledge and that of Hegel.

At the Kantian end there are those who view knowledge as static and 'truth'. They adopt a long learning/feedback loop in order to bring together all of the information in a structured and stable way. They don't see knowledge as context-specific, more truth as its own argument, and give greater concentration to getting the information right. Their aim is for accuracy of knowledge.

At the other end are those who look at information in context in order to build a platform of knowledge from which to launch further discovery - often through trial and error. They apply a short or single learning/feedback loop and base decisions around limited input - sometimes no input whatsoever. They apply creative chaos to knowledge in order to keep it alive and dynamic, and by doing so reveal information which might not have otherwise been revealed. The ideal at this end of the continuum is to minimise risk and avoid errors.

In between the two are those who use information to make sense of either 'truth in itself' or 'truth in context', and apply medium learning/feedback loops in order to position their knowledge and understanding between the two poles. They are happiest questionning wisdom, not for its own sake but in order to better place it in either the Kantian or Hegelian camp. And having done so they apply knowledge gained at either end of the continuum to gain a better understanding (eg, applying creative chaos to 'single source of truth' in order to test this truth, or adopting a more corporate view of dynamic knowledge in order to slow down the rush of knowledge in order to determine usability.

The first group 'accepts truth', the second 'creates truth', while the third 'interprets truth'.  

This continuum could also be seen as that which stretches between stability at one end and chaos at the other. In between is complexity, and learners could be anywhere along this continuum depending on whether or not they are clear on the information and its context. Moreover, any one of these could become a master in their chosen field but, as you state, this can only occur when the learner understand who he/she is and accepts their inherent and preferred way of dealing with information as it becomes knowledge and understanding.

 

In reply to Joyce McKnight

Re: Assessing Emergent Learning

by Barbara Berry -

Hi Joyce, 

Yes, you are correct and we do share some common insights about emergence as a part of the development of both practical knowledge while undertaking work and that it must be continuous and intentional if it will amount to anything. For me there is a creative dimension in the sense that to interrogate practice one must be willing to delve for truth and this often takes another to listen, to ask questions and to work through the process of reflecting in and on action. Sometimes it happens when you least expect it, at other times, I have found this practice only when push comes to shove and one has to reflect for one reason or another. 

My undergraduate degree is in Nursing and I worked in clinical practice (intenstive care, surgery and pediatric surgery) for a total of 6 years then in public health for another 5 followed by the emergence of my consulting practice after my graduate degree (still in the health sector). 

Depending on the practice context, clinical reasoning in my experience takes place in association with the practice of dealing with the patient or client or family or community. It is a complex, reasoning process that entails inductive and deductive reasoning happening at the same time and sometimes very quickly in actue settings. What is interesting is that good clinical reasoning includes "critical thinking" and this includes rigorous habits such as analysis, inferential, evaluative, predictive, and explanatory thinking all in order to make sound clinical decisions in context.  The intent is to resolve the particular clinical problem and treat/care for the patient/client as said with their safety uppermost in mind. Clinical reasoning is highly intentional processing and done by an individual and in certain situations by a team who "reason together". There is an "emergent" dimension to some situations in clinical practice where the patient's condition is in constant flux and responds based on judgements that are being made almost simultaneously. It can be a full-body experience in the sense that after all is said and done, a person can feel emotion, physical, cognitive and social exhastion. (no wonder I used to go home and collapse!). I am sure you have seen and or experienced this in your own professional work. I think that emergent learning (as I am learning about it in this seminar) can happen at the same time as clinical reasoning but how this all "happens" together and in fact how it might be similar to but different from clinical reasoning as I experienced it, I need to give more thought to! It has been years since I have "recalled" this kind of "work" and so, now I am once again curious : ) 

cheers, 

Barb

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Deirdre Bonnycastle -

Occupational training programs are their own kettle of fish because there is a large amount of absolutely essential memorized content and skills that must be learned to a point of automaticity.

There is also an apprenticeship component where you work in real or simulated situations with an expert in order to learn to think like an expert. This is where emergent learning makes an appearance.

Diagnostic Reasoning requires taking the patient symptoms, matching this to what you know about body systems and coming up with a couple of predictions about what is wrong. Clinical Reasoning takes the predictions, and refines the diagnosis through patient history, physical examination and tests, then determines treatment. Compare, contrast, research, analyse, identify systemic problems, juggle multiple factors are important strategies that must be developed in medicine.

You learn to do this by seeing a wide range of patients over time but you also need a constant feedback system from patients, nurses, preceptors and self reflection. This is where training programs often fail their students. Poor feedback loops demoralize on one hand because of their severity and allow negative behaviour to continue on the other extreme. So formative assessment is a critical element. Summative assessment in occupational training is usually done externally and is a client safety step that confirms this person is qualified to practice.

So you have the prescriptive classrooms, emergent clinical experiences and the chaos of the unsupervised experience, all existing  and clamoring for more time.

In reply to Deirdre Bonnycastle

Re: Assessing Emergent Learning

by Scott Johnson -

Hi Deirdre,

Having been diagnosed by 5 different doctors and a specialist over a period of 5 weeks based on the simple fact that no one bothered to read further back on my record than my last hospital test, I think we can add the power of proper listening or observation to the things medical practitioners should know. Humans make mistakes and when they build simulated environments like schools they rob themselves of the juicy details of reality in trade for the convenience of rightness, prediction and further simulated performance. On the belief there are people out there so well trained they can imagine reality into existence we follow their proofs and not the reality we are presented.

To me, “qualified to practice” is a comfort (or maybe an approximation of a comfort) but as an assurance that the system behind it functions properly is not good evidence. By saying that I know there are many things to learn and many people more qualified than myself to learn them from. Yet this doesn’t diminish my ability process the world as I see it. Though of course we don't ask the receiver--we test them.  

In reply to Scott Johnson

Re: Assessing Emergent Learning

by Scott Johnson -

Correction: should had said "mis-diagnosed" though I'm sure everyone got it.

Diagnostic thinking can go beyond projecting specific chains of cause and effect onto to something to explain it. In the sense that dissonance illustrates something being out of place, we could measure novelty by noticing one part fails to be explained by those around it. That part should show as a mistake or a misinterpretation and could as well indicate emergence? Do emergent thoughts need to be novel and out on their own? If they slipped in as connections between things we knew but previously couldn't connect they would not necessarily be noticible.

In reply to Scott Johnson

Re: Assessing Emergent Learning

by Joyce McKnight -

Hi Scott:  I am sorry for your unhappy experience with the medical system.  I had a similar one in 2011 that nearly killed me and from which I am still recovering...mine had to do with the unfortunate tendency that modern medicine (at least modern Western medicine) seems to have developed to "play the numbers"...one "can't" have something because it is rare, therefore one can't be tested for it because it is "rare", and, of course, it is "rare" because it never shows up in tests!!!    My particular example is hereditary hemochromatosis...the tendency of my body to collect too much iron over the decades and the most common life threatening hereditary disorder found in people of nothern European decent.  I nearly died before a wise physician's assistant thought to check my iron levels which were 20+ times the normal level.  I tell this story to emphasize that all medical practicitioners need to observe and think, not just play the numbers...and because I feel folks need to know about hemochromatosis which manifests itself with arthritis of the hands and feet, feels a lot like fibromyalgia, kills vital organs like your pancreas, liver, and eventually your heart and is kept in check by old fashioned phlebotomies...other than the arthritis it shows up most commonly as middle age onset diabetes which by the numbers (again) is most usually called Type II diabetes and blamed on lifestyle especially being overweight.  In fact, in the US endocrinologists aren't even allowed to  order the relatively inexpensive blood test to rule out hemochromatosis because the numbers "show" it is so rare.   At any rate, Scott you are so right!  And folks with northern European ancestry...if you have some or all of the symptoms please don't be afraid to bring the possibility up to your health care providers..

In reply to Joyce McKnight

Re: Assessing Emergent Learning

by Scott Johnson -

Joyce, your situation sounds similiar to mine in it being a balance of a medical problem that can kill you combined with sloppy diagnostics. Long story short, the take away for me is that mastry of anything is useless without the willingness to listen beyond the voice of your well-trained brain.

The doctors and the specialist I saw are not unskilled but they made a decision on what to treat me for based on the first condition which satisfied their incomplete investigation. To say we train people isn't enough. We need to impress on them that neither the jolly endorphan rush of being right nor your degree makes your decision correct.

Feeling like you know is a very attractive sensation. It seems the more training a person gets the more suseptable to this distraction a person becomes. This undoes the whole concept of "mastery" for me and suggests we should look for characteristics of the search for "knowing" in the uncertain habits of the beginner. Habits that my be emergent.

In reply to Deirdre Bonnycastle

Re: Assessing Emergent Learning

by Jenny Mackness -

Thanks Deidre for explaining where you think emergent learning fits into relation to medical situations. I like the idea of emergent experiences and I'm now wondering how they differ from emergent learning. Stil thinking..... :-)

In reply to Jenny Mackness

Re: Assessing Emergent Learning

by Jaap Bosman -

Hi Jenny, Deirdre, 

In my view the Emergent Learning Artefact  (I did mention this Artefact somewhere else in this forum) is a proof or evidence of this emergent learning of this student. You (Jenny) are right I think, the Emergent Learning Artefact does fit into the bonus area of the rubric of Deirdre Bonneycastle.
Great idea of Deirdre to make a special room for the unexpected. If we do not expect that some unexpected thing could be happening, we will not notice it.

(this is acomment on my blog http://connectiv.wordpress.com/2013/11/21/emergent-learning/ 

In reply to Jaap Bosman

Re: Assessing Emergent Learning

by Nick Kearney -

what happens when the "emergent learning artefact" is a behaviour, or an attitude, or something so ingrained that the artefact is the learner?

to me the idea of evidence in relation to emergent learning is problematic

it feels like a parent saying to an adolescent "so you fell in love, prove it"