Monday, 20 February 2012

The Prediction Situation

Every so often you have the good fortune to encounter a piece of information that, although not originally written to endorse a firmly held bias, does so anyway.  While becoming completely immersed in Daniel Kahneman’s Thinking Fast and Slow, I couldn’t help feeling the urge to climb a mountain and yell: “Would everyone involved in education please listen up.  Dr. Kahneman is telling us something we REALLY need to think about!”   

Now I am the first to realize that when you yell, people begin to wonder why you feel the compulsion to be so loud.  Educators, especially those who’ve been yelled at by an irate parent, will go so far as to avoid feeding into the yeller’s need.  Regardless, what the man who enlightened us with heuristics has to say about validity, intuition and expertise should hold our attention.   If nothing else, his insights should become part of the professional conversation (even with my volume emphasis) if we are to evolve our practices in any sort of enlightened and meaningful way.

Open Thinking Fast and Slow to the middle pages and you find Dr. Kahneman appearing to repeat a statement about expertise that we can no longer afford to ignore or undervalue.  Here is what he has to say:


         The main point of this chapter is not that people who attempt to predict the future make many errors; that goes without saying.  The first lesson is that errors of prediction are inevitable because the world is unpredictable.  The second is that high subjective confidence is not to be trusted as an indicator of accuracy (low confidence could be more informative).
         Short-term trends can be forecast, and behavior and achievements can be predicted with fair accuracy from previous behaviors and achievements.  But we should not expect performance in officer training and in combat to be predictable from behavior on an obstacle field – behavior both on the test and in the real world is determined by many factors that are specific to the particular situation (emphasis added). (Kahneman, 2011)

And then again 22 pages later:

         An experienced psychotherapist knows that she is skilled in working out what is going on in her patient’s mind and that she has good intuitions about what the patient will say next.  It is tempting for her to conclude that she can also anticipate how well the patient will do next year, but this conclusion is not equally justified.  Short-term anticipation and long-term forecasting are different tasks, and the therapist has had adequate opportunity to learn one but not the other.  [T]he clinical psychologist, the stock picker, and the pundit do have intuitive skills in some of their tasks, but they have not learned to identify the situations and the tasks in which intuition will betray them.  The unrecognized limits of professional skill help explain why experts are over confident. (Kahneman, 2011)

If you have been following along as I try to unravel how school currently works, you won’t be surprised when my immediate thoughts added educators to Dr. Kahneman’s ‘experts’ list.  You would also know that I like that the “particular situation” matters when generating behavior.  Knowing that I have spent the 10,000 hours required to gain insight into anticipatory thinking and situated learning; the statements above were the greatest validation yet on why we might want to start re-thinking how today’s learners prepare for an unpredictable future.

Both of Kahneman’s statements leave me asking a lot of questions starting with: Is there any need to further interpret a statement that says that long-term prediction failure rates by experts is inevitable and that behavior and outcomes that result from testing that were limited to representations, cannot be compared to any behavioral patterns that might be manifested later in the real world?  It would appear by how this is spelled out that these are givens.  Assuming for a moment that both statements are true, then does annotating a poem or applying exponent laws in a classroom have any connection whatsoever with the learner’s real world 10 years hence?  The learner’s argument that they don’t understand why they have to perform some “random” learning task certainly has some merit if ‘experts’ have been shown to be less than accurate when it comes to long-term prediction.  Short of the appreciation argument (you won’t know the value of poetry until you’ve studied it) and familiarity argument (decimals and fractions are now realized to be different representations of the same thing), just about everything done in school from grade eight on risks being categorized as collateral as opposed to a targeted effort. (note: Yes, I prefer 'targeted' liberal arts)

The ‘other shoe’ implication of Kahneman’s statement is that as an educational ‘expert’ I may be recognized as having exceptional abilities as a developer, organizer, and communicator of knowledge.  But should I assume from this that I have managed to evade the limitations of expertise?  Can I predict that ANY assignment that I distribute today to a grade nine class will shape the behaviors of those who took up that assignment, 10 years down the road?  The simple answer would appear to be no.  At least it would be “no” if, in the mind of the learner, the randomness question had not be resolved on the day the assignment was given.

So it would appear that even though we believe learners are expanding their appreciation and familiarity margins, the predictability piece of success, the “poetry annotation moment was when I discovered how experts interpret and translate” has no hope of materializing unless there is a bright line connection made with some known outcome being currently achieved in the real world.  (note: See Space-time learning for an explanation of how matching a learner up with a practicing professional builds a young learner’s anticipatory and adaptational skills)

Once again I am struck by how we have likely fooled ourselves.   What we consider to be mastery of concept, which is really mastery of the representations funneled into the curriculum, can somehow move beyond foundational learning and start to predict the anticipatory and adaptability skills that we hope are present when the learner graduates from the formal education process.  I say this knowing full well that many educators have gone to great lengths to examine what they are teaching and in turn have actively distilled the course content to get at ‘what is important’ prior to engaging their students.   At least in these thoughtful educator’s cases, those in their charge have been spared the ‘education through mental attrition exercise.”   This exercise can be summed up as: moving through the system and seeing no other choice than to reduce concentrations to one specialized area of interest or ability.

This doesn’t happen because what is being studied is progressively more complex.  Of course things get more complex, but the answer to this doesn’t always mean reductionism.  That being said, these educators manage the complexity AND try to cultivate a set of transferable skills (they aren’t just academia’s “trainers”).   As grateful as I am to the thoughtful among us in not contributing to the content pumping exercise, there is still little or no emphasis on the anticipatory and the adaptable.  Based on Dr. Kahneman’s assertions, we can’t predict too far into the future.  But can we do a better job of determining if we are giving learners an opportunity to exercise their anticipation and adaptation abilities?

I’ve been thinking about this for a while, even though reading this book has me writing about this topic…again.  I’m not going to pretend that I am a great prognosticator.  In fact it is my weak predictor gene that got me so interested in figuring out why schooling hadn’t improved my powers to divine the future.   Not that I’m looking to moonlight as a fortune-teller, it is just that I’ve never been able to conclude with confidence that what we’re told should be in the curriculum, is the thing the learner actually needs.

So maybe it IS time for us to take a look at the things we teach and ask ourselves if some of our students are ready to enter the world of anticipation and adaptation and not be limited to just appreciation and familiarity.  No predictions here, but if programming is not in place to develop and evolve thinking skills that involve recursion and iteration, then are we leaving something important out?  Who might we consider when trying to determine if the time has come to expand the recursive and iterative learning dimensions?

What I’m about to share should not be seen as a barrier to learning.  Dr. Kahneman has made a strong argument that when we allow our intuitions to fool us into thinking we can become prognosticators, things will inevitably go wrong.  This leaves educators with few options when it comes to determining who might be ready to go beyond the learning challenges that are presented in school every day.  My gut instinct is to let every learner situate their learning, as I too, often argue the appreciation and familiarity pillars of learning.  Still, I know that some learners need to show more evidence that they are actually ready to adopt a ‘field study’ component to their learning repertoire. 

 The term ‘barrier’ is mentioned because I know many an educator who has been accused of ‘denying access to learning’ on the grounds that the learner isn’t ready, at least in the opinion of the educator.  It is this murky area between teacher intuition and not knowing until success or failure has been achieved, that can generate all kinds of problems and conflicts.  To try and mitigate this to a degree, we might want to turn to data we can gather as evidence that a person might be ready to see school extending into the larger community.  Any teacher who has a gut feeling that their student can benefit from a learning enhancement opportunity has every right to exercise the option to build programming that creates this reality.  For those of us who are comfortable with data, the following Indicators can be used to see if someone is building an argument for being a situated learner.  Just remember to consider the circumstances as well as the facts you can gather below.

The table lists the indicators and accompanying arguments for why I think you can make a case for inclusion.

Number of subjects where result is in the top quintile or bottom two quintiles
At first blush one might conclude that this is only an indicator of specific aptitude(s) and concentration of effort, but this is more than an ability measure.  Top 20% achievement demonstrates learning beyond the statistical norm generated by the class.  Grade and subject specific mastery relative to peers is not only demonstrated on a relative basis.  The more times this outcome is achieved the more likely specific mastery becomes shared mastery, which implies a reduction in transferability barriers.
Bottom two quintile scores may require remediation. This then turns into adaptability within subject domain rather than emphasizing adaptability across subjects.   The emphasis will be on consolidation and refinement protocols, which are a type of rehearsal strategy more than adaptability strategy (at least in the short term).
Degree of commitment to organized activity outside of school
Fitting into learning structures that are different from school means adapting to the different kinds of expectations that are generated beyond the classroom.  The more variety in activity choices, the more exposure to adaptation necessary situations.
Parental or guardian monitoring of learning accommodative behavior
A significant adult who models and expects an accommodative attitude reinforces the obligation to remain open to different possibilities.  
Commitment to homework
Study extension beyond the direct supervision setting of the classroom reinforces self-control.  Self-controlling the learning effort demonstrates the ability to connect concepts without external assistance.  This builds both subject specific competence and the sense that one can adjust to novel circumstances with less outside support.
Adherence to punctuality
The ability to address time expectations of others is one of the easiest ways to measure the ability and willingness to adapt.  It should be noted that this should be measured when the person is in control of being on time.
Initiative generation and/or identified responsibility assumption
A demonstrated willingness to propose new ideas and take ownership of tasks, whether simple or complicated, influences empathy development.  Putting into practice one’s actions so as to improve the circumstances for many, means transfer of effort is in effect.

I don’t believe these indicators need to be given a weighting or that one indicator is more important than the others.  We need to be extremely careful that this is not an averaging exercise.  What I do know from using this indicator formula is that taken together (summed…I repeat, it is critical to understand you DO NOT get a score out of six), you can start to determine if an adaptability profile is developing.  As you collect the data in each category, you begin to see why some students seem to have an easier time ‘fitting in’ while others struggle with even minor deviations from their routines. Your choice of what to include as an indication and what score to provide becomes important if more than one teacher is using the same model.

Here is what your indications table might look like once the data is gathered:

Number of subjects where result is in the top quintile or bottom two quintiles
Student has 83% in math and 91% in science last term (score = +2)
Student has 45% in Spanish (score = -1)
Degree of commitment to organized activity outside of school
Student piano lessons year round, 2 hours per week (score = +8)
Student plays community outdoor soccer (score = +4)
Parental or guardian monitoring of learning accommodative behavior
Parents self score 4/5 on accommodation scale (score = +4)   
Commitment to homework
Student commits to 4 hours of homework per week (score = +2)
Adherence to punctuality
Student has three late slips last term (score = -3)
Initiative generation and/or identified responsibility assumption
Student started shoe rack club (score = +5)
Student babysits younger siblings 4 days per week after school       (score = +4)
= +25

The sum should be seen as the degree of development achieved thus far and should not be confused as a predictor of future performance.  As scores increase however, you might want to consider when a calculated risk can be taken to allow the learner an opportunity to introduce some autonomy into their learning routine.

None of this ‘teaches’ adaptability or anticipatory thinking skills.  I assume that anyone reading this knows how to infuse the learning with adaptation and anticipation requirements.  What Kahneman’s work has me contemplating (and now reinforcing) is the need to identify indicators of thinking transfer that are present, are consistent and are durable.  From a diagnostic perspective this won’t fly in the face of Dr. Kahneman’s claims, as we still won’t be able to predict who graduates from medical school and becomes a gastroenterologist.  What we might discover though is that on an individual level if we want to assist more learners to have the potential to become medical specialists 10 years from now we should foster a learning environment that develops learning experts who recognize both their current strengths and the limitations which only time will overcome.

No comments:

Post a Comment