Book: “Make It Stick: The Science of Successful Learning“; Peter C. Brown, Henry L. Roediger, Mark A. McDaniel

Why this book?

This is the 3rd – March – book from My Year of Reading 2019 list.

In that post I was specific on the reasons for reading this book, stating the following:

1. Specifically looked for a book for my One Word 2019 of “apply” (my learning in all contexts) and this one looked ideal to help me this year.
2. High hopes that this book will significantly help me apply my learning activities into daily practice.

and included this intro vid
https://youtu.be/tBjmBn5vyPc

My reading of the book

I take notes chapter by chapter for each non-fiction I read.

Where the books are part of a book club read that I facilitate, I endeavour to use supplementary materials as the basis for discussion and often use Q&As via questions asked in book club-type resources.

For this book I have found some resources including Sketchnote chapter summaries - https://www.retrievalpractice.org/make-it-stick - and questions - https://drive.google.com/file/d/1qBfk6oIX7sYipEq_dU67SJzexFghHlO9/view?usp=sharing.

For each section of the book, I include my research and application notes, my answers to the Book Club questions and my notes of the book content.

My overall assessment and response to the book

It has been amazing to discover the learning techniques in this book. As far as I can recall, I have never had any detailed input from educators on the learning process throughout my formal education. It has made me wonder if I would have done better at school in O and A levels (1973 to 1980) if I had have done.

I thoroughly recommend this book to all of any age 11+.

It made me think throughout about my learning in various ways over the years and encouraged me to think how to apply any of the material in my learning life.

The really helpful book club resources that I used during my reading of the book included some overall questions that I will answer as my summary response to the book.

Concluding Questions

Q: What, if anything, was surprising to you about how people learn  and remember?

A: It was eye-opening reading about the various techniques to learn more effectively and disconcerting to read how the techniques I have used for all my 57 years to date are sub-optimal despite them seeing to be effective!

Q: What evidence do the author’s give to support the book’s ideas? Is the evidence convincing?

A: Again the research to justify the techniques recommended in the book was eye-opening and often inspirational. I would say that the research that I read about in the book.

Q: Are you likely to change your learning strategies as a result of reading Make It Stick? How so?

A: I should really make the effort to do this both as a learner and as a facilitator of learners. This will be a challenge without more guidance e.g. most effective ways of creating flashcards, how to implement retrieval practice. I will explore the “how-to”s as I try to do this. I need to pick some learning that I am doing that I know I need or want to retrieve and become second nature in the future. I see there are additional resources in the Retrieval Practice web site and will explore these.

Preface

My research and application notes from the book

Right away, my first thoughts after reading the preface are that I done minimal study of how to learn. I have vague memories at school and university of some “how to study” materials but precious little of that I read in detail and applied to my studies. I am increasingly aware that there must be better ways for me to learn and retain and apply knowledge.

To reiterate, I have high hopes that this book will be practically helpful to me and the Preface reinforces that view.

My notes from the book

“Memory is the mother of all wisdom.”
(Aeschylus; Prometheus Bound)

people generally going the wrong way about learning

much of what we believe about learning turns out to be wasted effort

research over 125 years yielded growing science of learning: highly effective, evidence-based strategies to replace less effective but widely accepted practices rooted in theory, lore, intuition

BUT most effective learning strategies are not intuitive

book applies 2 of primary learning principles in the book: spaced repetition of key ideas & interleaving of different but related topics

means that reader will remember them better & use them more effectively as a result

book about what people can do for themselves right now to learn better & remember longer

responsibility for learning rests with each individual

teachers / coaches likewise can be more effective right now by helping students understand these principles & designing them into the learning experience

audience then is students and teachers but also anyone for whom effective learning is a high priority

including lifelong learners nearing middle age or older who want to hone their skills so as to stay in the game

lots more to learn about learning BUT there are principles & practical strategies from research can be put to work immediately at no cost and to great effect

Chapter 1: Learning is misunderstood

My research and application notes from the book

More light being put on how I have studied all my life and how that might have been done considerably better!

Reminded of a UT concert show image “Everything You Know Is Wrong”.

Also making me think that some of my critical thoughts about a specific person is actually deep-rooted in how both of us learn and how that has been explained. I am now realising that the other person sounds like they learn more like the content of this book so far than I do. This is a revelation .. and a challenge.]

Starting to feel a bit apprehensive about how I learn this new way of learning as I suspect I have tried minimal similar things in the past but have given up way too easily. This could be quite a journey and sounds like it may be quite painful!!

Also wondering now what the best practice is for note taking from books. You can see my current practice in this post. I assume I could be more effective so will see what guidance is offered throughout the book. I am aware of some of the research around notes by hand versus digital notes etc. My current practice is to take notes digitally direct into a blog post editor for publishing on a couple of platforms.

Book Club Questions

Have you had a specific experience with a student or educator who seemed to misunderstand how learning works?

I can recall no conversation about how to study or learn with anyone over the years. This is only coming to the fore these days as I mix in L&D circles online and as we discuss learning in the #PKMchat on Twitter. I remember having a slight interest in this when at school (1970s). I remember reading some of Maddox’s “How To Study” but not applying it to any great extent. Then at university (1980-1984) I remember getting a “how to study” guide booklet when we started the course. Then I did an MBA by open learning in the mid-1990s and I recall getting nothing on how to study. Usually, learning info was basically the dates and times of lessons, exams etc. I suspect I may have done better at school with more info on how best to learn. At university, I got a First so no room for improvement there BUT it would have been good to be able to recall more of that learning all these years since. I have never been that good at recalling facts such as definitions, dates, law cases with years and so on. I suspect even this week (!!), I may have been the person not understanding how learning works!

What did you share with this student or educator? What was challenging or successful about this interaction?

I would be able to share more now having read the chapter based on my head knowledge of the chapter (but I need to see this in practice to show I have learned this!). Making me think that I need to understand more what others think about how best to learn and start that dialogue. I have the same feeling now about learning as I did about careers when reading Liz Ryan’s Reinvention Roadmap in Q4 2018. Not many people are able to speak about or manage either in depth and with mastery.

What is the definition of learning?

Acquiring knowledge and skills and having them readily available from memory so you can make sense of future problems and opportunities.

What are the three immutable “aspects of learning”?

1. learning requires memory so what we have learned is still there later when we need it
2. we need to keep learning & remembering all our lives – if you are good at learning you have an advantage in life
3. learning is an acquired skill & most effective strategies are often counterintuitive

What claims do the authors make about learning (that will serve as chapters of the book)?

1. learning is deeper and more durable when it takes effort
2. we are poor judges of when we are learning well and when we are not
3. re-reading text & massed practice of a skill or new knowledge are by far the preferred learning strategies but also amongst the least productive cf cramming
4. retrieval practice is a more effective learning strategy than review by rereading
5. when you space out practice at a task & get rusty between sessions or interleave practice of 2/more subjects, retrieval is harder & feels less productive … but … the effort produces longer lasting learning & enables more versatile application of it in later settings
6. trying to solve a problem before being taught the solution leads to better learning, even when making errors when trying
7. learning style preferences not consistent with evidence – you learn better when you “go wide” with ways of learning
8. when adept at extracting underlying principles/rules for different types of problems, you are more successful at picking right solutions in unfamiliar situations
9. we are all susceptible to illusions that can hijack our judgement of what we know and can do – testing helps calibrate our judgements of what we have learned – in virtually all areas of learning, you build better mastery when you use testing as a toll to identify / bring up your areas of weakness
10. all new learning requires a foundation of prior knowledge
11. re brain full, true if just working on repetition .. but .. if you practice elaboration, no known limit to what you can learn
12. putting new learning into a larger context helps learning
13. many believe their intellectual ability is hard-wired from birth & that failure to meet a learning challenge is an indictment of their native ability … but every time you learn something new, you change the brain – the residue of your experiences is stored

What are common misconceptions about learning?

1. rereading something does not burn the content into our memory
2. making learning easier & faster does not make the learning better – research shows when learning is harder, it is stronger & lasts longer
3. single-minded focus for a concentrated period of time still leads to learning that is transitory & melts away quickly

Why does re-reading not work? What studies prove that re-reading does not work?

1. time consuming
2. does not result in durable memory
3. often involves a kind of unwitting self-deception – growing familiarity with content feels like subject mastery – the amount of study time is no measure of mastery

Studies:

1. 2008 article in Contemporary Educational Psychology, Washington University scientists reported on a series of studies they conducted at their own school and at the University of New Mexico to shed light on rereading as a strategy to improve understanding and memory of prose.
2. Subsequent experiments at Washington University assessed the benefits of rereading among students of differing abilities, in a learning situation paralleling that faced by students in classes.

What practices should we use when studying a text?

1. used the set of key concepts at end of each chapter to test yourself?
2. define a concept and use it in a para?
3. convert main points into a set of questions and tried to answer while studying?
4. rephrased main ideas in own words as reading?
5. relate ideas to what already knew?
6. looked for examples beyond the text?

What is the relationship between knowledge accumulation and creativity?

“one cannot apply what one knows in a practical manner if one does not know anything to apply.”

What is “mastery”?

mastery requires both possession of ready knowledge & conceptual understanding of how to use it

How can testing be used to enhance learning?

the power of active retrieval – testing – to strengthen memory & that the more effortful the retrieval, the stronger the benefit

What are the 2 “profound benefits” of learning?

1. tells you what you know & don’t know – so you know where to focus to address weak areas
2. recalling what you learned causes brain to reconsolidate the memory – strengthens its connections to what you already know & makes it easier to recall  in future

My notes from the book

when we talk about learning we mean acquiring knowledge & skills & having them readily available from memory so you can make sense of future problems & opportunities

foundational aspects of learning:-

1. learning requires memory so what we have learned is still there later when we need it
2. we need to keep learning & remembering all our lives – if you are good at learning you have an advantage in life
3. learning is an acquired skill & most effective strategies are often counterintuitive

principal claims the authors make in the book:-

1. learning is deeper and more durable when it takes effort
2. we are poor judges of when we are learning well and when we are not
3. re-reading text & massed practice of a skill or new knowledge are by far the preferred learning strategies but also amongst the least productive cf cramming
4. retrieval practice is a more effective learning strategy than review by rereading
5. cf flashcards, quizzes
6. neural pathways that make up a body of learning do get stronger when memory is retrieved & learning practiced
7. periodic practice:-
8. arrests forgetting
9. strengthens retrieval routes
10. essential for hanging onto the knowledge you want to gain
11. when you space out practice at a task & get rusty between sessions or interleave practice of 2/more subjects, retrieval is harder & feels less productive … but … the effort produces longer lasting learning & enables more versatile application of it in later settings
12. trying to solve a problem before being taught the solution leads to better learning, even when making errors when trying
13. learning style preferences not consistent with evidence – you learn better when you “go wide” with ways of learning
14. when adept at extracting underlying principles/rules for different types of problems, you are more successful at picking right solutions in unfamiliar situations
15. this skill better acquired in interleaved & varied practice
16. we are all susceptible to illusions that can hijack our judgement of what we know and can do – testing helps calibrate our judgements of what we have learned – in virtually all areas of learning, you build better mastery when you use testing as a toll to identify / bring up your areas of weakness
17. all new learning requires a foundation of prior knowledge – eg learn to fly with 2 engines before only 1
18. re brain full, true if just working on repetition .. but .. if you practice elaboration, no known limit to what you can learn
19. elaboration: process of giving new material meaning by expressing in your own words & connecting with what you already know
20. the more you do this connecting with prior knowledge, the stronger your grasp of the new learning will be & the more connections you create that will help you remember it later
21. putting new learning into a larger context helps learning
22. people who learn to extract the key ideas from new material and organise them into a mental model & connect that model to prior knowledge show an advantage in learning complex mastery
23. many believe their intellectual ability is hard-wired from birth & that failure to meet a learning challenge is an indictment of their native ability … but every time you learn something new, you change the brain – the residue of your experiences is stored
24. yes there is a start point
25. but we become capable through the learning & devt of mental models that help us to reason, solve, create
26. & so failure is a badge of effort & source of useful info – the need to dig deeper or try different strategy
27. understand that striving/setbacks are essential for surpassing current level of performance toward true expertise (cf video game scores)
28. making mistakes & correcting them builds the bridges to advanced learning

Empirical evidence vs Theory, Lore, Intuition

how we teach/study largely mix of theory, lore & intuition

now tho we have 40+ years of body of evidence to clarify what works & to discover the strategies that get results

cognitive psychology: basic science of understanding how the mind works via researching how people perceive, remember & think

developmental / educational psychologists: theories of human development & how they can be used to shape the tools of education

neuroscientists: understanding brain mechanisms that underlie learning

what advice to take for how best to go about learning?

wise to be sceptical

rest of the book runs through the research & indicates which is theory and which is theory & demonstrable practice

People misunderstand learning

much of what we have been (are) doing as teachers/students is not working well … but some small changes could make big difference

e.g. rereading something does not burn the content into our memory

e.g. making learning easier & faster does not make the learning better – research shows when learning is harder, it is stronger & lasts longer

e.g. single-minded focus for a concentrated period of time still leads to learning that is transitory & melts away quickly

eeek .. rereading text books is main study strategy of most people !!

issues with rereading:-

1. time consuming
2. does not result in durable memory
3. often involves a kind of unwitting self-deception – growing familiarity with content feels like subject mastery – the amount of study time is no measure of mastery

learning is stronger when it matters, when the abstract is made concrete and personal

cf flying simulator – provides retrieval practice, the practice is spaced interleaved, varied & involves as far as possible the same mental processes a pilot invokes in the air – also shows where practice needs to improve

simulator-type learning is rare

examples of incorrect study guidance on US college web sites

Early Evidence

fallacy of repetitive exposure builds memory well-established over years of investigations

no rereading benefits in close proximity

makes sense to re-read when gap … multiple readings in quick succession is time-consuming study strategy yielding negligible benefit at expense of much more effective strategies that take less time

Illusions of Knowing

why does re-reading happen?

bad advice to students

increasing familiarity gives illusion of mastery

need to master the ideas behind the words not the words themselves

suggestions to students with poor grades:-

1. used the set of key concepts at end of each chapter to test yourself?
2. define a concept and use it in a para?
3. convert main points into a set of questions and tried to answer while studying?
4. rephrased main ideas in own words as reading?
5. relate ideas to what already knew?
6. looked for examples beyond the text?

issue is not knowing how to study effectively

being accurate in your judgement of what you know & don’t now is critical for decision making

cf Rumsfeld quote

most diligent students can still be hobbled by 2 liabilities:-

1. failing to know the areas where their learning is weak i.e. where work is needed to improve their knowledge
2. preference for study methods that create false sense of mastery

Knowledge: Not Sufficient, But Necessary

creativity is more important than knowledge (Einstein)

but false dichotomy

what do you want from your surgeon or pilot …

need to do better at building knowledge and creativity

“one cannot apply what one knows in a practical manner if one does not know anything to apply”

mastery in any field is a gradual accretion of knowledge, conceptual understanding, judgement & skill

these are fruits of variety in practice of new skills & of striving, reflection & mental rehearsal

memorising facts is like stocking a building site with supplies to build a house

building house also requires not only knowledge of different materials but also conceptual understanding of how things work

mastery requires both possession of ready knowledge & conceptual understanding of how to use it

Testing: Dipstick Versus Learning Tool

testing is a battleground

BUT ..

if we stop thinking of testing as a dipstick to measure learning – if we think of it as practising retrieval of learning from memory rather than “testing”, we open ourselves to another possibility: the use of testing as a tool for learning

!! the power of active retrieval – testing – to strengthen memory & that the more effortful the retrieval, the stronger the benefit

cf flight simulator vs Powerpoint lecture

cf quiz vs rereading

benefits of actually retrieving learning:-

1. tells you what you know & don’t know – so you know where to focus to address weak areas
2. recalling what you learned causes brain to reconsolidate the memory – strengthens its connections to what you already know & makes it easier to recall  in future

retrieval – testing – interrupts forgetting

The Takeaway

most of us are going about learning the wrong way

we are giving poor advice to learners

lots of what we think about learning is based on faith/intuition & not research

we work with unproductive learning strategies – even true of researchers who know these do not work!

1 of best habits a learner can develop is regular self-quizzing to recalibrate their understanding of what they know and do not know

we now know simple/practical strategies that anyone, of any age, can use to learn better & remember longer:-

1. various forms of retrieval practice, e.g.
2. low-stakes quizzing & self-testing
3. spacing out practice
4. interleaving practice of different but related topics/skills
5. trying to solve a problem before being taught the solution
6. distilling underlying principles/rules that differentiate types of problems

Chapter 2: To Learn, Retrieve

My research and application notes from the book

A good reminder that I need to reflect more on my working practice. Topical as I read this chapter as there were some project planning basics that I had not done - completely missed parallel project tasks with scarce resource as I was more concerned about the individual plans.

I beat myself up at my inability to memorise things. I am in awe of my kids learning lines to act and sing in school plays/musicals. As a Christian I am encouraged to memorise Bible verses but always give up in abject failure.

Interested to see that Pooja Agarwal was one of the researchers mentioned in the book. https://www.poojaagarwal.com/ 

Pooja runs the web site that I am sourcing the Sketchnotes and questions from. https://www.retrievalpractice.org/make-it-stick.

As I read the chapter, I discovered that Pooja and Patrice Bain https://www.patricebain.com/ have co-authored a book that is due for publication – “Powerful Teaching: Unleash the Science of Learning” https://www.poojaagarwal.com/powerfulteaching

I had a flashback to open book exams which I have done a few of in my time. These were mentioned in passing but not in great detail. I would be interested in what the best practice techniques are for those sorts of tests.

I am not an instructor and it is very rare that I present things to people who will go on to be tested, However, the book is already giving me pause for thought in how i self-impose these learning strategies for my own learning.

I have tried to remember how I was tested at school and university. Past papers and mocks seemed to be the default technique. I do not recall spaced retrieval quizzes.

Again, this chapter is making me think how I would have learned differently if I knew then what I am learning in this book now.

The word pair example with missing letters helping retrieval whilst at the time seeming to be trivial reminded me of an American leadership training course I did over three years where the study books had questions/notes with missing fill-in-the-blank word sentences.

Again interested to see that we have not started using these techniques to any great extent in the book to date but we are being asked questions!

Book Club Questions

What is one specific strategy you already use for retrieval practice in your classroom?

This question does not relate to my work directly but asking questions about any material I have used for instruction at the end of a “unit” helps me clarify understanding of those I am instructing.

What are the benefits of reflection?

It leads to stronger learning:-

• retrieving knowledge & training from memory
• connecting these to new experiences
• visualising & mentally rehearsing what you might do differently next time
• can lead to new & innovative ways of doing things

What are the most important considerations when using retrieval practice? Why are they important?

Must be repeated again and again

In spaced out sessions to require cognitive effort

Ideally so responses become reflexive (or instinctive)

Explain the false dichotomy of learning basic knowledge vs creative thinking. How does knowledge enable creativity?

Both need cultivating.

The stronger one’s knowledge about the subject, the more nuanced one’s creativity can be in addressing a new problem.

Creativity without a sturdy foundation of knowledge builds a weak house.

What is the “generation effect”? Why does it improve retention? Give an example.

“where information remembered if it is generated from one's own mind rather than simply read” (Wikipedia)

e.g. word pairs; and in that case having some letters of one of the word pairs.

A professor reflected “My teaching is only so important. How I structure the class has a lot to do with it, may be more.” What does he mean? What led to that conclusion?

Seeing the results in his class of retrieval testing and spacing those tests out and being open about when the tests will take place.

How can you make feedback most effective for learning? How can feedback be harmful for learning?

Delay the feedback slightly.

Harmful when the feedback is part of the process so that the learner is dependent on the feedback as part of the process.

In what ways does retrieval help learning besides just rote memorisation?

Facilitates better transfer of knowledge to other contexts & problems .

Improves ability to retain & retrieve material that is related but not tested.

Anecdotally, under what circumstances do students most prefer retrieval quizzes? Why?

When frequent and when low stakes. Gets to the point where no cramming required.

My notes from the book

example of surgeon, planning for op via checklists of things to consider and do and then having to act on reflex as he found things during surgery – including using techniques he had invented himself, not taught anywhere

Reflection is a form of practice

what can we learn from this example?

there is an essential kind of learning that comes from reflection on personal experiences

for the medic it is about thinking how things went and how he could improve the process/care next time

reflection can involve several cognitive activities that lead to stronger learning:-

1. retrieving knowledge and earlier training from memory
2. connecting these to new experiences
3. visualising & mentally rehearsing what you might do differently next time

in the medic’s case, such reflection led him to new techniques

medic says to make sure the learning is there when you need it:-

1. memorise the list of things you need to worry about in a given situation: A, B and C
2. drill on them
3. there comes a time when no time to drill the steps, you act reflexively taking the right action – when no time to think
4. recalling it and practicing it over and over

The testing effect

retrieval ties the knot for memory

repeated retrieval makes the recall fast

forgetting curves:-

1. in short order we lose c70% of what we have just heard or read
2. forgetting then slows
3. last 30% falls away more slowly

a central challenge to improving how we learn is to find a way to interrupt the process of forgetting

testing effect: psychologists term for the power of retrieval as a learning tool

commonly, testing used to measure learning & assign grades

but the act of retrieving knowledge from memory has the effect of making that knowledge easier to call up again in the future

works far better than re-exposure to original content

testing effect aka retrieval-practice effect

to be most effective, retrieval must be repeated again and again, in spaced-out sessions so that the recall, rather than becoming a mindless recitation, requires some cognitive effort

repeated recall helps memory consolidate into cohesive representation in brain and strengthen/multiply neural routes by which knowledge can later be retrieved

as examples earlier, you get to the point where recall is reflexive – brain acts before mind has time to think

BUT little used in schools etc

if memorisation is irrelevant to complex problem-solving, do not tell your neurosurgeon

if you think of tests purely as measuring learning then miss the point of it being one of the most potent learning tools available to us

pitting learning basic knowledge vs development of creative thinking is a false choice – both need cultivating

knowledge provides platform & foundation for creativity

Studying the testing effect in the lab

1st large research study 1917 – better results by those who recited book content vs those who reread – best results from 60% of study time in recitation

2nd study 1939:  – 600 word articles: tests at various points - key results:-

1. longer 1st test delayed, the greater the forgetting
2. once a student had taken a test, forgetting nearly stopped and results dropped very little

1940 interest turned to forgetting which led to work stopping on potential of testing as retrieval practice and as learning tool – since testing interrupts forgetting, you cannot use it to measure forgetting as it contaminates results

interest in testing came back in 1967 – research re 36 word lists – repeated testing same results as repeated studying – renewed interest in testing

1978 cramming leads to higher scores on immediate tests but then faster forgetting in as littles as 2 days post 1st test – 50% loss vs 13%

next study on multiple tests effect on long term retention – story with 60 objects – immediate test 53% of objects recalled, 39% 1 week on – those who only had test 1 week later got 28% –so single test improved performance 11% 1 week on – next tests 3 immediate tests vs 1 got 53% = immunised against forgetting

so multiple sessions of retrieval practice generally better than 1, esp if spaced out

another study – fill in missing letters resulted in better memory of word

= generation effect – greater recall if test of word was 20 word pairs later vs straightaway

when retrieval practice is spaced, allowing some forgetting between tests, leads to stronger long term retention than when it is massed

Studying the testing effect in the wild

2005 – school principal not interested in memorisation – more interested in higher levels of learning – incentive was to get tech if took part – Patrice Bain teacher at the school keen – only change was addition of occasional quizzes, run for 18 months on subjects relating to Egypt, Mesopotamia, India, China – started 2006

Pooja Agarwal designed series of quizzes for third of content covered by teacher – no stakes for students on results – quizzes at start of class on material in reading but not discussed yet – 2nd at end of class after teacher covered that content for the day’s lesson – review quiz 24 hours before each unit exam – quizzes only few mins of class time

quizzes – slides – read them to students – either multiple choice q or statement of fact – clickers to answer the question – after all answered correct answer highlighted

unit exams were pencil and paper at end of semester and end of year

compelling results – full grade higher on material in quizzes – statements of fact in quizzes did not yield better results – meaning rereading does not help

2007 – same tests to 8th grade science students – end of 3 semesters – 79% C+ on material not in quizzes vs 92% A- - testing effect persisted for 8 months to end of year exams – would have been higher presumably if monthly tests till then

teachers involved in that study continued since

do not need tech – teacher reads text on slavery, kids then list 10 things about slavery

impact on kids – 64% reduced anxiety over exams, 89% felt increased learning – sad when clickers not involved – broke up class & enjoyable

teachers well advised to include in their instructional techniques

Are similar effects found at a later age?

university in St Louis – international political economics – 160-170 students – attendance issue – 10% early on then 25-35% mid-semester – ppt given out before lecture, tried to withold ppt but still students absent – 2 tests in syllabus – mid-term & final – added 9 tests that counted to score not announced re dates – students bailed rather than getting bad grades – increased numbers with high and low grades – reverted to 2 tests and added 3rd – after hearing about retrieval practice – still low scores and attendance issues – changed to 9 quizzes with no mid-/end-semester exam + dates announced – some improvement in attendance – all dates known so students could plan – better results & discussions in class – teacher said my teaching ability is only 1 factor, how I structure the learning has big effect, may be even more

Exploring nuances

college students learning prose passages on science topics – immediate recall test or restudied – after 2 days, recall test 68 vs 54%, a week later 56 vs 42%

another experiment after 1 week study-only group most forgetting 52% vs 10% in repeated testing group

how does giving feedback on wrong answers affect learning? studies show giving feedback strengthens retention more than testing alone, some evidence that slight delay in feedback produces better long-term learning

one theory is instant feedback becomes part of the task so when you do the task you miss the feedback

another says frequent interruptions for feedback makes learning sessions too variable preventing establishment of stabilised patterns of performance

in classroom delayed feedback also yields better long-term learning than immediate feedback, in prose example above, students revisited text while being asked qs about it cf open book exam – open book group got better scores on immediate test but those who got corrective feedback after completing the test retained learning better in later test

delayed feedback on written tests may help as it gives students practice that is spaced out – spacing practice improves retention

are some kinds of retrieval practice more effective for long-term learning than others?

tests requiring answer rather than multiple choice or true/false tests appear to be more effective

but even multiple choice tests can yield strong benefits

where more cognitive effort is required for retrieval, greater retention results

even a single test in a class can produce large improvement in final exam scores and these gains increase with test frequency

the testing effect is real – we may not know how! – the act of retrieving a memory changes the memory making it easier to retrieve later

retrieval practice not widely used

few students realise retrieval itself causes greater retention

is repeated testing simply a way of expediting rote learning?

testing vs rereading:-

1. can facilitate better transfer of knowledge to new contexts & problems
2. improves one’s ability to retain/retrieve material that is related but not tested

seems that retrieval practice can make info more accessible when needed in various contexts

students generally dislike testing as tool for learning esp high stakes tests but classes with frequent tests scored more highly

how does taking a test affect subsequent studying?

helps target areas of weak learning

double advantage of quizzing: know what they know and do not know, strengthening learning via retrieval

other indirect benefits of regular, low-stakes testing:-

1. improves student attendance
2. increases studying before class
3. increases attentiveness in class
4. enables students to calibrate what they know & where to concentrate on
5. helps dial down stress as no single test is make or break
6. helps instructors identify areas of low understanding amongst their students and then plug those gaps

benefits apply online & IRL

The Takeaway

practice at retrieving new knowledge or skill from memory is potent tool for learning & durable retention – applies for anything the brain is asked to remember

effortful retrieval makes for stronger learning & retention

after an initial test, delaying later retrieval practice is more potent for reinforcing retention than immediate practice – delayed retrieval requires more effort

repeated retrieval makes memories more durable but also produces knowledge that can be retrieved more readily in more varied settings & applied to wider variety of problems

cramming can produce better scores on immediate tests but advantage quickly fades because much greater forgetting after only re-reading – benefits of retrieval practice are long term

simply including 1 test yields large improvement in results – gains increase with more tests

testing does not need to be initiated by instructor – e.g. flashcards – may be unappealing as effort but that pays off in retention

taking practice tests mean student has better grasp of their progress than re-readers – helps instructor spot gaps & misconceptions & adapt instruction accordingly

giving students corrective feedback after tests prevents incorrect learning & produces better learning of correct answers

students in classes with low stakes quizzes embrace the practice  - students tested frequently rate those courses more highly

principal in the test at start of chapter – students have the base of knowledge on which to build, do not waste time going back to relearn, allows them to go to a higher level

========================

Chapter 3: Mix Up Your Practice

My research and application notes from the book

More new information for me. Fascinating reading this content and reading about the research findings. Challenged by how counter-intuitive the findings and consequent best practice are. Unsurprising, in a sense, that learners may bail out of using these techniques if it feels like they are making no progress!

Book Club Questions

Q: Discuss a rose, a bud and/or a thorn when it comes to spacing and interleaving where:-

• rose: a successful strategy you use
• bud: a strategy you would like to try
• thorn: a strategy to improve upon

A: This is a bad state of affairs. From what I have read none of my learning strategies that I have used historically feature in this book / chapter except negatively. The book continues to be an eye-opener. I have never used flashcards. I have come near to doing so in the recent past but never got around to it. I am not even sure what I would put on such cards! So flashcards may be one strategy to try. In terms of strategy to improve on, doing more than one thing at a time I could do more formally and consciously rather than as the mood takes me.

Q: What is the myth of massed practice? Why does massed practice not work?

A: Myth is that concentrating on one thing for a concentrated period of time until nailed is the best way of learning. MP does not work because it is more effective when the practice is spaced out in specific periods of training and practice.

Q: Why do we not think that spaced practice is effective?

A: It takes more effort. You do not feel the benefits. It feels like the learning is slower. No rapid improvements or affirmations.

Q: Why is spaced practice more effective than massed practice?

A: The added effort makes the learning stronger.

Q: What happened when medical students used massed (4 surgeries per day) vs spaced practice (1 each week)?

A: The latter scored better on all measures including some where damage was done by the former group.

Q: What is the takeaway from the bean bag toss where some students practiced 2/4 foot tosses and others practiced 3 foot tosses but the former won a 3 foot toss competition?

A: Varied practice improves your ability to transfer learning from 1 situation to apply in another successfully

Q: What is the takeaway from the study where students completed 4 types of maths problems either massed or interleaved?

A: Massed led to better short term results but interleaved led to better longer term results. Interleaved actually impeded short term performance.

Q: In addition to aiding retrieval, what is another benefit from interleaving?

A:  Interleaving & variation help us learn better how to assess context & discriminate between problems, selecting / applying correct solution from range of possibilities.

Q: How does using varied practice aid in building conceptual understanding?

A: Forces learning concepts and making judgements rather than memorising.

Q: What is the relationship between the speed of learning and the speed of forgetting?

A: Need to allow some forgetting time to deepen the learning.

Q: What is conceptual knowledge?

A: Conceptual knowledge requires an understanding of the interrelationships of the basic elements within a larger structure that enable them to function together. Conceptual knowledge is required for classification.

My notes from the book

may not be intuitive that retrieval practice is a more powerful learning strategy than repeated review & rereading – but most of us take for granted

practice in sport

beanbag test for throwing into a bucket 3 feet away, those who practiced into 2 and 4 feet did best always

The Myth of Massed Practice

faith in focused / repetitive practice of one thing at a time until nailed is pervasive among teachers, athletes, trainers, students

believed but our faith is misplaced

if learning is picking up new knowledge / skills and being able to apply them later, how quickly you pick something up is only part of the story – is it still there when you need it in your everyday world

practice is far more effective when broken into separate periods of training that are spaced out

practice that is spaced out, interleaved with other learning & varied produces better mastery, longer retention, more versatility – but comes at a price – needs more effort – benefits not felt – learning feels slower – you do not get the rapid improvements / affirmations

massed practice is everywhere! cramming

BUT the added effort is making the learning stronger

Spaced Practice

difference in performance of medics being trained when some trained in 1 day and others, better results, 4 lessons 1 week apart

embedding new learning in long term memory requires process of consolidation in which memory traces are strengthened, given meaning, connected to prior knowledge

rapid fire practice leans on short term memory

durable learning requires time for mental rehearsal & other processes of consolidation

increased effort to retrieve learning after some forgetting deepens learning

Interleaved Practice

of 2/more subjects/skills is more potent alternative to massed practice

interleaving impedes short term retrieval but yields better results in longer term

cf train process steps 1-10, do not do in sequence but jump about

feels slower than learning from massed practice

because short term slowness and cannot see long term benefits, interleaving is unpopular and seldom used

Varied Practice

back to bean bags – varied practice improves your ability to transfer learning from 1 situation to apply in another successfully

you develop broader understanding of relationships between different conditions and movements required to succeed in them – discern context better, develop a more flexible movement vocab – different movements for different situations

different kinds of practice engage different parts of the brain

learning gained through less challenging, massed form of practice is encoded in simpler / comparatively impoverished representation than learning gained from the varied & more challenging practice which demands more brain power & encodes the learning in a more flexible representation that can be applied more broadly

benefits of variable training for motor learning have been gaining broader acceptance albeit slowly

benefits too of variable practice for cognitive learning

cf testing on same anagrams for same word vs multiple anagrams of same word

Developing Discrimination Skills

interleaving & variation help us learn better how to assess context & discriminate between problems, selecting / applying correct solution from range of possibilities

cf text books with chapters each on same type of problem – concentrated learning

you have had no practice on the critical sorting process

but life comes at us unpredictably & out of sequence

must become adept at discerning “what kind of problem is this?” to then select / apply an appropriate solution

cf artists and their paintings and identifying / classifying birds …. commonalities proved less useful than the differences – interleaving also helped identify examples that the leaners had not seen before

myths of massed practice hard to exorcise even when experienced directly as a learner!

birds harder as members of family share traits but not all of them so needs learning concepts & making judgements, not simply memorising features

bird study suggests that strategies of learning that help students identify / discern complex prototypes (family resemblances) can help them grasp conceptual / functional differences that go beyond the acquisition of simple forms of knowledge & reach into higher sphere of comprehension

Improving Complex Mastery for Medical Students

distinction between straightforward knowledge of facts & deeper learning that permits flexible use of the knowledge may be a little fuzzy

bird classification similar to doctor diagnosing patient – every patient visit is a test

implicit memory: automatic retrieval of past experience in interpreting a new one

studies of teaching practice in a medical org – lessons:-

1. you do better on tests to demo your competency at seeing patients in a clinic if your learning experience has included that – not just reading about it
1. similar scores in written tests regardless of practice
2. best kinds of retrieval practice is that which you will use later
3. it is not just what you know but how you practice what you know that determines how well the learning serves you later
4. cf simulators
2. placing too much emphasis on variety runs the risk of underemphasising repeated retrieval practice on the basics – in the typical way a disease manifests itself in most patients
1. repeated retrieval practice is crucial to long term retention & is critical aspect of training
3. practical experience – we have lots of experience we do not learn from – what differentiates those that teach us something? reflection helps us learn from experience
1. how structure reflection as integral part of training, helping learners cultivate it as a habit
1. one medic requires students to do daily/weekly summaries of what they did, how it worked, what they might do differently next time for better results
2. speculates that daily reflection, as form of spaced retrieval practice, is probably just as critical in real world application of medicine as quizzes/tests are in building competencies in medical school

issue of lectures and conferences re listeners simply listening, cf medical conferences:-

1. chance whether they ever encounter that sort of patient
2. they do not study the material
3. not tested on the material
4. just listen
5. walk out at end

give a quiz at end of conference

email each week with 10 questions …

need to try to prevent / intervene in the process of forgetting and make learning processes systematic

These Principles are Broadly Applicable

cf college football – practice regime

coach Vince Dooley, Uni of Georgia, head coach Bulldogs football team 1964-1988, high win ratio

how players master complexities of the game:-

1. weekly cycle of 1 Saturday game to next
2. study the opposition’s game in classroom
3. discuss offensive/defensive strategies
4. taking discussion onto field
5. break strategies down to individual positions & try them out
6. knit parts into whole
7. repeat moves till clockwork
8. players keep skills sharp

believes that:-

1. have to keep practicing fundamentals from time to time to keep them sharp else atrophy
2. change methods of practice as repetition is boring

players’ mastery of playbook

spaced learning – special plays each Thursday, varied sequence

v specific daily/weekly schedule interleaving practice

variations based on different ways opposition may respond

not just physical but go through plays in mind

central role of quarterback with final meetings on morning of game

The Takeaway

we learn better through single-minded focus & dogged repetition – visible improvement that comes during practice – practice – practice

heightened performance in acquisition phase is “momentary strength” rather than “underlying habit strength”

the very techniques that build habit strength – spacing, interleaving, variation – slow visible acquisition & fail to deliver the improvement during practice that helps to motivate & reinforce our efforts

cramming – cf binge eating

simple spacing out study & practice in instalments, allowing time to elapse between them makes learning & memory stronger – building habit strength

interval long enough to not feel like mindless repetition – minimum to allow some forgetting but not enough to require relearning

sleep helps in memory consolidation so at least a day between sessions

flashcards 1 example of spacing – repetitions of 1 card amongst others

cf Leitner box: 4 file card boxes:-

1. the cards you are poor at remembering – need practice frequently
2. not as poor say half as much as 1
3. practised less often
4. even less often

the underlying idea: the better your mastery, the less frequent the practice

beware the familiarity trap – no shortcuts!

interleaving 2/more subjects during practice also provides spacing & develops ability to discriminate later between different kinds of problems & select right tool from your growing solutions toolkit

switch in interleaving before 1 subject is complete

varied practice: helps learner build broad schema – assess changing conditions & adjust responses to fit

arguably interleaving & variation help learners reach beyond memorisation to higher levels of conceptual learning & application, building more rounded, deep, durable learning aka underlying habit strength

“blocked practice” easily mistaken for “varied practice”: same things done in sequence cf doing flashcards in same sequence – do not do that – shuffle them!

spacing, interleaving & variability are natural features of how we conduct our lives = learning from experience

need to cultivate habit of reflection which is a form of retrieval practice:-

1. what happened
2. what did I do
3. how did it work out
4. + elaboration – what would I do differently next time

Doug Larsen: making the brain work is actually what seems to make the difference – bringing in more complex networks then using those circuits repeatedly which makes them more robust

Chapter 4: Embrace Difficulties

My research and application notes from the book

Again more content that is new to me. I have started saying to people that the whole book is new content for me!

Re spaced retrieval, I had spaced out driving lessons. Main reason was cost to spread the cost NOT to spread the learning! Dad also let me drive his car in evenings between lessons to help me learn. I passed my test first time.

Interesting reading about the use of missing words and getting learners to fill them in. I saw this in some American content on leadership. At the time I thought this was almost childish!! … now I am not so sure!!

The phrase “salt away” was used in the context of learning. I looked up the definition – I love doing this! – it means “to hoard or save (money, valuables, etc)”.

This thing about forgetting learning partially to help deepen learning later is defo a challenge. This made me think about the term “unlearning” that seems to be gaining increased currency. For me all learning is valuable and unlearning is possibly counter-productive in terms of forgetting what does not work and knowing not to do that again!

“unlearn”: discard (something learned, especially a bad habit or false or outdated information) from one's memory.

The section on how learners waste energy with anxiety about tests/challenges reminded me of something that I say a lot to myself and other people … lots of stress is not good but some stress is good to yield maximum performance. Wondering whether learners can channel all their energy into simply retrieving knowledge without any nerves whatsoever.

I was taken with the “blundering” title in the gardener example and looked up the definition: making or characterized by stupid or careless mistakes; clumsy.

It was good to see the recommendation to start projects before you fully know how to do it & before you know what you are getting into. I have been doing this increasingly over the past 5 or so years especially in the context of learning with others. I have been greatly encouraged being more like that when hearing Mel Robbins talk about her 5-Second Rule.

Book Club Questions

Q: If you could share only 1 thing with educators about embracing difficulties, what would it be?
A: Make it challenging for the students – do not make it too easy for them to recall what they are learning.

Q: If you could share only 1 thing with students about embracing difficulties, what would it be?
A: Channel your energy into recalling what you have learned not the anxiety about doing so.

Q: How does learning occur i.e. what are the steps? What happens at each step? What does each step mean?
A:
1 – encoding – converts sensory perceptions into meaningful representations in the brain – memory traces in the brain
2 – consolidation – strengthens mental representations for long-term memory – brain reorganises and stabilises memory traces
3 – retrieval – retrieve what we know – practice & apply what we have learned by recall or practice

Q: What 2 things are required for learning to be durable?
A:
1- recode & consolidate new material from short-term into long-term memory, we must anchor it there securely
2 - associate the material with a diverse set of cues that will make us adept at recalling the knowledge later

Q: Complete this sentence: There is virtually no limit if we ..
A: (to how much learning we can remember) … if we relate it to what we already know

Q: Complete this sentence: knowledge is more durable if ...
A: it is deeply entrenched – you have thoroughly comprehended a concept, it has practical importance or keen emotional weight in your life, connected with other knowledge in your memory

Q: What's the relationship between learning, unlearning and retrieval cues?
A: as you live your life, you often need to forget cues associated with older, competing memories so you can associate them successfully with new ones

Q: Describe how the basketball (Simon: baseball??) team increased effort during learning (practice). Why did their new practice habits improve outcomes?
A: 1 group did massed practice, same pitch types in sets, 1 group random pitch types
The players learned and practiced more when they did not know the pitch type they would face each time.

Q: What are the 6 ways that increasing effort during practice improves learning?
A:
1: reconsolidating memory
2: creating mental models
3: broadening mastery
4: fostering conceptual learning
5: improving versatility
6: priming the mind for learning

Q: What is generation? Why does it matter?
A: being asked to supply an answer to a problem rather than being presented with the answer or a solution. It has the effect of strengthening your memory of the material and your ability to recall it later.

Q: What is reflection? Explain the college psychology example.
A: Taking time to review what has been learned from an experience / class and asking yourself questions about that. Students in that class scored higher marks on subjects tested that they had reflected on in their own words.

Q: What distinguishes desirable difficulties from undesirable difficulties?
A: Desirable difficulties are ones that trigger encoding and retrieval processes that support learning, comprehension and remembering. Undesirable ones are those that the learner cannot overcome.

My notes from the book

story of Mia Blundetto, 1st Lt Marine Corps, Logistics – tough lady – her jump school training is great example of how some difficulties that elicit more effort & slow down learning will more than compensate for their inconvenience by making learning stronger, precise, enduring = desirable difficulties (Elizabeth & Robert Bjork) = short term impediments that make for stronger learning

jump school – cannot take notes – model of learning through desirable difficulty – no notebook or taking notes – listen, watch, rehearse, execute – testing is principal instructional medium – test is in the doing

parachute landing fall (PLF) – ground level then higher – different ways of falling – executing the fall randomly from all directions – mass exit, with eqt etc

knowledge / skills to be acquired are many & practice is spaced / interleaved both by default & by necessity to cover all that must be mastered & integrate the disparate components – if make it to end 5 jumps from aircraft to get “wings”

when she jumped 3rd time, fell into another person’s parachute but not phased and all OK because of training

when confidence is based on repeated performance, demo-ed through testing simulating real-world, you can lean into it

How Learning Occurs

1: Encoding

the process of converting sensory perceptions into meaningful representations in brain

still not perfectly understood

memory traces: new representations in the brain

the experiences / learning that we want to salt away for the future must be made stronger, more durable

2: Consolidation

strengthening these mental representations for long-term memory

new learning is labile (meaning not fully formed, easily altered)

the brain reorganises & stabilises the memory traces

may occur over several hours or longer

involves deep processing of the new material – during which it is believed that the brain replays / rehearses the learning, giving it meaning, filling in blank spots, making connections to past experiences & to other knowledge already stored in long-term memory

prior knowledge is pre-req for making sense of new learning & forming those connections is important task of consolidation

sleep seems to help memory consolidation but in any case consolidation & transition of learning to long term storage occurs over a period of time

analogy of how brain consolidates new learning – composing an essay:-

1. 1st draft – rangy, imprecise – discover what you want to say by trying to write it
2. 2 revisions – sharpened it up, cut some out
3. leave it to ferment
4. 2 days later, return to it and it has become clearer in your mind
5. 3 main points
6. connect them to examples and supporting evidence known to your readers
7. rearrange & draw together the elements of your argument to make it more effective / elegant

cf the process of learning

cf reconsolidation – old memory mixing with more recent learning

beware leaning on short-term memory where very little mental effort requred

aim for when learning is deeper and you retrieve it more easily in future

3: Retrieval

learning, remembering, forgetting work together in interesting ways

pre-reqs for durable / robust learning:-

1. as we recode / consolidate new material from short-term memory into long-term memory, we must anchor it there securely
2. we must associate the material with a diverse set of cues that will make us adept at recalling the knowledge later

effective retrieval cues often overlooked as aspect of learning

task is more than memorisation … also need to be able to retrieve it when we need it

example of knot tying …

knowledge, skills & experiences that are vivid & hold significance and those that are periodically practices stay with us

cf mental rehearsal – a form of spaced practice

4: Extending Learning: Updating Retrieval Cues

virtually no limit to how much learning we can remember as long as we relate it to what we already know

the more we learn, the more possible connections we create for further learning

our retrieval capacity is severely limited – most of what we have learned is not accessible to us at any given moment

this is helpful to us as it reduces the knowledge we need to sift through at point of need

knowledge more durable when deeply entrenched:-

1. you have firmly / thoroughly comprehended a concept
2. it has practical importance or keen emotional importance in your life
3. connected with other knowledge that you hold in memory

how readily you recall knowledge is determined by:-

1. context
2. recent use
3. number & vividness of cues that you have linked to the knowledge & can call on to help bring it forth

through life often need to forget cues associated with older, competing memories to associate them successfully with new ones

cf learning new language, you may have to forget old language translations for the same word/phrase

knowledge that is well-entrenched is easily relearned later after a period of disuse or after being interrupted by competition for retrieval cues

cues for new displace cues for old – cf driving on left or right depending on country

paradox – some forgetting is often essential for new learning

cf using Mac after using Windows

cf Jack and Jill from kids story when your friend Jack has girlfriend Joan! then later new girlfriend Jenny!

alliteration can be handy cue or subversive one

critical point is that as you learn new things you do not lose from long-term memory most of what you have learned well in life – rather through disuse or reassignment of cues you forget it in the sense that you cannot recall it easily

cf photos of past bring back memories

context can unleash memories

cf taste or smells

5: Easier Is Not Better

the easier knowledge of skill is for you to retrieve, the less your retrieval practice will benefit your retention of it – the more effort needed, the more retrieval practice will entrench it

cf baseball team practice in 2 halves, 1 with same pitches, 1 with varied, over a period of time – the latter better results eventually but not at 1st

the more you have forgotten about a topic, the more effective relearning will be in shaping your permanent knowledge

How Effort Helps

1: Reconsolidating Memory

effortfully recalling learning per spaced practice requires that you reload / reconstruct components of the skill or material anew from long term memory rather than mindlessly repeating them from short term memory

during this recall, the learning is made pliable again:-

1. most salient aspects become clearer
2. consequent reconsolidation
1. helps to reinforce meaning
2. strengthen connections to prior knowledge
3. bolster the cues and retrieval routes for recalling later
4. weaken competing routes

spaced practice, allows some forgetting between sessions, strengthens learning / cues / routes for fast retrieval when that learning is needed again

the more effort that is required to recall a memory or execute a skill, provided that the effort succeeds, the more the act of recalling  or executing benefits the learning

fluency gained through massed practice like rereading is transitory & our sense of mastery is illusory

the effortful process of reconstructing the knowledge is what triggers reconsolidation & deeper learning

2: Creating Mental Models

with enough effortful practice, a complex set of interrelated ideas or sequence of motor skills fuse into a meaningful whole forming a mental model

cf learning to drive – each aspect becomes a mental model eg parallel parking

mental models: forms of deeply entrenched, highly efficient skills or knowledge structures that, like habits, can be adapted / applied in varied circumstances

expert performance via thousands of hours of practice in your area of expertise in varying conditions – becomes a vast library of such mental models that enables you to correctly discern a given situation and instantaneously select & execute the correct response

3: Broadening Mastery

retrieval practice that you do at different times in different contexts and that interleaves different learning material has benefit of linking new associations to the material

this process builds interconnected networks of knowledge that bolster / support mastery of your field

also multiplies the cues for retrieving the knowledge, increasing the versatility with which you can later apply it

cf chef, fly fisher, trick cyclist

4: Fostering Conceptual Learning

how we learn concepts like difference between cats and dogs is by randomly coming across dissimilar examples

spaced / interleaved exposure characterises most humans’ normal experiences

a good way to learn as this type of exposure strengthens discrimination skills (noticing particulars) as well as induction skills (surmising the general rule)

cf bird types & painters earlier in book

interleaving produces difficulty in retrieving past examples which further solidified the learning

a 2nd type of boost of learning is how examples or types are distinctive and why they need different interpretation or solution

5: Improving Versatility  

the retrieval difficulties posed by spacing, interleaving, variation are overcome by invoking same mental processes needed later in applying the learning in everyday settings

= practice like you play and play like you practice

= transfer of learning: ability to apply what you have learned in new settings

cf all the earlier examples in the book

6: Priming the Mind for Learning

best to struggle with a problem yourself before being shown how to solve it as subsequent solution is better learned and more durably remembered

Other Learning Strategies that Incorporate Desirable Difficulties

certain kinds of interference can produce learning benefits – the positive effects are sometimes surprising

e.g. blurry text is better recalled than clear text!

e.g. ditto lecture in different sequence to text book

e.g. missing letters in some words still readable

each of these introduces a difficulty – disruption of fluency – that makes the learner work harder to construct an interpretation that makes sense

the added effort increases comprehension & learning

unless it completely obscures meaning or cannot be overcome!!

the act of trying to answer q or attempting to solve a problem rather than being presented with the info or solution = generation

e.g fill in the blank words

strengthens memory of the material and ability to recall it later

supply an answer rather than multiple choice often provides stronger learning benefits

doing short essay is stronger still

overcoming these difficulties is a form of active learning

when asked to supply answer a q new to you, power of generation to aid learning even more evident

as you retrieve related knowledge from memory, you strengthen the route to gaps in your knowledge even before answer is provided to fill it …

.. and when you fill it, connections made to related material that is fresh in your mind from the effort

unsuccessful attempts to solve a problem encourage deep processing of answer when it is later supplied – creating fertile ground for its encoding in a way that simply reading the answer cannot

better to attempt a solution & supply incorrect answer than not to make attempt

reflection: the act of taking a few mins to review what has been learned from an experience (or recent class) & asking yourself questions:-

• key ideas
• examples
• how do these relate to what I already know
• when you practiced new skills – what went well, not so well, what do I need to learn for better mastery, what strategies to use next time for better results

activities aiding reflection:-

1. retrieval
2. elaboration – connecting new to existing knowledge
3. generation – rephrasing key ideas in your own words, visualising & mentally rehearsing what you might do differently next time

“write to learn” – students reflect on recent class in brief writing exercise expressing the ideas in their own words & relate to other subjects covered in class or outside class

all yield learning benefits

Failure and the Myth of Error-less Learning

BF Skinner advocated “error-less learning” in 50s/60s – errors by learners are counter-productive & result from faulty instruction

e.g. short input, test right then = no chance of making error

such short-term memory recall is ineffective learning strategy, errors an integral part of striving to mastery

challenge is aversion to failure

instructors may believe that errors will be the things that are learned

errors produce stronger learning / retention of correct info than more passive leaning strategies with corrective feedback

those who are taught learning is a struggle will go on to have greater propensity to tackle tough challenges and on to mastery

beware creating aversions to risk taking and experimentation

can cause anxiety that in itself yields poorer results – part of memory used to monitor their performance instead of on the problem!

study in France with students with anagram test – difficulty can create feelings of incompetence that engender anxiety which then disrupts learning and students do better when given room to struggle with difficulty

so not all difficulties in learning are desirable

also difficulty in learning new things is not only to be expected but can be beneficial

view failure as sign of effort and a turn in road not the end of the road

failure becomes an essential experience on path to mastery

following French study, led to Paris’ Festival of Errors and later San Fran’s FailCon

Edison viewed failure as source of inspiration – found 10k ways that do not work – perseverance in face of failure is key to success

cf Steve Jobs getting fired from Apple – the heaviness of being successful was replaced by the lightness of being a beginner again, less sure about everything, freed me to enter one of the most creative periods of my life

not the failure that is desirable but the dauntless effort despite the risks, discovering what does or does not work that sometimes only failure can reveal

An Example of Generative Learning

learner generating the answer rather then recalling it

generation is equivalent of trial and error

Bonnie Blodgett – The Blundering Gardener – start your projects before you fully know how to do it & before you know what you are getting into – the risk of knowing what you are getting into is that it becomes overwhelming obstacle to starting – she gives us permission to make mistakes and get on with it – learn by doing – did not learn Latin plant names at 1st, but discovered they were helpful & started learning them because they were helpful – cues from Latin and French at school

clearly not everything can be learned in a blundering way re cost of failure – pilots & surgeons!

Undesirable Difficulties

Elizabeth / Robert Bjork - “desirable difficulties” - difficulties are desirable because “they trigger encoding and retrieval processes that support learning, comprehension, and remembering. If, however, the learner does not have the background knowledge or skills to respond to them successfully, they become undesirable difficulties.”

techniques in book so far all help lead to stronger learning & retention

cannot yet be definitive on “undesirable difficulties”

but include impediments you cannot overcome!

to be desirable, difficulty must be something learners can overcome through increased effort

undesirable difficulties:-

1. do not strengthen the skills you need
2. do not address the kinds of challenges you are likely to encounter in the real-world application of your learning]

will be different for different people

The Takeaway

learning is a 3-step process at least:-

1. initial encoding of info held in short-term working memory before being consolidated into a cohesive representation of knowledge in long-term memory
2. consolidation reorganises & stabilises memory traces, gives them meaning & makes connections to past experiences & to other knowledge already stored in long-term memory
3. retrieval updates learning & enables you to apply it when you need it

learning always builds on a store of prior knowledge

we interpret / remember events by building connections to what we already know

long-term memory is virtually limitless – the more you know, the more possible connections you have for adding new knowledge

the vast capacity means the ability to recall what you know when you need it is key – dependent on repeated use of the info (keep retrieval routes strong) & your establishing powerful retrieval cues that can reactivate the memories

periodic retrieval of learning helps strengthen connections to memory & cues for recalling it, while also weakening routes to competing memories

the more difficult the practice, the greater the benefit

effortful retrieval strengthens the memory but also makes the learning pliable again, leading to its reconsolidation which in turn helps update your memories with new info & connect them to more recent learning

repeated effortful recall / practice helps integrate learning into mental models in which  set of inter-related ideas or sequence of motor skills are fused into a meaningful whole that can be adapted and applied in later settings eg driving a car or baseball hitting

when practice conditions are varied or retrieval is interleaved with practice of other material, we increase our discrimination and induction and the versatility with which we can can apply the learning in new settings later  - builds new connections, expanding & more firmly entrenching knowledge in memory & increasing cues for retrieval

trying to come up with an answer before being told or solving a problem before seeing the solution, leads to better learning & longer retention of correct answer or solution even when your attempts are wrong as long as you get corrective feedback

Chapter 5: Avoid Illusions of Knowing

My research and application notes from the book

Another informative and helpful chapter.

Interested in the issue of forgetting the basic steps when our understanding and experience deepens which then makes it harder to explain the basics to a novice.

The discussion of basic steps and wider experience reminded me of Wikipedia content on subjects vs specialist textbooks vs articles in peer-reviewed academic journals. All are helpful but people need a way in – wiki-type pages are helpful for intro to a subject and then linking off to more detailed pages and so on.

Memories of being picked in teams last-ish at school and play. Always incentivised me to perform well!

I found this Eric Mazur – Peer Instruction – video: https://youtu.be/Z9orbxoRofI

cultivate – This word has started stalking me. I am hearing it and reading it in lots of areas of my life. This may be an early contender for my One Word of 2020. The word means:

1. prepare and use (land) for crops or gardening.

   • break up (soil) in preparation for sowing or planting.

   • raise or grow (plants), especially on a large scale for commercial purposes.

   • BIOLOGY

     grow or maintain (living cells or tissue) in culture.

2. try to acquire or develop (a quality or skill).

   • try to win the friendship or favour of (someone).

   • try to improve or develop (one's mind).

Calibrate is another word via this book that has resonated. It means:

• mark (a gauge or instrument) with a standard scale of readings.

• correlate the readings of (an instrument) with those of a standard in order to check the instrument's accuracy.

• adjust (experimental results) to take external factors into account or to allow comparison with other data.

• carefully assess, set, or adjust (something abstract).

The chapter references the power of learning lessons from practical experience. This is standard practice in most project management best practice from the start of, during and at the end of each project. Clearly, this can also be applied in non-project work.

Book Club Questions

Q: How do you correct students’ illusions by giving feedback that is quick, easy and does not require grading?
A: This is not something I am normally engaged in. I am not an educator. It was good to read the best practice on this in the chapter.

Q: Are your students aware of their own illusions of knowing? How do they react when you discuss illusions with them?
A: This student had some awareness but not much. This is scary. My reaction is shocked that these illusions are so bad! And then I move on to wanting to address those illusions!

Q: What are the 2 systems in the brain? What role do they play?
A: System 1 (Automatic) and System 2 (Controlled). System 1 is the reflex action one, how we do things unthinkingly out of practice & experience, System 2 is the thinking one that processes information to come to a conclusion.

Q: What is the paradox between our memory and our ability to learn? What should we do about it?
A: That the changeable nature of our memory not only can skew our perceptions but also is essential to our ability to learn. Use our memory to help us learn. Be aware that our memory may be misleading and factor that into your judgement.

Q: What are our major biases?
A: Hunger for narrative, distorted memory, imagination inflation, suggestion, interference, curse of knowledge, hindsight bias (knew-it-all-along effect), feeling of knowing, fluency illusions, social influence, false consensus effect, flashbulb memories

Q: What’s the lesson from the tapping the music experiment?
A: That the questioner overestimates the probability of the listener getting a tune from a list of 25 tunes correct,

Q: How do you make metacognition more accurate?
A: Get more accurate feedback on performance, understand why things did not work, understand yourself what makes for good performance, address our illusions & misjudgements

Q: Complete this sentence: even when we know the strategies …
A: we need to keep putting them into practice and not abandon too early those things that we think we know.

Q: Which cues should you use and not use to determine your level of mastery?
A: No – whether something feels familiar or fluent, ease of retrieving shortly after learning 
Yes – ease of retrieval after a delay in learning, create mental model including connections to past learning, explain a text in your own words

Q: What are the tools for calibrating your own judgement?
A: seek corrective feedback, peer review, working alongside a more experienced colleague, multi-disciplinary teams, simulations of real life situations

My notes from the book

at the core of our effectiveness is our ability to grasp world around us & measure our own performance

we constantly make judgements about what we know, don’t know & whether we are capable of handling a task or solving a problem

monitoring our own thinking is called metacognition

important part of this skills is  being aware of the ways we delude ourselves

consequences of poor judgments all around us

we are all hardwired to make errors in judgement

good judgement is a skill we must acquire, becoming an astute observer of one’s own thinking & performance

starting point is disadvantage:-

1. when we are incompetent, we tend to overestimate our competence & see little reason to change
2. as humans, we are readily misled by illusions, cognitive biases & stories we construct to explain the world around us & our place in it

to become more competent or even expert:-

1. we must learn to recognise competence in others
2. become more accurate judges of what we ourselves know / don’t know
3. adopt learning strategies that get results
4. find objective ways to track our progress

Two Systems of Knowing

Daniel Kahnemen’s “Thinking Fast and Slow” describes our 2 analytic systems:-

1. System 1 – automatic
1. unconscious
2. intuitive
3. immediate
4. draws on our senses & memories to size up a situation in blink of an eye
5. automatic
6. deeply influential
7. but susceptible to illusion
8. powerful because draws on our accumulated years of experience & our deep emotions
9. survival reflex in times of danger
10. deftness earned by thousands of hours of deliberate practice in our chosen field of expertise
2. System 2 – controlled
1. slower process of conscious analysis & reasoning
2. thinking that considers choices, makes decisions, exerts self-control
3. use it to train System 1 to recognise/respond reflexively
4. depend on System 2 to manage yourself
1. checking your impulses
2. planning ahead
3. identifying choices
4. thinking through their implications
5. staying in charge of your actions

interplay between the 2 systems is theme of Daniel Goleman’s book “Blink”

learning when to trust your intuition & when to question it is a big part of how you improve your competence in world at large and in any field you want to be expert in

“spatial disorientation”: aeronautical term for deadly combo of losing sight of horizon & relying on human sensory perception that is inconsistent with reality but so convincing that pilots conclude instruments have failed

Kahneman: System 1 can be very hard to overrule

Illusions and Memory Distortions

David Dunning: human penchant for motivated reasoning: the sheer genius people have at convincing themselves of congenial conclusions while denying truth of inconvenient ones

we have a hunger for narrative arising from our discomfort with ambiguity & arbitrary events

we search for explanations when surprising things happen

urge to resolve ambiguity can be surprisingly potent even when subject is inconsequential

eg overhearing 1 side of convo is more distracting than hearing both sides

we need a rational understanding of our own lives – a cohesive story

we gravitate to narratives that best explain our emotions

in this way narrative and memory become one

narrative provides not only meaning but also a mental framework for imbuing future experiences & info with meaning

you can see how quickly personal narrative is invoked to explain emotions via comments on online article comments re agree, not agree with personal examples

nearly impossible to avoid basing one’s judgement on subjective experience – narrative of memory becomes central to our intuitions regarding judgements we make and actions we take

our capacity to strengthen, expand, modify memory is central to how we deepen our learning & broaden the connections to what we know and what we can do

serves you well to stay open to the fallibility of your certainties – even your most cherished memories may not represent events in the exact way they occurred

memory can be distorted in many ways – we interpret a story in the light of our world knowledge, imposing order where there is none to make a more logical story – memory is a reconstruction – we cannot remember all aspects of an event, so we remember those elements with greatest emotional significance for us & fill in gaps with details of our own that are consistent with our narrative but may be wrong

we remember things that were implied but not specifically stated cf Helen Keller prose recall

imagination inflation: when asked to vividly imagine an event, we sometimes believe it actually happened – can be in our memory as deeply as real events

suggestion: simply due to how a question was asked cf leading questions

cf hypnosis – people’s minds roam freely, some recall will be flawed

interference from other events can distort memory cf seeing mug shots with police and then lineup includes one of those but was not him/her

curse of knowledge – tendency to underestimate how long it will take someone else to learn what we know

close relation to hindsight bias (knew-it-all-along effect) – believe that past events more predictable now than at the time

feeling of knowing – hearing accounts that sound true – cf repeated hearing of ads – cf big lie in propaganda, repeated, people believe true

fluency illusions – mistake fluency with a text for mastery of its content -  cf re-reading books

social influence – memories align with those around us including other people’s erroneous recall = memory conformity or social contagion of memory – note that other people’s correct memories can add detail to yours

false consensus effect – we generally fail to recognise the isiosyncratic nature of our personal understanding of the world & interpretation of events and that ours differ from others’

confidence in a memory is not reliable indication of its accuracy

cf flashbulb memories – vivid images we retain – where we were when we got the news, how we felt – but some of these may not be accurate – often the deepest memories at the time have changed most over time

Mental Models

the steps needed to successfully undertake a specific task

the better you know something the more difficult it is to teach it

as you get more expert in complex areas, your models get more complex & the steps fade into the background of memory (the curse of knowledge)

metacognitive error – misjudgment of matchup between what the teacher knows and what their student knows

best person to realise this is a fellow student

mental models may break when we encounter a new-to-us situation or worse we think it “fits” one of our existing models

we must cultivate the ability to discern when our mental models are not working

Unskilled and Unaware Of It

incompetent people cannot improve when they are unable to distinguish between incompetence and competence

this is the Dunning-Kruger effect

they can become more competent by learning the skills to judge their own performance – make their metacognition more accurate

why do incompetent people fail to learn through experience:-

1. people seldom receive negative feedback about their skills / abilities from others in life due to people not wanting to give it
2. even with feedback they need to understand what went wrong
3. for success everything must go right but to fail, lots of things could happen to make that happen
4. some people cannot identify how they are performing to learn

cf being picked last in sports teams in 2 teams with captains

this content should give us all pause

esp an issue for student-directed learning e.g. whatever intrigues the learner is what will result in the best learning

those students who employ the least effective study strategies overestimate their learning the most & then due to that misplaced confidence they are not inclined to change their habits

most students will learn most under an instructor who knows where improvement is needed and structures the practice to achieve that

to address illusion / misjudgement, replace subjective experience with objective measures outside of ourselves so that they square up

Tools and Habits for Calibrating Your Judgement

most important – frequent use of testing & retrieval practice – verifies what you actually know vs what you think you know

as a learner, you can use any number of practice techniques to self-test your memory

do not drop things to test when recalled OK once or twice

space your testing, vary your practice, keep the long view

peer instruction (Eric Mazur):-

1. read material before class
2. lecture
3. interspersed with quick tests presenting students with a conceptual q & gives 1-2 mins to grapple with it
4. in small groups reach consensus on right answer

pay attention to cues you are using to judge what you have learned

create a mental model of the material that integrates the various ideas, connects them to what you already know & enables you to draw inferences

how ably you can explain a text is excellent cue for judging comprehension – recall salient points, use own words, explain why they are significant and how relate to larger subject

instructors should give correct feedback & learners should seek it

in many fields peer review serves as external gauge providing feedback on one’s performance

e.g. case conference post patient dying or other poor outcome

your judgement & learning can also be calibrated by working alongside a more experienced  partner – apprentice model

teams with complementary experience and skills

training that simulates real world scenarios help learners / trainers assess mastery & focus on areas where understanding or competency need to be raised

mistakes in field are powerful source of learning

Chapter 6: Get Beyond Learning Styles

My research and application notes from the book

Interesting to see learning styles covered. Lots of anti- sentiment in L&D Twitter Chats. Personally agnostic so not religious about them either way. This chapter made interesting reading.

I do like a test to see what personality etc the test assesses me as for me to consider. I never blindly accept the result.

Interesting section on strengths and weaknesses in the dynamic testing section. Marcus Buckingham would advocate us concentrating almost exclusively on strengths. We often concentrate development time and money on weaknesses rather than further developing strengths. And then the book does go on to address the issue of weaknesses re making you more versatile if you address them. Good to read that but I remain more of a strengths-focused person.

The mental models and Lego example reminded me of city planning models for enterprise architecture in my IT field. John Zachman has been an inspirational person in my career re his Zachman Framework.

Never done any of this structure building on paper etc explicitly away from my day job of spec-ing/ designing IT solutions.

Still not really sure what this structure building entails as I go about learning new things and building on what I already know.

Book Club Questions

Q: What is one thing you would tell an educator to persuade them to move away from learning styles and move toward more effective strategies?
A: Different subjects need a greater emphasis on one learning style than others e.g. word-based subjects vs visual-based subjects.As educators we miss out by not using all methods to cater for all learning styles.

Q: What is one thing you would share with parents about the myth of learning styles?
A: Similar to the educators response. We want to teach your kids in a variety of ways that are specific and ideal to each subject they are learning.

Q: What is “rule building”?
A: Extract underlying principles or rules from new experiences that can be applied to the same or different situations later.

Q: What is always near the top of the list of differences that matter most?
A: How a person sees themselves and their abilities. Also the level of a person’s language fluency and reading ability.

Q: When does “learning style” matter?
A: When the subject demands the use of an instructional style matching the learning style regardless of what the learners’ preferences are..

Q: What are the two kinds of intelligence?
A: Fluid: ability to reason, see relationships, think abstractly, hold info in mind when working on a problem.
Crystallised: one’s accumulated knowledge of the world and the procedures / mental model one has developed from past learning / experience

Q: What are Sternberg’s buckets of intelligence?
A: Analytical, creative & practical.

Q: What is “structure building"? Explain the Lego analogy.
A: Building a coherent mental framework as things are learned. Means discarding non-essential info from what is being learned. Cf lego pieces, how you piece them together to make sense of the pieces in totality then mix them across box sets..

Q: Why are “high structure” learners typically stronger than “low structure” learners? How can we support low structure learners?
A: They can apply their learning in different contexts and build on their existing knowledge more formally. Low structure learners can be supported by including questions on core parts of the subject on the way through those subjects so it is clear what the main points are.

Q: What is rule vs example learning?
A: Abstracting the underlying principles/rules that differentiate the examples being studied vs looking at lots of examples of things being studied.

My notes from the book

all learners are different

the idea that we all have distinct learning styles is now part of the folklore of ed practice and part of how people see themselves

e.g. it is said that some learn better visually or textually – and that if you are taught in a style that is not your preferred then you are at a disadvantage in your learning

we agree that everyone has learning preferences but not persuaded that you learn better when the manner of the instruction fits your preferences

but there are other differences ion how people learn that are important

Active Learning from the Get-Go

story of Bruce Hendry illustrates some of these differences:-

1. take charge of your own education – a habit
2. draw lessons from experience to improve focus & judgement
3. knit what you learn into mental models

= rule learning, structure building

better to learn by extracting underlying principles / rules from new experiences than just accepting things at face value and not inferring lessons to apply later

single out salient concepts from less imp info and link these key ideas into a mental structure to be more successful learner

Sam Leppla taught Bruce some lessons in financial markets – relationship of price, supply, demand, value

Barney Donahue added knowledge on railroads

big success with one bankrupt railroad – moved on to Kaiser Steel

summary: he built own set of rules for what makes investment opportunity attractive, stitching rules into template, finding new & different ways to apply it

some stated reasons why he thought he was successful … but … really interesting = how he figured out what knowledge he needed, how he went after that knowledge, how early setbacks seeded skills of shrewder judgement, how developed a nose for value where others only saw trouble

the story of Bruce makes clear that some learning differences matter more than others

how you see yourself and your abilities

whether you think you can or cannot, you are right

cf Carol Dweck

stories of people with dyslexia who have become high business achievers despite their learning disabilities

Richard Branson: at some point I think I decided that being dyslexic was better than being stupid = Branson’s personal narrative of exceptionalism

the stories we create to understand ourselves become the narrative of our lives re past, present & future

what you tell about your ability plays a part in shaping the ways you learn and perform

but differences in skills & your ability to convert new knowledge into building blocks for further learning also shapes your routes to success

cf star players vs star coaches where coaches were not best players

other differences we may think count for a lot e.g. learning style actually do not

language fluency and reading ability

challenge of dylsexia in processing text

compensating for this disability, people possess, or develop, greater capacity for creativity & problem solving – better at big picture

some neurological differences can count for a lot in how we learn & for a subset of people a combo of high motivation, focused & sustained personal support & compensating skills or “intelligences” have enabled them to thrive

belief in learning styles is pervasive – schools and workplace

not supported by science – instils a corrosive, misguided sense of diminished potential

no consistent theoretical pattern in the multiple views of these styles

Neil Fleming – VARK – 1 aspect of a learning style, 18 dimensions including temperature, food intake etc

Kenneth/Rita Dunn – 6 aspects of learning – environmental, emotional, sociological, perceptual, physiological, psychological

Honey & Mumford – Learning Styles Questionnaire – popular in workplaces – styles of activist, reflector, theorist, pragmatist – address those with low scores to become more versatile learner

true that we all have preference for how we like to learn new material

premise of learning styles is that we learn better when mode of presentation matches the style in which an individual is best able to learn

2008 – 4 cognitive psychologists conducted a review to see if this was supported by scientific evidence:-

1. what forms of evidence are needed for orgs to justify basing their instructional styles on assessment of students’ learning styles?
2. does this kind of evidence exist? – answer was no – virtually none validate it, several contradict it

their review showed that more imp for mode of instruction match the subject – visual for geometry, text for poetry etc

lack of evidence does not mean all theories are wrong

some styles theories may be valid … but which

until evidence is produced, makes more sense to emphasise instructional techniques per this book with the evidence – benefitting learners regardless of their style preferences

Successful Intelligence

intelligence is a learning difference that we know matters

what is it?

two kinds:-

1. fluid
1. ability to reason
2. see relationships
3. think abstractly
4. hold info in mind while working on a problem
2. crystallised
1. a person’s accumulated knowledge of the world
2. the procedures or mental models one has developed from past learning & experience

together they help us:-

1. learn
2. reason
3. solve problems

traditionally IQ tests have been used to measure a person’s logical/verbal potential – ratio of mental age to physical age x 100

used to be thought that IQ was fixed from birth but traditional views of intellectual capacity are being challenged

Howard Gardner explains 8 different kinds of intelligence:-

1. logical-mathematical
2. spatial – 3D, visualise with mind’s eye
3. linguisitic – words/languages
4. kinesthetic – physical dexterity
5. musical – sensitivity to sounds, rhythms, tones, music
6. interpersonal – “read” other people & work effectively with them
7. intrapersonal – understand your own self, make accurate judgements of one’s knowledge, abilities, effectiveness
8. naturalistic – discriminate / relate to one’s natural surroundings (e.g. gardener, chef, hunter)

as with learning styles theory, the multiple intelligences model has helped educators to diversify the kinds of learning experiences they offer

the MI model elevates sheer variety of tools in our native toolkit

but both theories lack empirical validation

Gardner himself says assessing yourself vs his model is more art than science

Robert J Sternberg’s model proposes 3 intelligences:-

1. analytical – complete problem-solving tasks e.g. in  tests
2. creative – synthesise / apply existing knowledge/skills in new / unusual situations
3. practical – adapt to everyday life – understand what needs to be done in a specific setting then do it = street smarts

supported by empirical research – study in Kenya – some kids great at handling tribal medicines but scored very low in school tests – families valuing practical knowledge more than academic knowledge

2 important ideas here:-

1. trad measures of intelligence fail to account for environmental differences – the kids could also excel at academic knowledge given right environment
2. for kids whose environments emphasised practical knowledge, the mastery of academics is still developing

he says tests measure the now state and not a person’s potential

Sternberg study in Brazil – orphans starting businesses to survive – practical maths skills

Sternberg study of horse racing handicapping – IQ unrelated to handicapping ability – complex mental model with 7 variables

IQ tests can miss critical competencies

Dynamic Testing

Robert Sternberg and Elena Grigorenko proposed idea of using testing to assess ability dynamically

S’s concept of developing expertise is that with continued experience in a subject area we always move from lower to higher state of competence – and that standardised tests cannot accurately rate potential as limited to a static report

S/G proposed move from static tests to dynamic testing:-

1. determine state of one’s expertise
2. refocus learning on areas of low performance
3. follow-up testing to measure improvement
4. refocus learning to keep raising expertise

= weakness now no longer a fixed inability but a lack of skill / knowledge to be remedied

advantages of dynamic over standard testing:-

1. focuses learner & teacher on areas that need to be brought up rather than on accomplishments
2. measuring learner’s progress from 1 test to next provides truer gauge of their learning potential

focus for learner is what do I need to learn to improve

setbacks show us where we need to do better

choice re avoid these areas in future or redouble our efforts to master them thereby broadening our capacities & expertise

steps in dynamic testing:-

1. test of some kind – experience or paper – shows where I fall short in knowledge or skill
2. I dedicate myself to becoming more competent via reflection, practice, spacing etc (other effective learning techniques)
3. test myself again – what works better now, what do I still need to develop

most of us can learn to perform nearer our full potential in most areas by discovering our weaknesses and working to bring them up

Structure Building

there are differences in how we learn but not due to learning styles!

structure building:-

1. encounter new material
2. extract salient ideas
3. construct a mental model

high structure-builders learn new material better than low structure-builders

latter struggle to set aside irrelevant or competing info then tend to hang on to too many concepts to be condensed into a workable model or overall structure that can serve as foundation for further learning

cf building Lego town of bricks starting with streets then buildings etc

high structure-builders develop skill to identify foundational concepts & their key building blocks & to sort new info based on whether it adds to larger structure & one’s knowledge or superfluous

structure building is a form of conscious/subconscious discipline: does this fit or not: add nuance, capacity & meaning or obscures & overfreights

understanding cognitive differences in structure building is in early stages

is low due to faulty cognitive mechanism or is this a skill that some pick up naturally & others must be taught?

questions in text on key points help low structure builders identify the key points and their learning performance becomes similar to low

via neurosurgeon Mike Ebersold & pediatric neurologist Doug Larsen – cultivating the habit of reflecting on one’s experiences, of making them into a story, strengthens learning

cf structure building, reflect on what went right, wrong, how might I do it differently next time, helps me isolate key ideas, organise them into mental models & apply them again in the future with eye to improving / building on what I have learned

Rule versus Example Learning

this difference appears to matter …

whether you are a rule learner (abstract things to get to rules) or example learner (memorise examples)

example learners may improve when asked to compare 2 examples not just look at 1 example and identify similarities

example of taking a castle where only small numbers can cross certain bridges & radiation where have to go through healthy tissue to attack the bad tissue – students solving this then able to solve variety of different convergence problems

high structure-builders & rule learners more successful in transferring learning to unfamiliar situations than low structure & example learners

cf Bruce Hendry – model included supply/demand at start but since added to greatly

knowledge is not knowhow till understand the underlying principles at work and can fit them together into structure larger than sum of parts

knowing is learning that helps you go do

The Takeaway

1. be the one in charge
1. find out what you are after
2. mastery is a quest
2. embrace the notion of successful intelligence
1. tap into all of your intelligences to master the knowledge / skill you want to possess
1. describe what you want to know, do, accomplish
2. list the competencies required
3. what you need to learn
4. where you can find the knowledge or skill
5. go get it
2. consider your expertise to be in continuing development
3. practice dynamic testing as a learning strategy to discover your weaknesses
4. focus on improving yourself in these areas
5. smart to build on strengths but will become ever more competent and versatile if you also use testing & trial and error to improve in areas where your knowledge or performance are not pulling their weight
6. adopt active learning strategies like retrieval practice, spacing & interleaving
1. be aggressive – develop workrounds or compensating skills for impediments/holes in your aptitudes
2. do not rely on what feels best – be accurate in your understanding of what you know
3. ensure your strategies are moving you towards your goals
4. do not assume you are doing something wrong if it feels hard
7. distil the underlying principles: build the structure
1. different / similar situations – different / similar solutions
2. break idea / competency into component parts
3. ask periodically what are the rules, central ideas, supporting concepts/nuances
4. what scaffold / framework holds these ideas together
5. lots of examples of structures all around us in our world

by abstracting underlying rules & piecing them into a structure, you go for more than knowledge, you go for knowhow … that kind of mastery will put you ahead

====================

Chapter 7: Increase Your Abilities

My research and application notes from the book

Great to see “focus” and “discipline” used in this chapter. These were my One Words for 2017 and 2018.

Also good to see habit formation and stimulus mentioned and covered. James Clear’s book “Atomic Habits” is on my reading list for later this year.

I am familiar with “growth mindset” from a number of the MOOCs I have done so it was good to see that subject covered in this context too.

I remain challenged by memorising things. The mnemonic devices clearly help people remember things but they have always seemed too complicated for me to use. I intend trying harder and giving these a go when this seems appropriate. All 3 of my kids are coming to the end of various courses of studies with 2 revising for important exams. Flashcards seem to be much in evidence. I am reminded again of how hard I found it remembering facts and figures for exams 40 years ago.

Book Club Questions

Q: Were you familiar with growth mindsets, deliberate practice or memory cues before you read Make It Stick?
A: Only growth mindset.

Q: Of these three strategies, which one are you most likely to share with your students and why?
A: If I had students, probably deliberate practice as it encourages deliberate effort so fits with the effortful learning mantra.

Q: How does habit formation happen?
A: Repeatedly doing something until we get to the point where something is automatic and in some cases is hard to slow that down and break down the steps.

Q: What boosts IQ? And for whom?
A: Genetics, environment, being curious, stimulation of learning when under 5, nutrition, parents who help kids speak and identify objects, reading with the kids. Some socio-economic factors at work re education status of parents.

Q: Do brain training games work? Why or why not?
A: Improves fluid intelligence. I would say not work overall as too many issues to address and take into account from the one research study done. Nothing about lasting impact. Small sample size. Single task. Results not been replicated.

Q: What are the two components of IQ? What evidence do we have or do we not have about whether either or both can be improved?
A: Fluid (ability to reason, see relationships, think abstractly, hold info in mind while working on a problem)
and crystallised (a person’s accumulated knowledge of the world, the procedures or mental models one has developed from past learning & experience).

Q: Define growth mindset. What are the 2 Carol Dweck praise stories?
A: If you think you can, you can, if you think you can’t, you can’t. Attitude is key. Key is how to respond to failure.
Students do a test and then praised for being smart or for their effort. Former pick easier, latter pick harder. Tested again with tests from 2 people (1 gave unsolvable tests) those who were praised for being smart failed to complete the test a 2nd time (all solvable) when they had got it right first time. Instilled sense of defeat and helplessness.

Q: What is Paul Tough’s thesis?
A: Our success is less dependent on IQ than on grit, curiosity and persistence.

Q: What are the key ingredients of deliberate practice?
A: Goal-directed, often solitary, repeated striving to reach beyond current level of performance.

Q: How do you use mnemonics properly? What are the benefits?
A: Use to recall when you have mastered the subject. Recall can become automatic. Complex material can become second nature. Helps organise and cue the material for retrieval.

My notes from the book

marshmallow delayed gratification test (Walter Mischel) – some kids waited others ate straight away others agonised then ate – 1/3 from 600+ waited

follow-up studies – delaying led to those kids being more successful in school/careers

our success based on our genes but also our focus & self-discipline which are related to motivation & one’s sense of personal empowerment

mnemonic devices: mental tools used to help hold large volume of new material in memory cued for ready recall

James Paterson 1st heard of these from uni teacher – thought if he used these would learn things quicker, more time to play – 2006 took 1st place in comp – hooked

memory athletes start in various ways

brain remarkably plastic even into old age for most people

infant brain like infant nation – little in way of capability – would need cities, universities, farms, sea ports, roads, trains, comms lines

we are born with natural material but become capable via learning & development of mental models & neural pathways enabling us to reason, solve, create

you are not restricted by your birth brain & genes

Neuroplasticity

all knowledge & memory are physiological phenomena held in our neurons & neural pathways

brain reorganises itself with each new task

John T Bruer – reviewed neuroscience:-

1. we are born with 100 billion nerve cells (neurons)
2. synapse: connection between neurons that pass signals
3. pre/post birth “an exuberant burst of synapse formation” in which brain wires itself
4. neurons sprout microscopic branches – axons – that reach out for tiny nubs on other neurons – dendrites
5. when axon meets dendrite, a synapse is formed
6. vast distances travelled by axons
7. this circuitry enables our senses, cognition, motor skills, incl learning & memory, and forms the possibilities / limits of one’s intellectual capacity

synapses:-

1. peak at age of 1 – 2
2. 50% higher than average as adults
3. plateau to puberty
4. overabundance declines – process of synaptic pruning
5. 16 yo: 150 trillion connections

we do not know why these numbers or why pruning

may be use or lose it or determined by genes

Patricia Goldman-Rakic: most learning acquired post synaptic formation stabilising

Harry T Chugani: experience & environmental stimulation fine tune one’s circuits & make one’s neuronal architecture unique

2011 article: architecture & gross structure of brain appear to be substantially determined by genes but fine structure of neural networks appear to be shaped by experience & be capable of substantial modification

Paul Bach-y-Rita: pioneered device to help patients who suffered damage to sensory organs – regaining lost skills by teaching brain to respond to stimulation of other parts of their bodies – sub-ing one sensory system for another cf blind people using braille or stick to walk round

with practice the substituted sense gets better as if brain was rewiring itself

grey matter: neural cell bodies that make up the most part of our brains

white matter: the wiring

both are being studied intensely today – helped by advances in brain imaging tech

connectome: architecture of human neurocircuitry

part of maturation of connections is gradual thickening of myelin coating of the axons – starts at the back of the brain and moves forward reaching frontal lobes as we grow into adulthood

frontal lobes: perform executive functions of the brain where the processes of high-level reasoning & judgement, skills developed through experience

thickness of myelin coating correlates with ability – research strongly suggests that increased practice builds greater myelin along related pathways, improving strength & speed of electrical signals and as a result performance e.g. piano practice

study of habit formation provides interesting view into neuroplasticity:-

1. neural circuits we use when we take conscious action toward a goal are not the same as ones we use when something is automatic, the result of habit
2. habits directed from region deeper in brain – basal ganglia
3. when doing extended training & repetition of some kinds of learning (e.g. motor skills, sequential tasks) our learning thought to be recoded there
4. same area as eye movements
5. chunked together to be performed as one unit without multiple sequential decisions
6. become reflexive and quicker – automatic responses to a stimuli
7. cf computer macros
8. theories about chunking are integral to process of habit formation cf typing

another fundamental sign of the brain’s enduring mutability is discovery that hippocampus, where we consolidate learning & memory, is able to generate new neurons throughout life = neurogenesis

thought to play central role in brain’s ability to recover from physical injury and in humans’ lifelong ability to learn

the activity of associative learning & remembering relationship between unrelated items (e.g. names & faces) stimulates increase in creation of new neurons in hippocampus

rise in neurogenesis starts before new learning activity undertaken, suggesting the brain;s intention to learn, & continues for a period after the learning activity

suggests that neurogenesis plays role in consolidation of memory & beneficial effects of spaced & effortful retrieval practice have on long-term retention

learning & memory are neural processes

content of book to date = evidence of neuroplasticity and consistent with view that memory consolidation as agent for increasing neural pathways by which one is later able to retrieve & apply learning

Ann/Richard Barnet: human intellectual development is a lifelong dialogue between inherited tendencies & our life history

Is IQ Mutable?

IQ is product of genes & environment

compare it to our height history personally and across all people, we are getting taller

Flynn effect: IQ rising since 1st measured in 1932

US risen 18 points in last 60 years

rationale – schools, culture (e.g TV), nutrition have changed substantially since then in ways that have affected IQ

Richard Nesbit “Intelligence and How To Get It”: pervasiveness of stimuli in modern society that did not exist years ago (e.g. a puzzle maze in McD Happy Meal)

options for learning have expanded exponentially

may be small genetic difference that makes one person more curious than another but the effect is multiplied in environment where curiosity is easily piqued & readily satisfied

cf socio-economic status – increased stimulation & nurturing more generally available in families with more resources & education

ability to raise IQ is fraught with controversy and subject of several studies

e.g. findings:-

1. nutrition affects IQ e.g. 3.5 to 6.5 via eg supplements
2. poor children into education early – 4 – 7
3. gains when ed tools, books, puzzles, parents trained to help kids learn to speak, identify objects etc – gains
4. reading with the kids to age 4 – beyond that no gains but increased language ability
5. pre-school - 4+ points
6. pre-school with language training – 7+ points

Brain Training?

products came from 2008 Swiss study – participants tackling problems requiring increasingly difficult working memory challenges holding 2 different stimuli in mind for progressively longer periods of distraction – 1 was sequence of numerals, 2nd was small square of light in various places on screen

crystallised intelligence can be increased by effective learning & memory strategies

key determinant of fluid intel is capacity of a person’s working memory – number of new ideas/relationships a person can hold in mind while working through a problem, esp with some distraction

both numerals/location of light square changed every 3 seconds and decided while viewing these 2 things wherther matched combo from n times back and n increased during test

all participants tested on fluid intel at start then increasingly difficult tests of working memory over periods up to 19 days then retested for fluid intel – all better than when training started – longest training got greatest improvements

showed for 1st time fluid intel can be improved through training

criticism:-

1. only 35 took part
2. similar high intelligent population
3. only one training task
4. so are results valid generally or just for this task
5. durability of improved performance unknown
6. results not replicated by other studies

was replicated 2013 but failed to find improvement to fluid intel – participants believed their mental capacities had been enhanced but researchers said illusory – but authors said increased sense of self-efficacy can lead to greater persistence in solving difficult problems, encouraged by belief that training has improved one’s abilities

brain is not a muscle so strengthening one skill does not automatically strengthen others

cf learning & memory strategies such as RP & building mental models are effective for enhancing intellectual abilities in material or skill practiced but benefits do not extend to other material or skills

the ability to make practice a habit is generalisable

so brain training may get benefits from better habits (e.g. learning how to focus attention & persist at practice)

environmental multipliers that result in disproportionate effect from small genetic predisposition eg a more curious person becomes a lot smarter in environment feeding curiosity

if I cannot raise my IQ, what behaviours can serve as cognitive multipliers:-

1. embracing a growth mindset
2. practicing like an expert
3. constructing memory cues

Growth Mindset

if you think you can, you can, if you think you cannot, you cannot

attitude counts for a lot

Carol Dweck hugely influential – your level of intellectual ability is not fixed but rests to large degree in your own hands

study of 2 groups – 1 with presentation on memory, 2nd with explanation of how brain changes as a result of effortful learning – make connections in brain – make you smarter

2nd group became more aggressive learners & higher achievers than 1st group who had fixed mindset

fundamental different Carol found in how a person responds to failure – their own inability or insufficient effort or ineffective strategy – dig deeper & try different approaches

she found some students aim at performance goals (validate your ability) while others strive to learning goals (acquire new knowledge / skills) – former – choose goals you know you can meet – latter – ever-increasing challenges, setbacks helpful in learning

learning goals trigger entirely different chains of thought & action from performance goals

praise and power it has in shaping way in which people respond to challenges

study: kids pick harder exercise after being praised for their effort vs those praised for getting right answer

part of former study – one person gives students exercises that can be solved, one person gives one that cannot be solved, some kids praised for being smart and some for effort – 2nd set of exercises from both, this time all solvable

of students praised for being smart, few solved puzzles from 1 even tho same puzzles as had from the other person earlier – for those who saw being considered smart as paramount, failure to solve Bill’s puzzles in 1st round instilled sense of defeat & helplessness

when you praise for intelligence, kids get message that being seen as smart is name of game

emphasising effort gives child a rare variable they can control but emphasising natural intel takes it out of child’s control & provides no good recipe for responding to failure

Paul Tough “How Children Succeed” builds on Dweck: our success less dependent on IQ than on grit, curiosity & persistence

essential ingredient is encountering adversity in childhood and overcoming it

can happen too for those who never face adversity & challenges – denied the character-building experiences essential for success later in life

Deliberate Practice

expert performance does not usually come from some genetic disposition or IQ advantage

it arises from 000s of hours of deliberate practice

deliberate practice: goal-directed, often solitary, repeated striving to reach beyond current level of performance

slow acquisition of larger number of increasingly complex patterns and what actions to take in a vast array of different situations

usually not enjoyable

for most learners needs coach / trainer who can help identify performance improvement areas, help focus attention on specific aspects, provide feedback to keep perception & judgement accurate

effort & persistence of DP remodel the brain & physiology to accommodate higher performance

field expertise is specific to that field – no advantage etc in another field

a reason that some experts viewed as having uncanny talent is they can reconstruct every aspect of their performance in granular detail cf Mozart – not the result of 6th sense but perception & memory within that domain via years of acquired skill & knowledge in that domain

average of 10,000 hours or 10 years of practice was key finding

expert performance is product of quantity & quality of practice, not genetics,

becoming expert is not beyond reach of normally-gifted people who have the motivation, time, discipline to pursue it

Memory Cues

mnemonic devices: mental tools to help hold material in memory, cued for ready recall

ROY G BIV – colours of rainbow

I Value Xylophones Like Cows Dig Milk – Roman numerals from 1 to 1000

memory palace: more complex of these devices – useful for organising & holding larger volumes of material in memory

based on method of loci – associate mental images with series of physical locations to help cue memories

e.g. use features of your home to associate countless number of visual cues for retrieving memories later as you take imaginary walk through home

if sequence important, use sequence on your walk

can also use other familiar routes

surprisingly effective

derives from way imagery contributes vividness & connective links to memory

humans remember images more easily than words

images cue memories

Mark Twain images for talk, images for Kings/Queens of England

rhyme schemes can also serve as mnemonic tools

peg method: remembering lists, 1-20 paired with rhyming, real image:
1 bun, 2 shoe …
penny-1, setting sun – penny-2, airplane glue
peg on to these items you want to remember

rhyming images stay the same but the associations change each time you need a new list

cf use of songs you know well

used historically for sending messages long distances in person without forgetting

versatility of mnemonic devices is almost endless – deeply familiar common structure easily linked to things to remember

Oxford students – psychology – 5 major topics x 7 essay qs x 12 paras para to demo mastery of subject

many students freeze

explanation of how the student uses the memory palace e.g. major buildings are each of the 5 and so on inside

students’ teacher is James Paterson from start of chapter – they use memory palaces, peg method & images in fave songs & films

note that all the material has already been covered formally in class before the memory devices are designed

one of students continues to use this at university

memory palaces serve not as learning tool but a method to organise what’s already been learned so as to be readily retrievable at essay time

when used properly can help organise large bodies of knowledge to permit their ready retrieval

huge stress buster & time saver

stories of Paterson’s memory competition exploits, memorising card decks etc, learning peg images for 1-1000 ( a year to master)

without mnemonics max number of digits you can recall is 7 cf phone numbers

Paterson 1st drawn to this as shortcut for his studies

started and was OK with mountain tops but not the detail below that

value of mnemonics to raise intellectual abilities comes after mastery of new material

examples of other users – Matt Brown (jet pilot), Karen Kim (virtuso violinist), Mike Ebersold (neurosurgeon)

seeing pattern of physical movements as a kind of choreography, seeing the map of it all, are mnemonic cues to memory & performance

with continued retrieval, complex material can become 2nd nature to a person & mnemonic cues are no longer needed – recall and application of them becomes as automatic as a habit

The Takeaway

effortful learning changes the brain, building new connections & capability

a great answer to “why bother?”

effort itself extends boundaries of our abilities

what we do shapes who we become & what we are capable of doing

the more we do the more we can do

= growth mindset

path to mastery needs self-discipline, grit & persistence

conscious mnemonic devices can help to organise & cue the learning for ready retrieval until sustained, deliberate practice & repeated use form the deeper encoding & sunconscious mastery that characterise expert performance

====================

Chapter 8: Make It Stick

My research and application notes from the book

A good and helpful final chapter of the book summarising all that has gone before with further examples of the advocated learning techniques.

Helpful to have sections aimed at students, educators and trainers.

Good to see the airline pilot Sully, who landed an aircraft on the Hudson river in New York in which no one died.

Visualisation got a mention which is something I know I need to do and experience its benefits as a result.

Bloom’s Taxonomy was not new to me. I encountered it when doing the Making MOOCs on a Budget MOOC. A helpful tool! Some resources:-

• Bloom's Taxonomy of Learning Domains
• How to write specific, measurable, and observable learning objectives using Bloom's Taxonomy

I went on a rabbit trail discovering more about Sylvanus Thayer via the Thayer Method at West Point outlined in this chapter:-

• Sylvanus Thayer Wikipedia entry
• Big Picture: Sylvanus Thayer of West Point (video)
• Great Leaders Are Made: An evolutionary perspective on the Thayer method of teaching used at West Point.
• An Alternate Approach in the Application of the Thayer Concept of Teaching

While looking for further info on the Thayer Method I stumbled over Katie Gimbar Covington’s Why I Flipped My Classroom and lots of her teaching videos on YouTube. Relevant to the material in Make It Stick.

Book Club Questions

Q: What is one thing you would tell someone about Make It Stick who has not read the book yet?

A: It taught me how to learn most effectively in ways that I have never before been taught. This is a good book to get some input on how to learn. It revealed to me that in nearly all ways how I have learned in the past has not been ideal, efficient or effective.

Q: What is one thing (a research topic, strategy etc) you would like to learn more about?

A: Consolidating new information by connecting it to existing information that I know about.

Q: What are three major strategies in Make It Stick? What does our intuition tell us? What do they feel like?

A: Retrieving, Spacing Our Retrieval, Interleave Study of Different Problem Types

Q: What are other effective learning strategies? Describe them and how / why they work to boost learning.

A: Elaboration – relating new info to what you already know – strengthens your understanding of that info & more connections / cues to remember later

Generation – seek to solve a problem before you get input on the solution – prepares you to learn

Reflection – review what you have learned – a combo of retrieval practice & elaboration – adds layers to learning & strengthens skills

Calibration – accurate objective assessment of what you know and do not know – target learning on weaker areas

Q: What is “awful blurting” as described by the journalist McPheet? What role does it play in learning / producing?

A: Doing a 1st draft of something. Leave it and come back later to edit, add to etc. Gives you a learning starting point to then refine. You think about ways to refine it now that you have a starting point. You do not have that if you do not commit anything written down.

Q: What do experts have after thousands of hours of practice?

A: Mastery of their subject area & discipline. Expertise.

Q: In what ways can teachers apply these principles in the classroom?

A: Apply appropriately to your situation. Explain to students how learning works. Teach students how to study. Create desirable difficulties in the classroom. Be transparent (about what you are doing to help them learn and why).

Q: How does the professor at UW increase student learning?

A: Daily low-stakes quizzes that count for grades. Students know the schedule & the rules. No exceptions.

Q: How does the professor at West Point increase student learning?

A: The Thayer method. Specific learning objectives for each course, student responsibility to achieve them, quizzes / recitation. Too much input to be used so need to prioritise learning. Start with questions. Read for answers. Little or no lecturing. Shoot an azimuth: keep your bearings in unfamiliar territory.

My notes from the book

it is mastering the ability to learn that will get you in the game and keep you there

move now to practical advice

Learning Tips for Students

the most successful students take charge of their own learning & follow a simple but disciplined strategy

you may not have been taught how to do this

you can do it

you will likely surprise yourself by the results

significant learning is often, usually, difficult

setbacks are signs of effort not failure

effortful learning changes your brain, making new connections, building mental models, increasing your capability

powerful implications – your intellectual abilities lie to a large degree within your own control – knowing this makes the difficulties worth working through

3 keystone study strategies

Practice Retrieving New Learning From Memory

means self-quizzing

should become your primary strategy NOT rereading

when reading, pause, ask:-

1. what are the key ideas?
2. what terms/ideas are new to me?
3. how would I define them?
4. how do these ideas relate to what I already know?

many books have questions at the end of chapters

generating your own Q&A is also a good way to study

plan time through course to quiz yourself on current and past work

check that what you know and do not know is accurate

identify areas of weak mastery & focus studying there to make them strong

the harder to recall new learning from memory, the greater the benefit of doing so – correct your incorrect answers

your intuition tells you to highlight & reread

retrieval practice is better than this

rereading gives illusion of knowing but is not reliable measure of mastery of material

quizzing focuses you on the key ideas and gives reliable measure of learning and helps connect to prior knowledge

ends cramming & all-nighters

little studying needed at exam time

it feels awkward & frustrating esp when hard to recall but each time you recall strengthens your memory & ability to recall it later

Space Out Your Retrieval Practice

study info more than once but leave considerable gaps between

learning names / faces – within few minutes of 1st encounter

new material in text – 1 day then several days or week +

when feeling sure of material, once a month

repeat use flashcards you think you know – shuffle till well mastered

you can discard cards but ensure review later

interleave study of 2 or more topics – helps refresh mind

intuition says focus on one subject – massed practice

hard to distrust as we see our performance improving and we do not see that gains come from short-term memory & quickly fade

leads to cramming

spaced practice is better – reconstructs learning, helps connect to more recent learning

powerful learning strategy

again is not intuitive as we sense that concentrated practice is more productive but spacing strengthens mastery & memory

Interleave the Study of Different Problem Types

study more than one thing at a time so there is alternation between subjects

scatter problems through your subject as you study to quiz yourself across more than 1 subject/problem type

constantly challenge your ability to recognise problem type & select the right solution

intuition tells you to stick to one subject to master it before moving on

mixing it up improves your ability to discriminate between types, identifying unifying characteristics within a type, improves success in later tests or real-world settings

blocked practice seems like you get better mastery as you go, interleaving feels disruptive & counterproductive

you feel this even when you get the benefits!

Other Effective Strategies

elaboration: improves mastery of new material & multiplies mental cues available to you for later recall & application

process of finding additional layers of meaning in new material

relating material to what you already know, explaining to someone else in your own words or how it relates to your life outside of class

powerful form of this is to discover a metaphor or visual image for the new material

the more you can elaborate on how new learning relates to what you already know, the stronger your grasp of the new learning will be & more connections you create to remember it later

generation: makes mind more receptive to new learning

attempt to answer/solve a question/problem before being shown the answer/solution

e.g. fill in missing words

many perceive their learning is most effective when it is experiential – learning by doing and not just reading a book or hearing a lecture

explain beforehand the subject of next chapter/lecture without knowing what it is and how it will relate to what you already know

this will help even if reality is different from your expectation

reflection: combo of retrieval practice & elaboration that adds layers to learning & strengthens skills

act of taking a few mins to review what you learned in recent class or experience & asking yourself questions re what went well, what could have gone better etc

calibration: accurately / objectively assess what you know and do not know, address areas where score low

mnemonic devices: help you retrieve what you have learned & to hold arbitrary info in memory

mental filing cabinets – handy ways to store info and find it again when needed

e.g. memory places

Tips for Lifelong Learners

strategies in previous section are effective for all ages but are centred around classroom instruction

lifelong learners use the same principles in a variety of less-structured settings

examples of named individuals using the techniques in the book in varied ways

1. retrieval practice
1. actor learning lines as understudy – various forms of RP incl working backwards and forwards through script, use of recorder for all parts lines incl other techniques from book
2. generation
1. writer with writer’s block, writes letters to mum re how he is feeling etc, then go back and delete “Dear Mother”, 1st draft, leave it for a period of time i.e write for a period of time and then your mind takes over even when not writing, but something needs to exist for that to happen, “knit” the problem, learning using your own words, learning like writing is act of engagement, go wrestle the bear
3. reflection
1. … and mental rehearsal, cf Sully in his book re landing on Hudson River
4. elaboration
1. pianist – great example of the way elaboration strengthens learning & memory, learns new score physically, aurally, visually, intellectually (coaching herself through transitions), daily practice, examples of how some playing is improved subconsciously via doing all the other things

Tips for Teachers

authors concerned about being too prescriptive

find what is right in your classroom

1. explain to students how learning works
1. to help them better manage their own education
2. some kinds of difficulties during learning help make learning stronger & better remembered
3. when learning is easy, often superficial, soon forgotten
4. not all of our intellectual abilities are hard-wired (fixed), when learning effortful, it changes the brain, making new connections, increasing intellectual ability
5. learn better when wrestling with new problems before solutions rather than vice versa
6. to achieve excellence in anything, strive to surpass your current level of ability
7. striving inevitably has setbacks and these often provide essential info to adjust to get to mastery
2. teach students how to study
1. students will benefit from teachers who help them understand these strategies and stick with them long enough to experience their benefits which may initially appear doubtful
3. create desirable difficulties in the classroom
1. where practical, use frequent quizzing to help consolidate learning and interrupt process of forgetting
2. be predictable re planned tests and low stakes
3. create study tools that incorporate retrieval practice, generation & elaboration
4. altho low stakes do make results consequential
5. test on subjects from earlier in the course
6. space, interleave & vary topics / problems covered in class
4. be transparent
1. about the difficulties
2. explain how & why you have designed the study tools

examples of people and their methods

Bloom’s taxonomy:-

1. gaining knowledge
2. developing comprehension of underlying facts & ideas
3. being able to apply learning to solve problems
4. being able to analyse ideas and relationships to make inferences
5. being able to synthesise knowledge and ideas in new ways
6. being able to use learning to evaluate opinions & ideas and make judgements based on evidence & objective criteria

learn more by testing than rereading – hard to overturn all the bad practice that we have all learned including your students

ask lots of questions throughout your delivery

cf paths through a forest get more marked by use

end of day – blank piece of paper, what did you learn in class today? then after 1st attempt go back through the reality

summarise the week’s learning on a sheet

question asked then 5-6 sentences to answer at end of week

questions asked for each level of Bloom’s Taxonomy

high structure to your classes helps student performance

every discipline has a culture

Thayer method from West Point: academic, military & physical – specific learning objectives for every course, student responsibility for meeting those, quizzes & recitation in every class meeting – too much work to do in time so zero in on essence – start with questions … read for answers – group work in groups answering qs on boards and one then explains output to whole group – cf way finding on compass .. are you gaining the mastery you need to get where you are trying to go (azimuth) – helps all students keep at it

self-responsibility, taking ownership for finding your own way to the objective

keep learning objectives in mind throughout any course of study

Tips for Trainers

In-service Training

Lots of such training does not use any of these principles – lucky to retain anything!

get a copy of presentation materials and quiz yourself on key ideas

schedule future emails to quiz you

encourage your association to apply the principles in the book

two example platforms – Osmosis, QStream

help people get to solutions themselves – generation after root causes, options, how each would work – role playing for difficulties

onboarding then training new starters in an organisation’s products/services/ ways of doing things – what knowledge and skills do I need in order to succeed

when prospecting … FORE – Family, Occupation, Recreation, Enjoyment – for general getting to know people but also helpful if this info is key info about a prospect for your products/ services

practice is a central learning strategy

certification & recertification

getting staff to identify problems to then fix is good learning practice re wrestling with problems to get solutions

practice of tell, show, do, review – a learning culture places responsibility for learning with the employees and empowers them to change the system

throughout the book we have talked about learning not education

responsibility for learning is with the individual

responsibility for education is with society’s institutions

techniques for highly effective learning in the book can be put to use now & not wait for education to change

no cost, no structural reform, benefits are real & long-lasting