Increasing Retention Without Increasing Study Time [pdf](files.eric.ed.gov)
files.eric.ed.gov
Increasing Retention Without Increasing Study Time [pdf]
https://files.eric.ed.gov/fulltext/ED505647.pdf
138 comments
I think a lot of topics are much less sequential the further you go. As an adult I spend most of my time repeating the fundamentals of my field, and learning a topic deeply as needed. For children it probably makes sense to cram the multiplication tables.
Basically all undergrad stuff is fundamentals and "optimized cramming" (we call it bulimic learning, because you stuff knowledge into you just in time to throw it back up in the test) means tons of people "achieve" degrees without being even dimly aware of the existence of the fundamentals afterwards. If you have a bachelor's degree in e.g. computer science, the idea of encoding data shouldn't be an unknown unknown to you.
I would think that the opportunities to immediately "use" multiplication, instead of just practice it for tests, or some future numerate citizenship, would be omnipresent.
If you don't use something after you learned it, you miss out on:
1. Learning how it is actually applied
2. Discovering how the knowledge is useful for you personally, in ways you may not expect if you don't actually experience using it
3. Deeper understanding and mastery of the knowledge
4. Much much much better retention
It is worth creating some immediate use for new knowledge, even the smallest possible useful or creative project, for better retention alone.
> learning a topic deeply as needed
That is the ultimate use-driven learning model.
As for non-sequential, I agree. The more we manage our own learning, the more it is a directed graph (i.e. prerequisites translate to many follow up paths), and eventually just graphs (many ways to order topics, and alternate combinations of prerequisites for each topic, in any complex area).
If you don't use something after you learned it, you miss out on:
1. Learning how it is actually applied
2. Discovering how the knowledge is useful for you personally, in ways you may not expect if you don't actually experience using it
3. Deeper understanding and mastery of the knowledge
4. Much much much better retention
It is worth creating some immediate use for new knowledge, even the smallest possible useful or creative project, for better retention alone.
> learning a topic deeply as needed
That is the ultimate use-driven learning model.
As for non-sequential, I agree. The more we manage our own learning, the more it is a directed graph (i.e. prerequisites translate to many follow up paths), and eventually just graphs (many ways to order topics, and alternate combinations of prerequisites for each topic, in any complex area).
To be clear, in the context of this article, the alternative to drilling multiplication until you've memorized it is to space out the learning over a period of time. It's probably good to study for the test, because next week we're learning PEMDAS, so you better be solid on your multiplication.
They don't teach multiplication tables anymore as far as I know. I didn't learn them and I was born in 1995
Wouldn't that be weird given that it's part of common core?
> By the end of Grade 3, know from memory all products of two one-digit numbers.
[1] https://www.thecorestandards.org/Math/Content/3/OA/#CCSS.Mat...
> By the end of Grade 3, know from memory all products of two one-digit numbers.
[1] https://www.thecorestandards.org/Math/Content/3/OA/#CCSS.Mat...
Knowing the products is not the same as using or cramming the tables.
It is quite sufficient (and probably superior) to learn the facts through use, for that makes them robust.
It is quite sufficient (and probably superior) to learn the facts through use, for that makes them robust.
I learned that through brute force. They just had us multiply numbers over and over again rather than memorizing the table
They still teach them in Germany, though I think maybe only up to 10x10 these days.
In the US AFAIK they are called "times tables"; in Germany "one times one" („Einmaleins‟). It's funny what people emphasize for the same thing.
In the US AFAIK they are called "times tables"; in Germany "one times one" („Einmaleins‟). It's funny what people emphasize for the same thing.
"these days"?
The goal is to memorize single digit products and use the multi-column algorithms when you need to do more digits. To which days are you referring where kids would be called upon to memorize multidigit multiplications?
In the 1980s I had a trapper keeper with a 12x12 table printed on it but even my seven year old ass knew back then that everything >10 was wasteful to memorize. :P
20 years ago my kid still had to learn up to 12x20. The 12s are pretty handy for a bunch of reasons, and especially if you live in the USA. Using base 10 for the metric system was a mistake.
That’s why I made Manabi Reader for language learning (currently only Japanese): https://news.ycombinator.com/item?id=41275227
It encourages finding content from native sources, learning new words in context from materials that interest you rather than textbook content, and helps you build your own personal corpus of sentences as you encounter them in the wild. Will soon expand to more media types than just web and epub, to include YouTube/comics/HDMI input/game emulators etc
If you find flashcards from others, you also get easy tools for discovering more sentences from source materials that might interest you or from your own corpus. More coming for this and other word/kanji-level tracking analytics. Offline-first and privacy friendly.
It encourages finding content from native sources, learning new words in context from materials that interest you rather than textbook content, and helps you build your own personal corpus of sentences as you encounter them in the wild. Will soon expand to more media types than just web and epub, to include YouTube/comics/HDMI input/game emulators etc
If you find flashcards from others, you also get easy tools for discovering more sentences from source materials that might interest you or from your own corpus. More coming for this and other word/kanji-level tracking analytics. Offline-first and privacy friendly.
This is great. Reminds me of the writings of Khatzumoto (All Japanese All The Time). On that note, do you know how to contact him? His site got deleted recently and I have an idea for restoring a static version.
Thanks! Yes I've read his stuff and basically all discourse online since ~2005 when I wrote my first web app version of Manabi. I was also heavily influenced by the Breaking into Japanese Literature book and similar ones which had translations of words and pages, sometimes additional explanations, for short stories from Akutagawa etc. (as I've included in the Rashomon EPUB in Manabi Reader)
I don't know how to contact Khaz sorry I've never talked w/ him. There's a subreddit dedicated to his approach r/ajatt so you might find leads there. I'd also check archive.org of course
If you try out my apps pls drop any feedback, easiest is Discord (chat or issue tracker) https://discord.gg/4aF9yuASzb Thanks - recently went full-time on this so looking to grow it quickly and have lots more on deck
I don't know how to contact Khaz sorry I've never talked w/ him. There's a subreddit dedicated to his approach r/ajatt so you might find leads there. I'd also check archive.org of course
If you try out my apps pls drop any feedback, easiest is Discord (chat or issue tracker) https://discord.gg/4aF9yuASzb Thanks - recently went full-time on this so looking to grow it quickly and have lots more on deck
[Mods: it might be helpful to tag this paper as written in 2007]
It seems to me this paper is bringing to light the idea of spaced repetition for learning and this has become quite popular in the productivity/learning culture of today (e.g. Ali Abdaal).
> Alternatively, mathematics textbooks could easily adopt a format that engenders spacing.
I tutor middle school students in mathematics and this is definitely being implemented in their textbooks! At the end of each chapter, there a is normal chapter review practice test followed by a "Cumulative Practice" which reviews topics from the previous chapters in the book. These are especially beneficial to my students as, like the paper highlights, it promotes long-term memory of those topics.
> For example, although computer-based instruction typically provides extensive retrieval practice and rapid feedback, it offers a currently unexploited opportunity to schedule study sessions in ways that optimize long-term retention.
There is an immensely popular software called Anki which implements exactly this "spaced repetition"-type protocol.
It seems to me this paper is bringing to light the idea of spaced repetition for learning and this has become quite popular in the productivity/learning culture of today (e.g. Ali Abdaal).
> Alternatively, mathematics textbooks could easily adopt a format that engenders spacing.
I tutor middle school students in mathematics and this is definitely being implemented in their textbooks! At the end of each chapter, there a is normal chapter review practice test followed by a "Cumulative Practice" which reviews topics from the previous chapters in the book. These are especially beneficial to my students as, like the paper highlights, it promotes long-term memory of those topics.
> For example, although computer-based instruction typically provides extensive retrieval practice and rapid feedback, it offers a currently unexploited opportunity to schedule study sessions in ways that optimize long-term retention.
There is an immensely popular software called Anki which implements exactly this "spaced repetition"-type protocol.
There is a comparable software that has a friendlier UX: https://mochi.cards/. It's basically Anki, if Anki were smoother. Does cost a tiny bit though.
Mochi is excellent and the dev is responsive.
well. Knowt (knowt.com) offers spaced approach too. I love it better
Anki is very much... an expert's tool? I think there's quite a lot of opportunity (not necessarily monetarily :)) for SRS software with a good UX.
Anki's data model is rather strange, which partly relates to its flexibility but has some unexpected downsides and trap-doors. I would love a slightly more opinionated SRS tool! (Maybe with a bit more UI polish, rather than rewriting the backend in Rust)
Anki's data model is rather strange, which partly relates to its flexibility but has some unexpected downsides and trap-doors. I would love a slightly more opinionated SRS tool! (Maybe with a bit more UI polish, rather than rewriting the backend in Rust)
I don't see how anki is an experts tool...unless you want it to be.
You can simply dump everything into a single collection, use basic 'front' and 'back' cards...and review. Yes, it may not be 'optimal' depending on how you mark cards, but for most people, it's good enough. Better than nothing, and probably an improvement as an addition to whatever they are currently doing.
If you want expert features, you can. You can drill, tag, organise, create different templates....but none of that is really necessary, or particulary pushed on you as something you should or have to use.
You can simply dump everything into a single collection, use basic 'front' and 'back' cards...and review. Yes, it may not be 'optimal' depending on how you mark cards, but for most people, it's good enough. Better than nothing, and probably an improvement as an addition to whatever they are currently doing.
If you want expert features, you can. You can drill, tag, organise, create different templates....but none of that is really necessary, or particulary pushed on you as something you should or have to use.
> SRS software with a good UX.
This is a common complaint, but feels like a "delicate flower" argument. The UX is pretty spartan, but a fancy whiz bang UI isn't necessary to remember facts.
Not everyone needs all that, and the people that do (which is fine, btw), is a reflection of the person more than the tool.
This is a common complaint, but feels like a "delicate flower" argument. The UX is pretty spartan, but a fancy whiz bang UI isn't necessary to remember facts.
Not everyone needs all that, and the people that do (which is fine, btw), is a reflection of the person more than the tool.
Anki is only an expert's tool if you fall into the all too common trap of trying to optimize everything by microtweaking the settings.
I mean, sure, if that's your thing and you enjoy it, go for it. But it is not necessary, and (SM2 vs FSRS arguments aside), it works right out of the box with the default settings and algorithm.
Like anything, trying to get that last 10% optimization is WAY more work than getting the first 90%. But "using Anki" (in any way) is such a multiplier over NOT using it that just taking that step is enormous.
I mean, sure, if that's your thing and you enjoy it, go for it. But it is not necessary, and (SM2 vs FSRS arguments aside), it works right out of the box with the default settings and algorithm.
Like anything, trying to get that last 10% optimization is WAY more work than getting the first 90%. But "using Anki" (in any way) is such a multiplier over NOT using it that just taking that step is enormous.
Quizlet has a spaced repetition mode you can use, you might need to pay for it? I used it a few years ago for some stuff I needed to memorize in school.
"Teach others what are you trying to teach yourself better" is the best long-term retention strategy in my personal experience.
I push the new guys to fix project documentation once they’ve figured out a tricky bit. It helps solidify their knowledge, and helps us double check that they understood, and it’s something they can contribute when they still haven’t become part of the bus number on anything yet.
That all sounds reasonable and smart, but the real reason I do it is the Curse of Knowledge. People in a system can’t see it from the outside. They make assumptions, use opaque or even misleading jargon, and employ circular logic. The new guy doesn’t know the lingo, or the circular logic. Their explanation will make more sense to the next hire than anything I can say. And having it written down this way can also give me new perspective on the system. Maybe it doesn’t have to work this way.
That all sounds reasonable and smart, but the real reason I do it is the Curse of Knowledge. People in a system can’t see it from the outside. They make assumptions, use opaque or even misleading jargon, and employ circular logic. The new guy doesn’t know the lingo, or the circular logic. Their explanation will make more sense to the next hire than anything I can say. And having it written down this way can also give me new perspective on the system. Maybe it doesn’t have to work this way.
Adding a new smart person to the team is one of those golden moments for a team, IMO. You get a tiny window of watching them struggle until the tribal team knowledge seeps into them by osmosis. During that time they don't yet know who to ask the questions to directly so they will post to team slack channels or the lead directly.
One must capitalize on this brief period because smart programmers are flexible and adaptable. Very quickly they will acclimatize themselves to the mess that surrounds them and they will become as blind to the deficiencies as the rest of the team.
One must capitalize on this brief period because smart programmers are flexible and adaptable. Very quickly they will acclimatize themselves to the mess that surrounds them and they will become as blind to the deficiencies as the rest of the team.
Adjacent to this: ignore criticism as feedback, but treat questions as feedback.
People who don’t “get” the code ask questions that contain feedback they often don’t even register as feedback. By the time they distill it to an actual criticism, they’re often so wrapped up in the problem they can’t be constructive, or they present an XY problem. But if three people ask you the same question about your code? You have a design issue. Fix it. Fix it now.
That puts me in a weird relationship with FAQs. FAQs aren’t informational, they’re confessional. Here’s all the times I fucked up and won’t admit it. Let me explain why I am right and you are wrong.
People who don’t “get” the code ask questions that contain feedback they often don’t even register as feedback. By the time they distill it to an actual criticism, they’re often so wrapped up in the problem they can’t be constructive, or they present an XY problem. But if three people ask you the same question about your code? You have a design issue. Fix it. Fix it now.
That puts me in a weird relationship with FAQs. FAQs aren’t informational, they’re confessional. Here’s all the times I fucked up and won’t admit it. Let me explain why I am right and you are wrong.
Funny enough, many organizations will actually punish questions due to optics, noise, or RTFM (which fair enough do read it, but also don't presume the asker didn't just because you understand what you wrote. Being written down doesn't mean it's understandable by a new reader, or that it's optimally setup for discovery)
Nice. Somewhere I picked up the phrase "problem of assumed knowledge" for the phenomenon you describe.
But your phrase is spot on, well defined, and the subject of ongoing research.
https://en.wikipedia.org/wiki/Curse_of_knowledge
Cheers.
But your phrase is spot on, well defined, and the subject of ongoing research.
https://en.wikipedia.org/wiki/Curse_of_knowledge
Cheers.
> "Teach others what are you trying to teach yourself better"
A joke in Russian universities:
- A teaching assistant tells a student: "Look, I've been explaining it to you for so long, that I myself understood it!"
A joke in Russian universities:
- A teaching assistant tells a student: "Look, I've been explaining it to you for so long, that I myself understood it!"
I tried to help some dude understand integration by parts, which I’m not sure I’d understood myself. I warned him. But it was late, he was desperate, and I was just going to go play computer games anyway.
I figured it out, but I’ve no idea if I got him sorted out for his final.
I figured it out, but I’ve no idea if I got him sorted out for his final.
A joke in American universities:
The professor is interrupted in the middle of his lecture by a student who asks about one of the previous week's problems:
— Could you please work an example for us?
— No problem!
The professor writes a problem statement, then scratches his head, tugs on his collar, rubs his chin, and at last scribbles a complicated integral on the board.
— There you go!
— Excuse me, professor, but could you please be a little more explicit?
— What do you want from me? I just did the problem 3 different ways.
The professor is interrupted in the middle of his lecture by a student who asks about one of the previous week's problems:
— Could you please work an example for us?
— No problem!
The professor writes a problem statement, then scratches his head, tugs on his collar, rubs his chin, and at last scribbles a complicated integral on the board.
— There you go!
— Excuse me, professor, but could you please be a little more explicit?
— What do you want from me? I just did the problem 3 different ways.
So the reason I have trouble in math is that I don't wear collared shirts, so I only get 2 ways to solve problems?
Only getting 2 ways to solve it brings us back to russian jokes:
A soviet maths prof finds out he'd make more money as a labourer in the shipyards, so after more than a few carefully-distributed bottles of vodka he manages to change profession. A month or two working in the yards, and he sees a notice on the bulletin board offering a bonus to workers who sign up for a special "maths for the proletariat" evening school course.
Figuring it's easy money, he signs up, brings a novel, and sits in the back of the class reading instead paying attention. Then the prof calls him up to the brownboard, and asks him for the circumference of a circle.
Suddenly, our ex-prof blanks. Frantically he scribbles and scribbles on the brownboard, only to wind up with ... -2πr. No, that can't be right, but what went wrong?
Just when he's about to crack trying to figure out his error, he hears a friendly voice from one of the front-row students:
— Psst, colleague, swap order of integration!
A soviet maths prof finds out he'd make more money as a labourer in the shipyards, so after more than a few carefully-distributed bottles of vodka he manages to change profession. A month or two working in the yards, and he sees a notice on the bulletin board offering a bonus to workers who sign up for a special "maths for the proletariat" evening school course.
Figuring it's easy money, he signs up, brings a novel, and sits in the back of the class reading instead paying attention. Then the prof calls him up to the brownboard, and asks him for the circumference of a circle.
Suddenly, our ex-prof blanks. Frantically he scribbles and scribbles on the brownboard, only to wind up with ... -2πr. No, that can't be right, but what went wrong?
Just when he's about to crack trying to figure out his error, he hears a friendly voice from one of the front-row students:
— Psst, colleague, swap order of integration!
Also possibly the most time consuming. Best and efficient with time are two different metrics.
Someday you will die. Then you will die again when people forget you. Only the effect you’ve had on the world, including teaching others, who pass that along, outlives that second death.
So what’s this about efficiency? You should be more worried about effectiveness, not efficiency. Particularly in this profession.
So what’s this about efficiency? You should be more worried about effectiveness, not efficiency. Particularly in this profession.
> Someday you will die. Then you will die again when people forget you.
I've already heard that line somewhere but can't remember where and who said it.
I've already heard that line somewhere but can't remember where and who said it.
It's one of those quotes that gets attributed to a lot of different people. Here is goodreads claiming it was Ernest Hemingway [1].
“Every man has two deaths, when he is buried in the ground and the last time someone says his name. In some ways men can be immortal.”
1. https://www.goodreads.com/quotes/9556005-every-man-has-two-d...
“Every man has two deaths, when he is buried in the ground and the last time someone says his name. In some ways men can be immortal.”
1. https://www.goodreads.com/quotes/9556005-every-man-has-two-d...
All I know is whoever said it wasn’t a teacher.
Whoever said it was obviously a teacher or else you do not understand what this line is trying to teach others.
Best thing I have read after a long time. Thank you!
What is consider efficient if it yields less best results?
I understand time constraints and end goals may prohibit the 'best' approach.
What I do not understand is how can you say something is more 'efficient' if the yield in understanding is less than what you would get with another method.
Hopefully this will clarify my thought process here:
If 'Best' is to teach others and requires 10,000 hours to yield 90-99% understanding.
In contrast, 'Efficient' method requires 2,500 hours to yield 30-40% understanding. However, there is diminishing returns meaning that doubling your hours to 5,000 does not return you with 60-80% understanding, rather maybe closer to 50-60% understanding. With 7,500 hours closer to 65-70% and 10,000 hours may around 75-89% understanding.
Here you've spent the same amount of time but did not achieve the same level of understanding. I think you may have a dynamic 'Best' vs 'Efficient' curve and to switch between those options to optimize maximizing your level of understanding in the least amount of time.
I understand time constraints and end goals may prohibit the 'best' approach.
What I do not understand is how can you say something is more 'efficient' if the yield in understanding is less than what you would get with another method.
Hopefully this will clarify my thought process here:
If 'Best' is to teach others and requires 10,000 hours to yield 90-99% understanding.
In contrast, 'Efficient' method requires 2,500 hours to yield 30-40% understanding. However, there is diminishing returns meaning that doubling your hours to 5,000 does not return you with 60-80% understanding, rather maybe closer to 50-60% understanding. With 7,500 hours closer to 65-70% and 10,000 hours may around 75-89% understanding.
Here you've spent the same amount of time but did not achieve the same level of understanding. I think you may have a dynamic 'Best' vs 'Efficient' curve and to switch between those options to optimize maximizing your level of understanding in the least amount of time.
The problem I have with this argument is that you cannot consider "efficiency" in a vacuum. You need to have a metric against which to measure it.
Consider these two scenarios -
Goal: remember where to look up information when it comes up in $JOB Metric: how much you remember, how quickly you find the info
Goal: discover new hyper-efficient method of training an AI (or insert popular ML topic here). Metric: percent improvement vs current pubished best practice (deliberately vague) Required understanding to make progress: "like a Ph.D. from Stanford"
Now you can possibly measure something.
The idea achievement of "90% understanding" is VERY topic dependent. Simple topic? Sure 100% understanding, I remembered the Latin names of all of the plants in my house. Complicated topic? The information for "100% understanding" might not even be written in the textbook - it probably includes things like seeing the interconnections between the topics and being able to apply them in slightly different contexts.
Make sure you read the studies so you know what they're talking about. In this area, I think summaries are frequently misleading. You have to know what the real evidence is that substantiates the claims. (I cannot tell you how many times I have looked at the evidence and just rolled my eyes - obviously not applicable in settings where I wanted it to be.)
Edit: See this comment (not me) - https://news.ycombinator.com/item?id=41275869
Consider these two scenarios -
Goal: remember where to look up information when it comes up in $JOB Metric: how much you remember, how quickly you find the info
Goal: discover new hyper-efficient method of training an AI (or insert popular ML topic here). Metric: percent improvement vs current pubished best practice (deliberately vague) Required understanding to make progress: "like a Ph.D. from Stanford"
Now you can possibly measure something.
The idea achievement of "90% understanding" is VERY topic dependent. Simple topic? Sure 100% understanding, I remembered the Latin names of all of the plants in my house. Complicated topic? The information for "100% understanding" might not even be written in the textbook - it probably includes things like seeing the interconnections between the topics and being able to apply them in slightly different contexts.
Make sure you read the studies so you know what they're talking about. In this area, I think summaries are frequently misleading. You have to know what the real evidence is that substantiates the claims. (I cannot tell you how many times I have looked at the evidence and just rolled my eyes - obviously not applicable in settings where I wanted it to be.)
Edit: See this comment (not me) - https://news.ycombinator.com/item?id=41275869
Tangent: efficiency nearly always comes at the expense of flexibility.
Efficiency and effectiveness are indeed different metrics. And I think balancing time and retention needs is key
Different metrics can be interdependent, efficiency and effectiveness are among them.
Yes, possibly the most time consuming if you are thinking of writing a book.
How effective something is comes with what is best and compromise effectiveness for efficiency.
How effective something is comes with what is best and compromise effectiveness for efficiency.
The expression goes "to know is to be able to explain".
Yes, but there are plenty of exceptions. I know brilliantly knowledgeable and extraordinary clever people that are so socially awkward that they can't explain virtually anything.
To be people who also understand it
For language acquisition?
For language acquisition: find a partner who speaks the other language and wishes to learn yours, then converse with each of you using your L2, falling back to L1 only when circumlocution fails or to make corrections.
(then how do you practise listening to your L2? Eavesdropping, ancillary encounters, news, movies, etc. give plenty of opportunity, but speaking requires you to make an extra effort.)
(then how do you practise listening to your L2? Eavesdropping, ancillary encounters, news, movies, etc. give plenty of opportunity, but speaking requires you to make an extra effort.)
or the reverse
Also try for other things that are difficult to grasp.
[deleted]
consf(1)
Sounds great but I was expecting more evidence. They talk about this shuffling method, which interleaves material as a way to provide spacing and reintroduce material. But it appears to just be their opinion that it will help improve retention. Also what is the deal with this "hypothetical interaction between ISI and RI"? Why not do enough experiments to actually plot it out? Anyone can graph out a hypothetical interaction.
Years ago there was a great extended comment here on HN about teaching a linear algebra course using, among other tactics, a similar spacing of the homework problems. He reported excellent results. I wish I remembered the username. Ben something?
He also later expanded it into a blog post: https://bentilly.blogspot.com/2009/09/teaching-linear-algebr...
I really appreciate HN. There aren't many public online forums were one could ask about a user comment made on a related post posted years ago and in less than 20 minutes get two different responses linking to the comment from 15 years earlier.
Perhaps it helps the topic was "increasing retention". :-)
Perhaps it helps the topic was "increasing retention". :-)
This comment reminds me of a recurring nightmare I have about a math class I forgot to drop, never attended, and have to pass for to graduate. I’ve no idea if I pass it or not, since I always wake myself up by then.
This guy wouldn’t have helped me though, since I never attended the class.
This guy wouldn’t have helped me though, since I never attended the class.
Yes, thanks!
I think it was Ben Tilly and I think his user name is btilly. I also remember that comment you're referring too.
That's the one, thank you.
I think Accelerated Expertise comes to the same conclusion and references the studies to back it up. It's a good book!
The "Make it Stick" book https://www.amazon.com/Make-Stick-Science-Successful-Learnin... cites many many studies in this arena if you're interested.
I'd highly encourage anyone interested in the contents of this article/learning about learning to read through the submitter's blog: https://www.justinmath.com/blog/
I also recommend https://www.learningscientists.org/posters from scientists in the field, which covers additional scientifically-effective approaches that a course designer or more long term approach might take. For example, not only applying Spaced Repetition, Interleaving and Active Recall (all possible through automated spaced repetition apps based on simply input), but elements like Dual coding (related to "Varied practise") - mixing visual and other elements when learning (which requires effort to create them), and elaboration practise (like free recall). To the end of applying these - and other critical elements like focusing on the motivation of the learner and enabling them to understand what they are missing with progress & identified "blindspot" misconceptions (on their own incredibly powerful), I've been developing Revision.ai since before GPT-3, through a Psychology MSc.
See also the wonderful https://betterexplained.com
Content is good but he is part of a paid math program and some articles and submissions appear to be indirect advertising/marketing if you think about it. (but it's effective and less forceful)
https://www.justinmath.com/why-is-the-edtech-industry-so-dam...
Yes. The submitter uses spaced repetition techniques to build persistent memory of his spaces reposition math education business.
It's only a four page article about a particular spaced study strategy. Nothing really groundbreaking nor a comprehensive collection of techniques
any other recommendation for articles regarding techniques that works for most people?
This is pretty much the current state of the art in learning research:
https://bjorklab.psych.ucla.edu/research/
This is good stuff but I'll say that it isn't as comprehensive as all that. These studies and findings are almost entirely focused on simple recall knowledge (isolated facts, vocabulary). That's an important part of learning certainly and it informs research on learning higher order concepts but it's not the full story.
Just to name one example, folks might look into research on conceptual change theory (eg Chi or Posner). This theory helps explain why a concept like electricity is so challenging to learn. The reason, in brief, being that naive conceptions make a category error and think of electricity as a thing rather than a process. And this theory then informs instructional practice. Specifically, teachers should be aware of difficult concepts and should design activities that force students to confront the contradictions between their naive models and more accurate/complete ones.
Mickie Chi also has fascinations research on active learning (ICAP) and related work on the effectiveness of peer learning.
Just to name one example, folks might look into research on conceptual change theory (eg Chi or Posner). This theory helps explain why a concept like electricity is so challenging to learn. The reason, in brief, being that naive conceptions make a category error and think of electricity as a thing rather than a process. And this theory then informs instructional practice. Specifically, teachers should be aware of difficult concepts and should design activities that force students to confront the contradictions between their naive models and more accurate/complete ones.
Mickie Chi also has fascinations research on active learning (ICAP) and related work on the effectiveness of peer learning.
Thanks for your example of the category error here. Are there other cases though? For example, sometimes I doubt my understanding of gradient descent because it is very hard to implement in code on data and show error reducing over time (writing from scratch). But in some examples of a few nodes I can calculate it perfectly. I can do the Maths manually.
What error might I be making? For the future, is there a list of types of errors?
What error might I be making? For the future, is there a list of types of errors?
From a more applied angle, a book like "10 steps to complex learning" might be helpful.
I come from a similar cog psych background as the Bjork Lab, so am a big fan of their research, but books like 10 steps come from instructional design, which is a bit more focused on the big picture (designing a whole course vs individual mechanisms).
I come from a similar cog psych background as the Bjork Lab, so am a big fan of their research, but books like 10 steps come from instructional design, which is a bit more focused on the big picture (designing a whole course vs individual mechanisms).
Thanks for the tip.
Scott Young has a lot on this, including going deep on the literature and explaining it to populist student audience:
https://www.scotthyoung.com/blog/the-best-articles-on-learni...
For instance, on memory:
https://www.scotthyoung.com/blog/2019/02/15/memory/
https://www.scotthyoung.com/blog/the-best-articles-on-learni...
For instance, on memory:
https://www.scotthyoung.com/blog/2019/02/15/memory/
The default algorithm in Anki, in case you don't know it
Or not, since they recently added FSRS as an alternative.
You're both right - SM2 is still the default in Anki.
It is super unlikely that an algorithm that was made based on feels would turn out to be the most effective. Especially since it requires some learning and errors to even learn how to use it.
Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT. Improving Students' Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Psychol Sci Public Interest. 2013 Jan;14(1):4-58. doi: 10.1177/1529100612453266. PMID: 26173288.
The comment I wrote below is weird. What I mean to say is google for articles on writing good prompts. The rest matters less.
-----
Ignore this:
Spaced repetition, if you can stick with it, works. The details around schedules etc. don't matter as much as doing it in a way that you can keep up.
Also writing good prompts is important. I have yet to write more extensively about this but here is a start: https://news.ycombinator.com/item?id=41126734
-----
Ignore this:
Spaced repetition, if you can stick with it, works. The details around schedules etc. don't matter as much as doing it in a way that you can keep up.
Also writing good prompts is important. I have yet to write more extensively about this but here is a start: https://news.ycombinator.com/item?id=41126734
FSRS v5 is state of the art
It's just typical learning "science" fluff
> Because people forget much of what they learn, students could benefit from learning strategies that provide long-lasting knowledge. Yet surprisingly little is known about how long-term retention is most efficiently achieved.
I've always thought that the real problem is information relevancy. People need practical uses for remembering something beyond synthetic bullshit exams. Efficient techniques are great, but nothing demotivates more than not having a reason to learn beyond being told that you have to.
Nobody has to reach for flashcards, extensive notes, or advanced techniques when trying to learn something they're actually interested in, it's retained almost immediately and effortlessly. There has to be some kind of subconscious gauge of information relevancy that physically controls the level of absorption, a sort of "learning rate" if you will.
I've always thought that the real problem is information relevancy. People need practical uses for remembering something beyond synthetic bullshit exams. Efficient techniques are great, but nothing demotivates more than not having a reason to learn beyond being told that you have to.
Nobody has to reach for flashcards, extensive notes, or advanced techniques when trying to learn something they're actually interested in, it's retained almost immediately and effortlessly. There has to be some kind of subconscious gauge of information relevancy that physically controls the level of absorption, a sort of "learning rate" if you will.
I agree motivation to learn and maintaining interest is key, to effect the learning rate you mention. The enjoyment itself, although a crucifying word to use in discussions when teaching students considered the "best of the best", still matters and predominates over the effectiveness factor for most of the semester for students, at least. Because of just being "told to learn" it.
Basically 70% of the semester most students are not studying 40 hours - they are doing 30 hours of real work, and perhaps only 15 hours effectively. And for good reason: Nothing is bridging or rewarding them in a way that interests/motivates them, for courses where the interest isn't natural.
A bridge to motivate them would be ideal. In 2021 I started using GPT-3 to generate motivational "reasons to learn a concept" cards for my flashcard app, - Revision.ai - which you can read about here in the 3rd item: https://www.revision.ai/articles/20ThingsRevisionAIDoesForBe... - the reason we disabled them was simple: we could never quite time the cards right to help the student when they needed it. When the app is closed, they aren't motivated - they don't see them. Mid study session? Showing such cards (or AI generated examples) interrupted the flow [https://www.instagram.com/p/CVVlIuVg31W/] We have also tried recommending relevant short/mid length youtube videos for visual/"breaks" from overwhelming learning. That did not boost student success either. I guess it doesn't address that you are still being told to do it, not naturally flowing into learning.
I welcome any technical or conceptual ideas you have to improve this and help increase interest amongst students. We have found that turning lecture slideshows into sets of exercises with clear visuals[https://www.instagram.com/p/C5ByftwiJ00/], breaking content up, and showing progress does motivate students to study more(and in a semi-related dissertation I wrote, possibly reduce Test Anxiety and tension). Please let me know of any ideas you have!
Basically 70% of the semester most students are not studying 40 hours - they are doing 30 hours of real work, and perhaps only 15 hours effectively. And for good reason: Nothing is bridging or rewarding them in a way that interests/motivates them, for courses where the interest isn't natural.
A bridge to motivate them would be ideal. In 2021 I started using GPT-3 to generate motivational "reasons to learn a concept" cards for my flashcard app, - Revision.ai - which you can read about here in the 3rd item: https://www.revision.ai/articles/20ThingsRevisionAIDoesForBe... - the reason we disabled them was simple: we could never quite time the cards right to help the student when they needed it. When the app is closed, they aren't motivated - they don't see them. Mid study session? Showing such cards (or AI generated examples) interrupted the flow [https://www.instagram.com/p/CVVlIuVg31W/] We have also tried recommending relevant short/mid length youtube videos for visual/"breaks" from overwhelming learning. That did not boost student success either. I guess it doesn't address that you are still being told to do it, not naturally flowing into learning.
I welcome any technical or conceptual ideas you have to improve this and help increase interest amongst students. We have found that turning lecture slideshows into sets of exercises with clear visuals[https://www.instagram.com/p/C5ByftwiJ00/], breaking content up, and showing progress does motivate students to study more(and in a semi-related dissertation I wrote, possibly reduce Test Anxiety and tension). Please let me know of any ideas you have!
I disagree. Motivation will keep you reading the material, but that alone will not make you remember it that much better.
When you're motivated about something you can practice, you may remember more by doing it (which also acts a spaced repetition). But good luck trying that with, say, astrophysics or macroeconomics.
When motivated, you're also more likely to pick up other books on the subject, which is another form of spaced repetition.
When you're motivated about something you can practice, you may remember more by doing it (which also acts a spaced repetition). But good luck trying that with, say, astrophysics or macroeconomics.
When motivated, you're also more likely to pick up other books on the subject, which is another form of spaced repetition.
I'd guess, that overlearning is so widespread not because it is beneficial for an individual, but as a way for a teacher to deal with a lot of students at once. To measure just the right amount of learning the teacher needs to work with each student individually, and give exercises for them based on a quality of their newly formed knowledge. But the system is mostly tuned for mass-education to amplify the impact of a teacher's work.
I’ll mention that I’ve gone full-time on an iOS/macOS tool for learning Japanese through reading called Manabi Reader: https://reader.manabi.io
It combines reading and flashcards, such that it tracks every word and kanji you read and learn in order to show you analytics on what you need to learn to be able to read something or achieve JLPT goals, and highlights unknown/learning words in texts. Next up for the flashcard part is to replace the SM2 algorithm with FSRS for the flashcards, as well as having flashcards get passively reviewed simply by reading content.
I also suspect people are missing out on speed of learning when reviewing flashcards one at a time for hours. Besides actively recalling reviewing flashcards passively while reading, I’ll experiment with other review techniques like seeing a page at a time of vocab / revealable answers. Our minds absorb at the periphery of our vision and scanning/inputting a bunch of information at once, too. I’m unconvinced flashcard UI is the final expression of forgetting curve research based learning apps
Also working on Reader features such as manga/pdf/youtube/game emulators, plus expanding to all languages.
It combines reading and flashcards, such that it tracks every word and kanji you read and learn in order to show you analytics on what you need to learn to be able to read something or achieve JLPT goals, and highlights unknown/learning words in texts. Next up for the flashcard part is to replace the SM2 algorithm with FSRS for the flashcards, as well as having flashcards get passively reviewed simply by reading content.
I also suspect people are missing out on speed of learning when reviewing flashcards one at a time for hours. Besides actively recalling reviewing flashcards passively while reading, I’ll experiment with other review techniques like seeing a page at a time of vocab / revealable answers. Our minds absorb at the periphery of our vision and scanning/inputting a bunch of information at once, too. I’m unconvinced flashcard UI is the final expression of forgetting curve research based learning apps
Also working on Reader features such as manga/pdf/youtube/game emulators, plus expanding to all languages.
If you are interested in similar research, take a look at https://supermemo.guru (somewhat hidden "about" page: https://supermemo.guru/wiki/SuperMemo_Guru:About). Author spent quite some time on the topic and its history, including writing applications.
A lot of what's on that page is just nonsense. The page has a very clear bias toward one method and writes all sorts of unscientific drivel as a result.
https://supermemo.guru/wiki/School_damages_your_brain
As someone who also publishes in neuroscience I can tell you this is total and utter trash.
https://supermemo.guru/wiki/School_damages_your_brain
As someone who also publishes in neuroscience I can tell you this is total and utter trash.
I'm not in a position to judge this author's work as a whole, as this is not my area of expertise.
I know this person as an author of "super memo" algorithms for spaced repetition learning, so yes, he most likely have a bias, probably even financial stake to some degree, although not sure, I don't follow him in any way, so don't know if this ever was or still is the case.
In the past, I've used some of his algorithms in particular, with quite interesting results - subjectively speaking, not in any scientific regime, And, obviously, those are just one of many. I also remembered this was his area of interest in general, so assumed his wiki to be of some interest in the context of discussed article.
Thanks for your critical opinion, it will contribute to my priors about his writing.
I know this person as an author of "super memo" algorithms for spaced repetition learning, so yes, he most likely have a bias, probably even financial stake to some degree, although not sure, I don't follow him in any way, so don't know if this ever was or still is the case.
In the past, I've used some of his algorithms in particular, with quite interesting results - subjectively speaking, not in any scientific regime, And, obviously, those are just one of many. I also remembered this was his area of interest in general, so assumed his wiki to be of some interest in the context of discussed article.
Thanks for your critical opinion, it will contribute to my priors about his writing.
I enjoy browsing r/Anki from time to time, and that site is frequently mentioned there. Spaced repetition is definitely a real thing, but the rest of the SuperMemo site is a bit “crazy”. The author does have a “20 rules” for good card writing which is pretty good advice though. http://super-memory.com/articles/20rules.htm
[deleted]
From the abstract (emphasis added):
> Because people forget much of what they learn, students could benefit from learning strategies that provide long-lasting knowledge.
What learning strategies were taught in your secondary educational years?
> Because people forget much of what they learn, students could benefit from learning strategies that provide long-lasting knowledge.
What learning strategies were taught in your secondary educational years?
One thing I’ve found when I spend most of a day trying to learn things, is that having a ~1.5 hour nap is pretty important. Not sure what goes on, but it feels like it turns the disconnected “fog” of information in my head into something that is a least loosely linked and can be vaguely recalled upon rather than completely forgotten.
Unfortunately I kinda need to plan these sorts of days ahead, because trying to nap past 2PM is a good way to fuck my nighttime sleep.
Unfortunately I kinda need to plan these sorts of days ahead, because trying to nap past 2PM is a good way to fuck my nighttime sleep.
I'm pretty sure the current thought is that you ONLY "learn" during sleep, if you consider learning to be filtering, organizing, and throwing out vs keeping.
The time between study and testing (retention interval) is the most salient factor for students. Repeating material at intervals attenuates the forgetting curve somewhat, but not enough to generate passing scores. Cumulative exams only force students to study all material in a massed study session prior to a final exam.
I say this as someone who works on education and as a lifelong learner who has tried many ways of improving retention.
I say this as someone who works on education and as a lifelong learner who has tried many ways of improving retention.
[2007]
The simplest way of increasing retention is being interested in the subject.
You can just make yourself interested in whatever you want? If not, then it probably isn't the simplest way.
I would counter, why are you spending time learning something you're not interested in?
I couldn't care less about aspects of my company's business, but I damn sure need to know them in order to get paid.
Real life usefulness, money, time investment for future use, learning prerequisites for the things you are interested in, etc. There are so many reasons...
Simple, but not sufficient since you still need to pair your interest with some methods of study and practice. Which methods of studying are more effective or efficient is the question they're researching.
The topic isn't about simplicity, but performance and efficiency.
Real question. Does this count as a published paper?
Yes, with 263 citations
Rohrer, D., & Pashler, H. (2007). Increasing Retention Without Increasing Study Time. Current Directions in Psychological Science, 16(4), 183-186. https://doi.org/10.1111/j.1467-8721.2007.00500.x
Rohrer, D., & Pashler, H. (2007). Increasing Retention Without Increasing Study Time. Current Directions in Psychological Science, 16(4), 183-186. https://doi.org/10.1111/j.1467-8721.2007.00500.x
It was published in a scientific journal, so yes.
Why do researchers need to use acronyms for a 2 pager article?
spaced repetition
High IQ. A component of IQ is memory--working and long-term memory. There is a reason why doctors are smarter than average--you got to be smart to memorize all that stuff. I am skeptical that hacks make much of a difference. There is also a difference between recall and understanding. This is why speed reading courses are dubious because you're not really understanding things, but just recalling items in the text.
Optimizing "study, retention phase, test" for greatest knowledge retention at a delayed test time, is very different from optimizing for greatest value of knowledge learned.
To optimize learning value, learn things that are immediately useful, you can immediately incorporate into learning something else, and ideally both.
The sooner and more you use something, the greater its value AND the greater your retention will be.
If you have to learn something valuable but with no short term use (how to handle a rare brain surgery complication), find a way to use it. Create an ongoing useful project that will revisit that knowledge during the "retention interval" (e.g. a concise summary of rare situations you need to handle, for you and others, that you can revisit and improve with additional and updated knowledge).
So optimize "topic choice", "topic progression", and "study, (optionally) test, use, use, use", for total value of learning.
"Use" is motivation, test, study review, and value realization put together.