Tuesday, January 30, 2018

Predictive Learning

I’m gonna start this post by bragging a bit. One of my greatest interests through my entire life has been learning. Not just “I want to be a lifelong learner,” but understanding the act and process of learning, getting better at it, refining it, etc. How do you jump into a new skill, possibly in some arena that you’ve never tried before, and acquire competence?

This is one reason that I’ve struggled to acquire real mastery over anything. “Competence” comes from knowing the principles of a skill and having some experience applying them. It allows you to reliably attain good outcomes. “Mastery,” on the other hand, comes from feeling the principles of a skill so deeply that you apply them automatically. Not only do you achieve good outcomes, but you see good outcomes where other people don’t, and you achieve them seemingly effortlessly.

Mastery demands time. No matter how quickly you acquire your competence, you will eventually hit a wall where the conscious application of your knowledge is not enough. It needs to be automatic, to let your mind be free to think about more and more difficult problems. Difficult concepts (or techniques, if you’re looking at a physical task) need to become simple for you, so that they can become the building blocks of even more intricate ideas. This lets you make hard things look easy while the impossible becomes possible. For us humans, in most cases mastery is a heck of a grind.

While the strongest imaginable learner could approach mastery quickly and easily, for most people, the objective of being a strong learner means approaching competence quickly. In this regard (brag incoming) I’m happy to say that I’ve become a relatively strong learner. I’ve been able to achieve competence at most things that I set my mind to, and the more I practice learning, the better I get at doing it. Especially when it has come to video games, I have achieved competence at certain games and genres that, in the past, I never thought I’d be able to. Not mastery by any stretch of the imagination, but competence.

What’s interesting to me is that I don’t feel particularly smart or talented. I make plenty of very stupid mistakes all the time. Even my strongest skills--verbal ones, specifically--are rooted in a lot of goofy failures and errors; I just got them out of the way when I was younger. What has helped me become “smarter”--a better learner, in other words--has been acquiring learning tools that I can consistently apply when learning things in new arenas.

Based on things I’ve been studying in machine learning, neuroscience, and based on my own experience, I’ve been thinking about one of the strongest tools in my learning toolkit, and I’ve realized just how damn strong it is. It’s the kind of thing that I would automatically do sometimes, forget to do other times, and not even realize it counted as a technique! The best name I can think of for it is “predictive learning,” and that’s the real topic of today’s post.


The first thing to know is that your brain does not really “store” information. It’s not a big filing cabinet that just pulls up sheets of paper with information on them. It recreates information based on rules and past experiences.

Here’s what I mean. Imagine two video games playing two different cutscenes. The first game plays the exact same cutscene every time; it stores a video file and runs it when you trigger the cutscene. There is no way to change the cutscene unless you actually went in and screwed with the files in the game.

Now imagine the second video game recreates the cutscene every time. It grabs the current model of a bunch of characters, and then executes the cutscene according to a bunch of rules. If your character is wearing a different costume, then the cutscene runs with them wearing that costume. If it were an RPG, and your party could have different characters in it, then the cutscene runs with your different party members in the background.

The first game just stores the cutscene, and runs it when asked. The second one rebuilds it, every time, based on certain information and principles. Your brain works a lot more like the second game than the first one, and when you are learning how to do a skill, your brain is coming up with rules for how to do things. The brain is flexible though. It changes its rules when it learns that they aren’t quite getting the job done, and that’s the basis of learning.

Reread that last sentence. Your brain changes the rules based on how good of an outcome you get. However, you don’t just want to get good outcomes. You want your brain to be good at predicting what actually happens based on what it does. That’s a key distinction! When you get too focused on getting a “good” outcome, then you become satisfied with rules that don’t actually match reality. The heart of becoming a strong learner is not being satisfied with good outcomes, but striving for accurate predictions. You can get an outcome that you didn’t expect, but still succeed, and convince yourself that you knew what you were doing. Understanding this allows you to employ predictive learning to get better faster. It also has the benefit of making you look like a psychic genius, but that’s (probably) not your main goal here.

Initially, whatever you are learning about, you will start just by observing things. There’s not much point in trying to predict what will happen in a new situation when you have no rules for the system. You just start with nearly random inputs, or you start by watching other people. This phase gives you some correlations. It gives you some information on what to expect. At this phase, if somebody asks you what you think is going to happen, then your initial answer should be “I have no idea” or “here’s a wild guess but it’s probably wrong.”

If you are watching attentively, your brain will automatically start developing some rules for how the system works. It will start to say “based on what I saw, or what I know, this is what I think will happen.” Your brain has been doing this since you were a baby! You watched adults walking, and got an idea for the outcome that should happen. This is why kids will get themselves into the general walking posture; they know that it’s associated with walking. But without any information on how it felt to make the muscles do the job, all it could do was say “uh, get in this general position? I guess?” and then you tip over. Walking, it turns out, is actually really hard when you’ve never done it before.

That’s where the next phase comes in. The prediction phase. “If I do this, then THIS should happen.” Once you have some data, you will start to make active guesses about outcomes. Making these active guesses is super important. It forces you to take the rules inside your head and apply them to the system. It forces you to go through the steps of recreating the system in your head. It’s a test! Your goal is to get your rules to match up with reality. If I do X, then Y happens… yeah! That’s right! Now if I do A, then B should… no, it’s C. Hmm. Maybe… I do A, but a little more like this, and… yeah! It worked! You’re learning!

This is where predictive learning, however, gets trickier. You aren’t just trying to predict only the good outcomes. You are trying to develop an internal model of the system that is accurate all over. You want to get detailed. You want to get imaginative. This is what separates predictive learning from just memorizing. You don’t just say “this is how I get the good outcome,” but you can accurately answer the question “now what happens if I do things this way?” When you truly try to model the system that you’re learning, there are no trick questions. There is only the system and your inner model of it. You can predict the weird outcomes just as well as you can predict the normal ones. In fact, “weird” stops being a thing. If something is weird, it means that your inner rules aren’t accurate.

Let’s say I practice free-throws. I can “memorize” the free-throw, and I can get very good at it. But does that mean I will really understand the interactions between my body and the basketball? Does that mean I will be able to effectively take any other shot?

The short answer is no. Predictive learning, in its strongest form, means that you will do things like guess just how successful something will be. Or you guess where it will go wrong, how wrong, and to what extent. Predictive learning means you don’t just try to get a good result, it means that you try to model the end result, and you change your rules when your model is wrong. This is especially true the more and more detailed you try to get. You can apply predictive learning to your free-throw shot when you say “if I shoot like this, is that enough to get the ball to hit the backboard and sink?” Or “if I do this wrist movement, then the ball will bounce in if it hits the back rim… oh, no it, bounced out? I wonder why?” This keeps you from simply saying “okay I can sink my free throw shot all the time.” It means you understand, deep down, the interaction between the ball and your body, between the ball and the hoop. It lets you take those ideas and apply them elsewhere! It also lets you correct things when they start to go wrong; that day when you have a little tweak in your shoulder and your normal shot doesn’t work, you can work through the problem.

I want to be clear though, I actually don’t have anything against memorization. The free-throw shot is one you will probably have to take, on average, more than any other shot in basketball. It can pay off big time when you memorize the most common situations, especially if, in a given system, you can guarantee that those situations will occur. For example, you get your star player to practice free-throws all the time because they get fouled constantly to stop them from running wild on the court, and if they can sink all their free-throws then that means free points.

"Memorizing" in this case also just means "practice." Particularly when it comes to physical skills, there is exactly one way to get your body used to certain techniques, and that's to do them over and over again while comparing them against the ideal. Even when you aren't trying to become a master of something, as a human being this particular phase of learning can't be ignored, and I don't want you to think it can be. Complicated sequences of muscle interactions will demand that you grind and rehearse to get them perfect. Even while trying to employ predictive learning, if you want consistent performance then you need to develop consistent technique, and repetition is the cornerstone of that.

(Memorizing can also be useful to memorize certain things when you try your absolute hardest but just can’t seem to come up with the rules that cover everything. There’s this one thing that you have a habit of getting wrong… so you have your rules for everything else, and then you memorize that one weird case. It’s okay to admit that your inner model isn’t perfect. It’s a band-aid solution, and band-aids aren’t ideal but they’re better than bleeding everywhere)

The key here, however, is that you develop a strong inner model of the system you are learning. You constantly, actively try to guess how things will play out. Being wrong is a good thing, because it lets you improve your model. When other people’s rules have exceptions, but yours don’t, that means you have the real ones, the ones that account for the strange and the weird and the uncommon. That means that even the really wacky stuff just seems obvious to you, because that’s what your model says will happen… and then it turns out you’re right.

So don’t just watch, and don’t just try to force yourself towards memorizing how to secure a good outcome. Actively predict how things will turn out, and when you’re wrong, look for the thing you missed, or the thing that you paid too much attention to. When you are wrong, tweak yourself in the direction of being a better predictor. It takes a bit more effort, but it pays off.

Thanks for reading!