Tuesday, January 30, 2018

Predictive Learning

I’m gonna start this post by bragging a bit. One of my greatest interests through my entire life has been learning. Not just “I want to be a lifelong learner,” but understanding the act and process of learning, getting better at it, refining it, etc. How do you jump into a new skill, possibly in some arena that you’ve never tried before, and acquire competence?

This is one reason that I’ve struggled to acquire real mastery over anything. “Competence” comes from knowing the principles of a skill and having some experience applying them. It allows you to reliably attain good outcomes. “Mastery,” on the other hand, comes from feeling the principles of a skill so deeply that you apply them automatically. Not only do you achieve good outcomes, but you see good outcomes where other people don’t, and you achieve them seemingly effortlessly.

Mastery demands time. No matter how quickly you acquire your competence, you will eventually hit a wall where the conscious application of your knowledge is not enough. It needs to be automatic, to let your mind be free to think about more and more difficult problems. Difficult concepts (or techniques, if you’re looking at a physical task) need to become simple for you, so that they can become the building blocks of even more intricate ideas. This lets you make hard things look easy while the impossible becomes possible. For us humans, in most cases mastery is a heck of a grind.

While the strongest imaginable learner could approach mastery quickly and easily, for most people, the objective of being a strong learner means approaching competence quickly. In this regard (brag incoming) I’m happy to say that I’ve become a relatively strong learner. I’ve been able to achieve competence at most things that I set my mind to, and the more I practice learning, the better I get at doing it. Especially when it has come to video games, I have achieved competence at certain games and genres that, in the past, I never thought I’d be able to. Not mastery by any stretch of the imagination, but competence.

What’s interesting to me is that I don’t feel particularly smart or talented. I make plenty of very stupid mistakes all the time. Even my strongest skills--verbal ones, specifically--are rooted in a lot of goofy failures and errors; I just got them out of the way when I was younger. What has helped me become “smarter”--a better learner, in other words--has been acquiring learning tools that I can consistently apply when learning things in new arenas.

Based on things I’ve been studying in machine learning, neuroscience, and based on my own experience, I’ve been thinking about one of the strongest tools in my learning toolkit, and I’ve realized just how damn strong it is. It’s the kind of thing that I would automatically do sometimes, forget to do other times, and not even realize it counted as a technique! The best name I can think of for it is “predictive learning,” and that’s the real topic of today’s post.

*

The first thing to know is that your brain does not really “store” information. It’s not a big filing cabinet that just pulls up sheets of paper with information on them. It recreates information based on rules and past experiences.

Here’s what I mean. Imagine two video games playing two different cutscenes. The first game plays the exact same cutscene every time; it stores a video file and runs it when you trigger the cutscene. There is no way to change the cutscene unless you actually went in and screwed with the files in the game.

Now imagine the second video game recreates the cutscene every time. It grabs the current model of a bunch of characters, and then executes the cutscene according to a bunch of rules. If your character is wearing a different costume, then the cutscene runs with them wearing that costume. If it were an RPG, and your party could have different characters in it, then the cutscene runs with your different party members in the background.

The first game just stores the cutscene, and runs it when asked. The second one rebuilds it, every time, based on certain information and principles. Your brain works a lot more like the second game than the first one, and when you are learning how to do a skill, your brain is coming up with rules for how to do things. The brain is flexible though. It changes its rules when it learns that they aren’t quite getting the job done, and that’s the basis of learning.

Reread that last sentence. Your brain changes the rules based on how good of an outcome you get. However, you don’t just want to get good outcomes. You want your brain to be good at predicting what actually happens based on what it does. That’s a key distinction! When you get too focused on getting a “good” outcome, then you become satisfied with rules that don’t actually match reality. The heart of becoming a strong learner is not being satisfied with good outcomes, but striving for accurate predictions. You can get an outcome that you didn’t expect, but still succeed, and convince yourself that you knew what you were doing. Understanding this allows you to employ predictive learning to get better faster. It also has the benefit of making you look like a psychic genius, but that’s (probably) not your main goal here.

Initially, whatever you are learning about, you will start just by observing things. There’s not much point in trying to predict what will happen in a new situation when you have no rules for the system. You just start with nearly random inputs, or you start by watching other people. This phase gives you some correlations. It gives you some information on what to expect. At this phase, if somebody asks you what you think is going to happen, then your initial answer should be “I have no idea” or “here’s a wild guess but it’s probably wrong.”

If you are watching attentively, your brain will automatically start developing some rules for how the system works. It will start to say “based on what I saw, or what I know, this is what I think will happen.” Your brain has been doing this since you were a baby! You watched adults walking, and got an idea for the outcome that should happen. This is why kids will get themselves into the general walking posture; they know that it’s associated with walking. But without any information on how it felt to make the muscles do the job, all it could do was say “uh, get in this general position? I guess?” and then you tip over. Walking, it turns out, is actually really hard when you’ve never done it before.

That’s where the next phase comes in. The prediction phase. “If I do this, then THIS should happen.” Once you have some data, you will start to make active guesses about outcomes. Making these active guesses is super important. It forces you to take the rules inside your head and apply them to the system. It forces you to go through the steps of recreating the system in your head. It’s a test! Your goal is to get your rules to match up with reality. If I do X, then Y happens… yeah! That’s right! Now if I do A, then B should… no, it’s C. Hmm. Maybe… I do A, but a little more like this, and… yeah! It worked! You’re learning!

This is where predictive learning, however, gets trickier. You aren’t just trying to predict only the good outcomes. You are trying to develop an internal model of the system that is accurate all over. You want to get detailed. You want to get imaginative. This is what separates predictive learning from just memorizing. You don’t just say “this is how I get the good outcome,” but you can accurately answer the question “now what happens if I do things this way?” When you truly try to model the system that you’re learning, there are no trick questions. There is only the system and your inner model of it. You can predict the weird outcomes just as well as you can predict the normal ones. In fact, “weird” stops being a thing. If something is weird, it means that your inner rules aren’t accurate.

Let’s say I practice free-throws. I can “memorize” the free-throw, and I can get very good at it. But does that mean I will really understand the interactions between my body and the basketball? Does that mean I will be able to effectively take any other shot?

The short answer is no. Predictive learning, in its strongest form, means that you will do things like guess just how successful something will be. Or you guess where it will go wrong, how wrong, and to what extent. Predictive learning means you don’t just try to get a good result, it means that you try to model the end result, and you change your rules when your model is wrong. This is especially true the more and more detailed you try to get. You can apply predictive learning to your free-throw shot when you say “if I shoot like this, is that enough to get the ball to hit the backboard and sink?” Or “if I do this wrist movement, then the ball will bounce in if it hits the back rim… oh, no it, bounced out? I wonder why?” This keeps you from simply saying “okay I can sink my free throw shot all the time.” It means you understand, deep down, the interaction between the ball and your body, between the ball and the hoop. It lets you take those ideas and apply them elsewhere! It also lets you correct things when they start to go wrong; that day when you have a little tweak in your shoulder and your normal shot doesn’t work, you can work through the problem.

I want to be clear though, I actually don’t have anything against memorization. The free-throw shot is one you will probably have to take, on average, more than any other shot in basketball. It can pay off big time when you memorize the most common situations, especially if, in a given system, you can guarantee that those situations will occur. For example, you get your star player to practice free-throws all the time because they get fouled constantly to stop them from running wild on the court, and if they can sink all their free-throws then that means free points.

"Memorizing" in this case also just means "practice." Particularly when it comes to physical skills, there is exactly one way to get your body used to certain techniques, and that's to do them over and over again while comparing them against the ideal. Even when you aren't trying to become a master of something, as a human being this particular phase of learning can't be ignored, and I don't want you to think it can be. Complicated sequences of muscle interactions will demand that you grind and rehearse to get them perfect. Even while trying to employ predictive learning, if you want consistent performance then you need to develop consistent technique, and repetition is the cornerstone of that.

(Memorizing can also be useful to memorize certain things when you try your absolute hardest but just can’t seem to come up with the rules that cover everything. There’s this one thing that you have a habit of getting wrong… so you have your rules for everything else, and then you memorize that one weird case. It’s okay to admit that your inner model isn’t perfect. It’s a band-aid solution, and band-aids aren’t ideal but they’re better than bleeding everywhere)

The key here, however, is that you develop a strong inner model of the system you are learning. You constantly, actively try to guess how things will play out. Being wrong is a good thing, because it lets you improve your model. When other people’s rules have exceptions, but yours don’t, that means you have the real ones, the ones that account for the strange and the weird and the uncommon. That means that even the really wacky stuff just seems obvious to you, because that’s what your model says will happen… and then it turns out you’re right.

So don’t just watch, and don’t just try to force yourself towards memorizing how to secure a good outcome. Actively predict how things will turn out, and when you’re wrong, look for the thing you missed, or the thing that you paid too much attention to. When you are wrong, tweak yourself in the direction of being a better predictor. It takes a bit more effort, but it pays off.

Thanks for reading!

Tuesday, January 23, 2018

My Rain World Retrospective

One of my favorite video games of last year recently turned into one of my favorite video games of all time. Before, I might have placed it in my top 20, but now, as the result of a recent content update, it is now actually in my top 3. If you have watched any of my streams semi-recently, you’ll know that I’ve been hooked on it. I wanted to write a review, or retrospective, or something about the game for some time now, and since I said I would write more blog posts this year, this is it.

For people who haven’t played or seen it, Rain World is a 2D platformer with a focus on survival and exploration. You play as a little creature called a Slugcat that has been separated from its family, and you begin the game by questing to find them. Each night it rains, and your job is to find food and shelter. Certain rooms are special refuges from the rain, and when you’ve eaten enough food each day, you can hibernate to wait out the rain. Gameplay is a balance of exploring the world, finding shelter, finding food, and traveling from region to region while you figure out what’s going on.

Like a lot of my favorite games, it blends cuteness and ruthlessness. The main character is very adorable, which makes it a shame that you are going to see it die so much. Slugcat is rather fragile and preyed on by the many creatures roaming around. It’s easy to get lost, the controls take a bit of getting used to, and you will find yourself alternating between confused and dead with great frequency.

I’ve been playing video games for most of my life, and Rain World is near the top of my list. If you go looking at reviews, you’ll see a lot of complaints, and I can’t imagine many people want to play through it. So what do I like so much about it?

It’s a Roleplaying Game, For Real

There are a couple genre names that don’t really mean much except by convention. The two that come to mind are MOBA and RPG. “Multiplayer Online Battle Arena” does not explicitly tell you to expect a team-based game that usually has lanes, farming, towers, items, and a base to blow up, nor does “Roleplaying Game” explicitly tell you that there will be leveling and weirdly-dressed teenagers.

Despite all the other labels I gave it, Rain World is an honest-to-goodness roleplaying game. It actually gets you feeling like your character would. There are a few main criticisms of the game that you will see if you check out reviews, and they look something like:

1) I'm confused
2) I'm lost
3) I'm frustrated
4) I keep dying
5) The game isn't helping

To be very, very fair, these are all totally reasonable reactions to playing Rain World. I don't feel like these are inaccurate or unexpected responses.

Here’s the thing. Your little Slugcat roams a slowly decaying civilization, surrounded by creatures and buildings and structures that are all bigger than it. The world does not make sense, outside of “eat, sleep, don’t die.” You go searching, but it’s easy to get lost. The buildings stand tall and imposing, but are mostly host to your attempts at survival; their actual purpose is difficult to determine.

And this is what makes it a roleplaying game. If you say “hey, you’re a plucky teenager making his way in the world on some kind of wacky adventure,” but a few hours in I am absolutely manhandling a bunch of experienced mercenaries twice my age and size… well, it’s a game, and it might be a fun game, but I sure as heck don’t feel like it’s me against the world. I get to feel cool and powerful, but I don’t go mowing down hordes of nameless soldiers and monsters and come away feeling like I’ve stepped into any particular role other than God of Destruction.

Rain World, on the other hand, is not looking to make you feel powerful. The game is looking to make you feel like Slugcat. It is looking to make you feel like a little creature dropped in a world that does not care about you, does not make sense to you, will not be kind to you, and feels no need to explain itself. You are lost and you are small. It is not actually a pleasant feeling.

Every single enemy represents a real threat to you. Having played the game a lot, I can safely say that every single creature in that game has gotten me killed, whether it was friendly, hostile, or neutral to me. It’s a scary world out there, and the only time the game lets you really breathe is when you reach a shelter.

There’s a quote out there (whose origins I don’t know) that goes: “Every morning in Africa, a gazelle wakes up. It knows it must run faster than the fastest lion or it will be killed. Every morning a lion wakes up. It knows it must outrun the slowest gazelle or it will starve to death. It doesn’t matter whether you are a lion or a gazelle: when the sun comes up, you’d better be running.”

Like a lot of people in developed civilization, my day-to-day life is not closely entangled with concerns about my own mortality. Meat grows in supermarkets. Hunting and gathering means “finding restaurants.” We are mostly surrounded by each other and have a lot of rules against hurting one another. We’ve walled up and shoved out most of the creatures that can kill us easily. Death is mostly considered a tragic accident that nobody should have to worry about. Our lives are relatively predictable, and there is no shortage of information on how to navigate the world we’ve built for ourselves. Even if you decide to go camping in the wilderness, you probably travel in a secure vehicle and bring your own food and shelter and have some way of getting in contact with other people. It’s not exactly behind glass, but it’s close.

Rain World asks “what if you didn’t have any of that?” and the answer is “it’s not pleasant.” There’s a reason we went to so much effort to put up the walls and roofs and bring the more edible creatures inside with us. There’s a reason we spend years trying to teach children how the world works before we throw them into the middle of it. But once you’re just a creature in the wild, your primary concern is “find food somewhere, find a place to sleep, and don’t die,” every day. Lions don’t hand you flashcards and sharks only give you one short and violent pop quiz. If you can’t handle it, you can be the meal for something else.

Rain World, as a game, honestly feels like it just doesn’t care that much about me. It doesn’t feel like it really privileges me as the player. I get the feeling that Rain World is like some mountain range off in the wilderness that doesn’t care whether you ever reach the peak or experience its marvelous vistas. If you want to, that’s on you. The mountain will keep being a mountain, and Rain World will keep being Rain World, as cold and uncaring as nature. You just don’t matter to it.

It didn’t feel like the devs screwed up here, either. Sometimes you play a crappy game and you can tell that whoever made it just didn’t really know what they were doing. That’s not the feeling I get from Rain World. It seems that this is what they wanted. It might not be everybody’s cup of tea, but it was certainly unique.

It Feels Like a Living System

Now that I’ve streamed it a bunch, one of the most frequent comments I’ve seen is “watching this game feels like watching a nature documentary.” In part, this is because the creatures in the game have been carefully designed to seem alive. Not only are the animations are procedural, but their behaviors feel very animal. They will bob and move erratically, they will pause, they will stop and turn around for no apparent reason, and though you can learn their patterns they still have a knack for surprising you or doing weird things.

Most importantly though, the creatures all interact with each other. Their lives do not revolve around you, the player; they are all trying to eat and survive, just like you. When you wake up each day, your entire region loads, and all the enemy behavior is simulated even when offscreen. Though they do have common spawn locations, from the moment they wake up, it’s a mixed bag where they’ll go. But wherever they go, they will act like themselves.

This, by the way, is one of the reasons it took a content update to shoot Rain World so high up my list. It not only added a few new creature designs, but it also added a harder difficulty in which, among other things, more creatures spawn. This means there are a lot more opportunities to see them interact (or get caught in the middle of them, and try to use their interactions to your advantage). You are more likely to enter a room and see other two enemies in the middle of a scuffle, and you are more likely to get different species of creatures nearby one another to use as resources (since same species typically won’t fight).

Again, you get that feeling that you’ve honestly been dropped into a living world, and you’re just one of the many. You also have many opportunities to learn the behavior patterns of all the creatures, and they are surprisingly detailed. With a few hundred hours of time in the game (don’t judge me) I still learn little things about them. Sometimes by getting eaten.

You Are Not Powerful, But You’re Capable

Speaking of getting eaten… one of the things about this game that will drive you mad is how unfair it all feels. As I said above, this is a big part of that feeling that you are a small creature in a big world. You survive by eating fruits and little bats that flit about dumbly, and your main weapons are small rocks and sticks that you pick up and throw at things. Learning how to reliably kill enemies is a process that involves you getting bitten and snapped at and knocked around, and even once you have a lot of knowledge under your belt, little things go wrong and then you’re dead again. It really does feel unfair.

Yet there are many creatures, small and fragile, who still live. They are the survivors, the latest links of an unbroken genetic chain that has been forged through millions of years of trial, error, life, death, and luck. Your Slugcat is very much the same. It’s a big world, but the tools you need to survive are there. Most of the deaths are your fault, and you could have done something more to survive. Some of them are because you didn’t know how things worked, but that’s life. Ants will cluelessly walk into antlion traps and be devoured. A fish is dead in the water when a faster swimmer spots it from outside its own vision. The only answer is “don’t do that thing you did,” but you learn too late. At least Slugcat gets to respawn! With each death, you ideally learn a little more about how to navigate the world, where you are strong, where you are weak, and how to keep going. Some deaths are bad luck, and come from being in the wrong place at the wrong time. Again, this is all very realistic, which is a popular trend in games these days, I hear.

Even in your more “realistic” games though, you get to feel powerful. The least powerful you feel is probably when you play multiplayer games and get trashed by people better than you, but at least you probably have guns or spells or special abilities or something. In Rain World, you start out with sticks and rocks, and you pretty much end with sticks and rocks. In between, you will find things in the world that you can use to survive. Many of the items are multipurpose, but you will only find out what they do by experimenting, because the game doesn’t go out of its way to tell you.

Nevertheless, the path to victory is there. As you learn, as you get more clever and knowledgeable, the game becomes more and more traversable, and you start to realize that the tools for success are lying everywhere. The game that once felt so unfair almost starts to seem benevolent. Sometimes I get through a region and almost get embarrassed at myself for ever thinking the game was tough. I typically get humbled shortly after that, but nonetheless, I’m amazed at how rich the world is with opportunities to survive and succeed. Dying to “bad luck” is almost always the result of pushing my good luck too far, or doing something even when I knew it was a risky idea.

With all that in mind, the result is that I feel good about succeeding in this game. It does not hold my hand. It usually tries to take whatever’s in it and then bite the hand off. It is not a game where you get a quicktime event, hit one button, and the game tells you how legendary you are. It’s a game where victory is making it through the day, and if you succeed, then you probably earned it. You are not powerful, but you are capable of survival, and ultimately triumphing. There aren’t many games that have made me feel as accomplished as I do when playing this game well.

Other Stuff

That’s my experiential take on Rain World. Lastly, here is some of the other stuff you’d expect from a review.

From a visual perspective, I think the game is beautiful, detailed, and lovingly crafted. Even if the game didn’t have an ending, your reward for playing, surviving, and exploring would just be getting to see everything. After struggling to navigate the earlier regions and finally surviving, seeing some of the backgrounds and vistas gave me a feeling similar to what, I imagine, mountain climbers feel when they reach a summit after a grueling ascent.

The sound design is phenomenal, and much of it lies in the sound effects. There is music, but it rarely plays. I find that a shame, because I enjoyed the music quite a bit, but silence serves the tone and feel of the game well. Music either kicks in when enemies arrive (it becomes one of your most useful cues), or when you reach an area that is mostly devoid of threats and you have some time to appreciate your surroundings.

There is lore and background to the world, and you can ignore it or investigate it as you please. I feel like the atmosphere of the game is enough to carry it, but the developers put quite the detailed backstory and information into the game, which was fun to go hunting for (or just reading via wiki online, when I decided I didn’t want to do all the work myself). As you go through the game and piece things together from a behavior standpoint, with the objective being “don’t die,” it’s nice to also get a wider-view picture of why things are the way they are. Like the rest of the game’s information, it is not particularly easy to access, nor does it intend to be.

The controls are precise, but they end up feeling clunky because of how precise they are. Do things correctly and they work every time. Do things slightly wrong, and you usually get a very wrong outcome. If you are handling yourself well, you will understand why your character is called a cat; if you aren’t, you understand why your character is called a slug. Things that seem inconsistent have, time and again, turned out to be very consistent once I actually investigated them. However, when the action gets frenetic, or you are racing against time to reach a shelter, if you don’t keep your cool and input things properly, you can end up paying big time.

Ultimately though, the controls are mostly quite good. Except the swimming. The swimming can go drown itself.

Conclusion

Strangely enough, as much as I adore this game, it can be hard for me to recommend this game to people because of how demanding and aggravating it can be. If you think you have the necessary tolerance for frustration though, then I do recommend it, wholeheartedly. It is full of countless, delicious rewards, in the form of the environments you see and the knowledge you gain and the accomplishment you feel just from surviving. It’s a system full of simple but detailed elements, all interacting and offering you new (and often lethal) surprises. You really won’t play another game quite like it.

Thanks for reading. See you next time.

Wednesday, January 17, 2018

Compete Complete in 2018

I didn’t make it a resolution, but I probably should have: I’d like to write more in 2018.

I know a lot of other people had it rough in 2017, but it turned out to be a good year for me. A lot of cool things happened for me, chief among them finally finishing a bachelor’s degree! I’m jumping straight into a master’s program now, and it promises to be very busy.

(Since several people have asked, the BS was in Cognitive Science, and the MS is for “Applied Cognition and Neuroscience” with a focus on machine learning)

There has been a lot on my mind lately, and I have sat down to write numerous times, but ultimately I haven’t finished anything. I think that my blogging mostly remained consistent when I was 1) competing and 2) had a schedule. I don’t plan to enter tournaments for much in the near future (I’ll be going to Frosty Faustings this weekend and that’s about it), but I miss writing and I think it has helped me in many ways, so we’re going to at least stick with #2.

Right now, based on my class schedule, I will strive to release one post a week on Tuesday. I’ll keep it up for a month and see how that goes, and if the drive is strong, I’ll bump it up to Tuesday and Friday like it was some time ago.

I am not sure what I will post about after this first month. Probably learning (it’s been a big focus of my education) and my own psychology, as usual. However, I do have my next three posts picked! In no order:

  • Prediction as a learning tool
  • A short primer on doing esports commentary
  • A review/revisitation of one of my favorite video games (not Smash)

Thanks for reading. I’ll see you next week!