Imagination

Imagination: a feedback loop of expectations

Imagination is a difficult concept to define because it’s so tied up in our idea of what it means to be conscious, or even what it means to be human. We can imagine things that are very simple things or very in depth, and we can talk about an idea like “imagine what it would be like to imagine”, and we do these kinds of things constantly. There’s a lot of room for complexity and confusions because it’s an idea that’s so tied up with the way that we talk about thinking. To help define how imagination works let’s first focus on one very specific characteristic of our consciousness – continuity. I’m going to describe my experience of consciousness because that’s the only one I can accurately describe, but I’m assuming that other people have the same kind of experience.

My consciousness is continuous, this seems pretty straightforward, but I’m not sure what it would be like if it wasn’t true. My experience isn’t made of separate parts, when I look at something, it’s not pixelated or segmented. No matter where I focus my attention it can flow smoothly from any one part of my experience to any other part without going over any gaps or “seams.” That extends to other senses too, touch, sound, taste, etc. are all continuous, and they’re continuous with each other. My attention can move from any part of my conscious experience to any other party seamlessly. If I’m holding a rock I can focus on its color, and then its weight or its feeling of hardness and coldness, and I have the sense of it being the same feeling attention smoothly transitioning between all these different areas of my consciousness. I don’t have the experience of pixelation or parts of consciousness, but of a continuous whole.

Not only is my experience seamless across space, but across time as well. By that I mean that I don’t observe a series of frames or points of data that are strung together. In the same way that it’s impossible to identify the smallest amount of space I can look at, or any separate parts of my experience, I also can’t identify the smallest amount of time I’m aware of. It’s always blended in to a single continuous moment with no obvious beginning or end, and no experience of breaks between moments or experiences. This continuity experience isn’t something I have to create, it’s a fundamental feature of my consciousness, I don’t experience motion as something I need to calculate or figure out, but as a characteristic of the things I’m observing. Continuity across all the things I can experience is the way that my brain creates the entirety of my consciousness, at least as far as I can tell.

However, that’s not always the case, there’s people who because of brain damage are unable to see motion, a condition called Akinestopsia^[1]Wikipedia article on Akinestopsia. They actually see the world as a series of stills, and are unable to naturally process that information to create the sense of movement. It’s part of their conscious experience that their brain is unable to create, and just being intelligent or observant or able to learn isn’t enough to create the ability to experience motion. They don’t lack the physical ability to take in motion, they lack the ability to create a conscious experience that includes motion. This is similar to people who have blindsight^[2]Wikipedia article on Blindsight, the problem isn’t with their eyes, it’s caused by a problem with the way their brain creates their conscious experience. In both of these cases a part of our experience that we take for granted is left out.

To talk about imagination I want to highlight another part of our conscious experience that we take for granted, something our brains just create for us all the time, and that’s so integral to our experience of the world that we don’t usually think about it. We constantly have expectations based on our current and past observations and experiences. When I’m holding a rock I expect it to stay the same in some ways, and I also have expectations about the ways it will change. If I squeeze it it’ll stay hard, but might warm up slightly. If I release it it will drop, and as it falls I have expectations about the way it will look as its distance and lighting changes. Expectations are another kind of thing that we’re capable of experiencing. In the same way we can experience sound or sight or temperature.

These expectations are a constant part of conscious experience, and we might not always be aware of them, but they seem to always be present. And we can consciously choose to shift our attention to them and focus on them, and we can do that smoothly across our continuous experience of consciousness. For example, we’ve all had the experience where we’re “zoning out” and not paying attention to what’s right in front of us. Our attention is elsewhere, so that even though we’re looking at something we might not actually see it or recognize it. And then we can focus our attention on what’s there and, when we do, we actually see it. There was a part of experience that was obvious, we just didn’t notice it when we weren’t paying attention to it.

We have expectations constantly, and most of the time they’re right and obvious and we don’t even notice them. But accuracy isn’t necessary to have an expectation, in fact all expectations involve making an assumption of some kind. Assuming that things will keep changing in the same way, or that they’ll stop changing or change in some new way. An assumption is just taking information from two observations and piecing them together. I assume that the subway in Chicago works like the subway in Boston because I combined my observation of how the subway looks in one city, with the way it runs in another. An assumption or “what if” can happen when we find two patterns that have some similarities or overlap and we combine them. And our assumptions certainly aren’t limited by what’s likely or even physically possible. I can assume that a car could fly like a plane, and see what I’d expect that to look like.

We can have these complex, and even unrealistic expectations, but they’re still part of our conscious experience. And sometimes we can pay attention to our expectations and even string several expectations together. When we do this intentionally, and can explain the steps we’re taking, we call this a prediction. A prediction is like a very concrete form of imagination, it’s when we can picture all the steps, all the assumptions and individual expectations that lead to the next that lead to the prediction. But imagination doesn’t have to be concrete and predictable, when we’re imagining we’re focusing on our expectations and these expectations are feeding back in to the next layer of expectations and the next. This feedback loop of expectations allows our imagination to move further and further away from our current observations and become more varied and less grounded. The assumptions and “what ifs” that are part of our expectations start to contribute more and more to our experience, and our expectations have less and less to do with what we’re observing right now, and more to do with how we can piece together expectations based on our past experiences. It’s the fact that our expectations are a seamless part of our consciousness that lets us do this. We can focus our attention on this one part of our experience, and the feedback loop can then create a lot of complexity and depth. When this part of our experience starts to take over all of our available attention we call it day dreaming, or “getting lost in our thoughts”.

Imagination doesn’t always have to be consuming though, it can just be a process of paying slightly more attention to our expectations than usual. We’re so used to these expectations and ability to focus on them that it can be weird to think of it as being anything other than the default. But, let’s try to picture what it would be like if our imagination was much more limited, if it wasn’t so easy to focus on our expectations or create a complex feedback loop of imagination? Perhaps that’s even what it’s like to be for some of ancestors, like apes. Without our ability to imagine complex possibilities, it would make many kinds of planning much more difficult, we wouldn’t be able to picture all the different possible ways to solve a problem. We would be more limited in what kinds of choices we could make because we couldn’t picture the different options. Instead we’d be much more locked in to responding directly to what we observe instead of responding to what might be possible.

I also want to look specifically at one very special kind of expectation, that feeling of knowing what someone is about to say. When you know someone very well and have a lot of experience with them we can learn what kinds of things they say, when they say them, etc. Our brain is great at making predictions and forming expectations and we do this naturally all the time. But there’s one case that stands out, one person who our brain has the most experience predicting – ourselves.

What would it feel like to be consciously aware of an expectation of what I was about say out loud? It would essentially be imagining what we would say to ourselves, or to put it another way, it’s would seem like our own voice talking to us. Which is a description of what we’d normally just call “thinking”.  We can’t know for sure exactly what processes in our brain create consciousness or the ability to focus on our expectations, predictions and imagination. Maybe “thoughts” really are some special kind of creation that comes from an entirely different process? But when I try to predict what I would say to myself, and when I try to focus on what it’s like to “think to myself”, the two seem exactly the same. And so defining “thoughts” as an “expectation of what you would say to yourself” seems like a useful way to describe this process, as essentially as special form of imagination. This also ties together a lot of important parts of what we consider human intelligence – imagination, thoughts, and predictions.

I’d go so far as to say that when we talk about AI, while it originally meant “artificial intelligent”, what it should really stand for is “artificial imagination”. This would explain why all of our advances in computing and “artificial intelligence” are often dismissed as “not real intelligence” We want AI to solve problems the way humans do, and that means picturing possible solutions, thinking of which one would be best, making complex plans, and maybe even talk to itself with a “stream of consciousness” to figure out what it wants to do. Basically we want it to use imagination, like we do. We want it to use its past experience of the world, combined with a feedback loop of current expectations to come up with possibilities and then settle on one. What we really want our machines to do, to solve problems like us, is to have artificial imagination.