Why virtual reality (VR) might matter more than you think….

I have heard it claimed that the famous novelist William Gibson uttered his famous quote, “the future is already here — it’s just not very evenly distributed”, for the first time after experiencing virtual reality, decades ago.

We are fast arriving at a point where virtual-reality will be dirt cheap, and it will work really well.

A core issue right now, and that might surprise you, is that most people, including those who have tried virtual-reality goggles, cannot really say what virtual-reality is.

The naïve answer is that virtual reality provides immersive three-dimensional world view. That is, when thinking about virtual reality, people think about the display. And they could be excused for doing so by the fact that the physical devices appear to be focus so much on displaying pixels. We have goggles with embedded screens, and so forth.

But, actually, I submit to you that the display is not entirely essential. Of course, you need perception for an experience to make sense, but you could have virtual reality without any light whatsoever. You would probably have to focus on sounds, touch, and smell.

Virtual reality also does not need to be realistic. It is not at all obvious that the more realistic the representation, the better it is. You could have great experiences with a cartoonish worldview. That would side-step the uncanny-valley issue. I actually suspect that some of the best applications of virtual reality will not involve photo-realistic worldviews.

What actually matters with virtual reality is that it engages your whole body. That’s the crucial point. When you use a computer, your fingers (mostly three of them on each hand) do most of the work. I can sit in my campus office working, and because the lights are automated, it might go dark just as I am finishing off a sentence… because I am hardly moving at all when I work in a traditional manner with my computer.

If you were paralyzed, virtual reality would not help you in the least. At a minimum, for virtual reality to make any kind of sense, you must be able to move your head around. It is not so with traditional computing where as long as you can move your arms and use your fingers, your head can be mostly stationary.

I believe that it explains in part how virtual reality affects our perception regarding the flow of time. Virtual reality is somewhat tiring, compared with sitting at a desk, so fifteen minutes of interaction in virtual reality feel (as far as the body is concerned) as tiring as hours sitting at a desk. Thus, time is somewhat accelerated in virtual reality.

But I also theorize that virtual reality affects how you think in a less trivial manner. It favors embodied cognition. An athlete or a chef has a particular type of intelligence where the space around them becomes an extension of their own mind.

It is easy to dismiss such ideas as verging on the mysticism. Yet it is undeniable that we think differently when our bodies are involved. I have now reached a point where I set a clear separating line between in visu meetings and videoconferences. They are drastically different experiences, resulting in very different cognitive outcomes. For example, I believe that it makes no sense to conduct job interviews using video conferencing. And I say this as a nerd that avoids social interactions whenever possible.

That is, the view that we are brains in a jar is hopelessly naïve and wrong. The idea that we “think with our brains” is, in my view, only true as a first approximation. There is a continuum between our brain cells and the objects around us. A spider without a web is a useless animal. The spider uses its web as an extension of itself, to measure distances, track directions, and even as a perception device. Human beings do not have physical webs coming out of their hands, but we are simply much more advanced spiders, with the ability to create our own webs, like the world-wide-web.

I believe that many of the paradigm shifts that we have encountered as intellectuals come about through changes that have little to do with pure reason and a lot to do with our bodies and their perception:

  • Museums often present very little textual information. Mostly, you get to see, and often touch, artifact. It is through the presentation of inanimate objects that people acquire a feeling of how things were many centuries ago. Try, as an experiment, to view a three-dimensional representation of the object on a screen. It is not the same! The idea that you should collect and display objects to convey information is not entirely trivial, and yet we take it for granted today.
  • Though we might credit much of the rise of statistics to the formal mathematical results introduced by famous mathematicians… I believe that we should rather credit authors such as Playfair for introducing the modern-day line graph (in 1786!). If that’s all you had, you could still study effectively inequalities and climate change. But plots are much less rational than it appears: if you were to present line charts to people and ask them to describe what they see, they would have a hard time elaborating beyond a first-level interpretation. And the provided linguistic description would not allow others to understand what was in the graph. There is more in a graph than we can tell. In some sense, it is also easier to lie with statistics than with a plot: try plotting your own weight over the last few months… and compare the result with whatever statistical rationalization you might come up with. Lying with a plot requires a more deliberate attitude. I believe that there is a deeper story to be told about the relationship between the emergence of science and the scientific method: it seems clear that the line graph preceded science. I believed that it might have played an important role.
  • The industrial revolution came about after we got to experience automatons, these popular toys from the Victorian era (and earlier) where one could see gears moving underneath. The physical reality of these devices and the fact that you could, as a kid, look at them and eventually hold the gears in your hands, probably made a huge difference.
  • The early computers were programmed using plugs and cards… but soon we imported the keyboard into computing… the keyboard is an obvious cognitive extension first created to help us make music more precisely. Without the keyboard we would not have modern-day programming, that much is certain. Isn’t it amazing how we went from musical instrument to software programming?

All of these examples illustrate how altering our environment even in a minute way allowed us to think better.

My theory is that there are entire threads of thoughts, that we cannot have yet, that we cannot even imagine, but that virtual reality will enable.

There are still massive challenges, however. One of them is affordance. For example, many virtual-reality games and systems use the concept of “teleportation” to move you from one point to another. In my view, this is deeply wrong: it uses your hand as a pointing device, just as you would do in conventional computing. Grabbing and moving objects, interacting with objects in general, is awkward in virtual reality. I don’t think we know how to enter text in virtual reality. There is also a bandwidth issue. The screens of current virtual-reality goggles have a relatively low resolution which makes reading small fonts difficult, and reading in general is unpleasant. Interactions are also at a relatively large scale: you cannot use fine motor control to flip a small switch. Everything has to be large and clunky.

Still. I think that chances are good that new world-changing paradigms are made possible by virtual reality. It should allow us to build better webs as the spiders that we are.

5 thoughts on “Why virtual reality (VR) might matter more than you think….”

  1. I don’t even think of the goggles as real VR (as cool as they are). For me at least, it gets much more interesting once decoupled from our real bodies and all sensory input is generated artificially.

  2. Interestingly, I finally got around to reading TechCrunch’s article, “This VR Cycle is Dead”, yesterday, here:
    https://techcrunch.com/2017/08/26/this-vr-cycle-is-dead/

    While the article isn’t really opposed to any of your ideas in this post, it does point out that (at least currently) the implementation is too clunky and isolated to really gain traction. Will VR end up being the laserdisc of the 00s?

    The article also spends some time talking about how AR turns out to be a good middle ground… which rings true after seeing both VR & AR booths at Microsoft’s Build conference last spring.

    Specifically, I can easily imagine this type of scenario:

    imagine a $10k piece of equipment (a new jet engine or something?) that you want to carry around for training/sales purposes… you could replace that with $10k of AR equipment within a year or two (i.e. existing technology), reduce the costs, and expand what you are able to show people (zoomed-in views, more models or variants). Interaction between the instructor an students is much more natural.

    On the other hand, doing the same thing with VR strikes me as a lot clunkier (esp moving around, as you noted) and harder for people to take seriously (and a bit gimmicky).

    But of course… that could just be a failure of imagination on my part. Maybe VR just needs a couple of tweaks (resolution, movement) before it can reach critical mass. It’ll be interesting to watch things unfold

    1. While the article isn’t really opposed to any of your ideas in this post, it does point out that (at least currently) the implementation is too clunky and isolated to really gain traction. Will VR end up being the laserdisc of the 00s?

      That’s possible. But I am betting that we are simply over the hype, and down at the point where prices fall and people starting to build interesting things in their garage.

      The article also spends some time talking about how AR turns out to be a good middle ground…

      My view has remained the same in this respect. AR is a lot harder to do than VR. VR already works, right now. You can get fully immersive, convincing experiences today. I have not seen anything that works for AR. The closest thing to convincing AR I have seen is my iPhone (or iPad) adding Ikea furniture to my office. And I work with people who work on AR professionally. But, really, looking at the world through your phone? There are some niche applications, for sure, but not broad market, I think. Of course, if we are talking about 2025… then I have a different view.

      imagine a $10k piece of equipment (a new jet engine or something?) that you want to carry around for training/sales purposes… you could replace that with $10k of AR equipment within a year or two (i.e. existing technology), reduce the costs, and expand what you are able to show people (zoomed-in views, more models or variants). Interaction between the instructor an students is much more natural.

      You don’t need such a contrived scenario. Think about giving a class about… elephants. Imagine that everyone can see the elephant in the middle of the room. And so forth.

      Nice!

      But we don’t have anything like that! You cannot see the elephant in the middle of the room. You might get to see if through your iPhone, or something, but how natural is that?

      In VR, right now, I can show you an elephant and you will really feel like the elephant is there. It works. Today.

      Give me $200 headsets and I can distribute them to a class, and they can all get to see me by the elephant. That’s almost here. AR isn’t almost here.

      On the other hand, doing the same thing with VR strikes me as a lot clunkier (esp moving around, as you noted) and harder for people to take seriously (and a bit gimmicky).

      How is AR less gimmicky than VR?

      The reason moving around in VR is difficult right now is that you need to be connected to a powerful PC for high-quality rendering. This will not last very long. But note that exactly the same problem exists with AR… and it is worse. Adding realistic 3D objects to your environment in a way that tricks the brain is crazily hard. Yes, Apple has good technology… but it is not good enough to trick your brain.

      both VR & AR booths at Microsoft’s Build conference last spring

      I haven’t been particularly impressed by what I have seen from Microsoft. And yes, we have a HoloLens thingy.

      1. When I say that AR is ready, I was really basing that on some HoloLens demos 🙂 Definitely unimpressive (the small viewable area felt similar to just holding up my phone).

        Contrasting an AR demo booth with a VR one:

        I went into a (mostly) empty room with a tire. “clicking” on the tire created an expanded schematic view and a virtual instructor appeared and started explaining things. I was able to move around the room, walk closer or further from the different parts of the tire, and interact with the other people in the room… including asking how long the tire demo took to create (answer: a few months. It was fairly simple)

        (The tire acted as a fixed reference point for the rest of the graphics. Definitely a limit, but not too bad for a training environment.)

        OTOH there was a cool VR demo where you stood on a platform (irl) and rode up the side of a skyscraper (in the simulation). At the top, a gate opens (both irl and in the simulation) and you step out. People who tried it said it was pretty intense.

        While I was watching the VR demo–but not bothering to get in line to try it–is when I started wondering how viable VR really it. It’s definitely cool for super immersive experiences… but it also was a lot more hassle to get the gear set up and to interact with staff when you had a question. It seems really isolating.

        I also spent some time talking to someone at the Pearson booth (also at Build), and they’re in the middle of a longitudinal study with high school students… which she said is showing promising results.

        > How is AR less gimmicky than VR?
        haha. because I said so? talk about an arbitrary statement! sorry :-/

        anyway, great post 🙂 I’m just trying to synthesize the opposite outlooks on VR

Leave a Reply

Your email address will not be published. Required fields are marked *