OCTOBER 19, 2020
IF A NEIGHBOR’S DOORBELL rings while you are reading a book, you are forced to hear it but can usually easily ignore it. If that same doorbell rings while you are watching a movie, you might spend a subconscious moment or two wondering if it was from the film before quickly deciding it wasn’t. If the doorbell rings while you are in a virtual reality (VR) game, things get more interesting. The subjective VR world comes crashing down and multiple, successive thoughts have to happen to sort through the existential mess before arriving back at the ground truth: “I am in my living room. I am playing a game. That was my neighbor’s doorbell. This is not real.”
One of the scariest parts of VR is that when you close your eyes you are still in its world. If you nap during a movie, or while reading, you wake up on your couch or in your chair. If you wake from a nap in VR, you wake up not to where your body is, but to where your mind is.
To some artists, VR is a playground; to some psychiatrists and therapists, its fidelity to R provides a potential path to healing trauma or addressing fear through desensitization; to some writers, it is an über, multisensory stream-of-consciousness; and to some neuroscientists, it is experimental data for theories about how the brain creates, validates, remembers, and infers its assumptions about the world.
A common tack in early VR games, like Virtual Virtual Reality (V-VR), has the player putting on ever more goggles in order to nest virtual experiences, like subjective Russian Dolls. Having already donned the first headset — the real one, the one you buy and charge with electricity and put on your actual head — you then become comfortable enough with the act to keep doing it. A formulaic and common ending to these stories thus involves a final revelation, the likes of an et voilà or curtain reveal that the virtual world was, in fact, a layer over a photo-realistic one (e.g., The Key, A Fisherman’s Tale, etc.).
In a few of the games I played, my character’s death often brought me back “up” a layer, rather than back to the start of the level, which is what happened in the simpler days of video gaming. This works extremely well as a narrative device because everybody knows the feeling, which is that of snapping awake from a dream because of a near-death fall. It is as effective a narrative sleight of hand to the modern mind as I imagine was the revelation to 17th-century readers that the first half of Don Quixote had been published in the fictional world of the second half because, like rapid edits in film, or like appearing in Valhalla, heaven, or an afterlife, the experience should be more jarring in retrospect than it actually is but the brain makes quick, confabulative work of the surprise.
Multiple VR games or stories — (e.g., Superhot, where time moves only if you do, as if in a chapter from Alan Lightman’s Einstein’s Dreams; or Accounting+, where you play a virtual accountant trapped in a lonely, grotesque teenage boy’s VR set) — go one embodied step further, requiring a player’s character to feign the act of suicide with a gunshot to the head, a fall from a tall building, or, in a particularly difficult one, by pouring acid on the character’s head. The difference between this and a fall into a pit in Super Mario Bros. is that in these games you take the virtual hand, which is in the approximate place where your real hand is and take the virtual gun or the virtual jar of acid and point/pour in the approximate place where your actual head is.
But there is a moment in the V-VR which stood out to me as more difficult, harder to understand, and more existentially frightening than any other I had tried. In V-VR, playing as one of the last humans on earth, you are a kind of gig-economy laborer for various artificial intelligences. You work at an employment center and are given a pair of VR goggles for each new AI client. It’s an elaborate, Rube Goldberg box of simulation and escape room, thrilling to experience. But in one scene, you are a few goggles down the rabbit hole and the game tells you there has been an error. You are stuck in this one scene. Some sort of software bug. Can’t go back up anymore. Sorry.
Because the virtual hand tracks so well to your actual hand, and because the game brilliantly made you repeat the gesture required to take off the real, Subjective Russian Doll Layer Zero (SRD-0) headset in order to also take off each of the virtual ones, the effect, on me, at that moment, was cave-dive claustrophobia. Previously, to “escape” a layer — to go from SRD-5 to SRD-4, say — you would bring your real hand up to your real face and pull the virtual headset off in a motion very similar to what would remove the real SRD-0 headset. After performing this motion hundreds of times, it feels routine, which is exactly what the game exploits in that moment of software error because removing the virtual headset no longer works. Which means that my brain also thought that removing the real headset wouldn’t work. I was stuck. Here. In this blocky desert landscape, forever. I had internalized the dilemma as if I was the final pip of the Russian Doll set; had, for a few seconds, truly forgotten that the last and final layer was this, the real world. My brain had assumed I could only reverse out iteratively, from SRD-10 to SRD-9 to SRD-8 and so on until I was back to 0.
I knew on some level that I was in my living room but did not remember how to get back to it. The only time something vaguely similar ever happened to me was, once, while snorkeling in the Atlantic Ocean, floating, face down, with plastic SRD-0 snorkeling goggles sucked to my face. I saw a small fish coming at me and my brain couldn’t tell whether it was a large fish far away or a small fish up close. Like a doubly exposed photograph, the dual interpretations bled into each other, flipping back and forth, as if each fish had the face of a Necker Cube. Similarly, while in V-VR while also in my living room, my mind held two simultaneous but incompatible thoughts, which I can only imagine resembles something like a micropsychosis: I was stuck, forever, in V-VR and also stuck, forever, in my living room.
Eventually, like pinpointing which layer of reality a neighbor’s doorbell exists in, I realized what was happening, pulled off my SRD-0 headset, and sat back on my suddenly-real-again, no-more-levels-to-go couch.
VR’s immersiveness excites some clinical researchers, like the gastroenterologist Dr. Brennan Spiegel. His new book, VRx, explains how VR can be helpful in a variety of therapeutic ways. Early on, he recounts how a patient, convinced he had food stuck in his throat, but who was actually having a panic attack, became calm:
Rather than admit him to the hospital and perform a food disimpaction, I ordered a VR headset to the emergency department. Once it arrived, I dialed up a tranquil beach scene with calming music and asked him to give it a try, which he did. And then…nothing. Absolutely nothing. He sat down and stared straight ahead, completely transfixed, silent, and motionless.
Had the VR treatment worked? Spiegel thinks so, noting tears pooled in the man’s goggles and invoking humoral logic — “VR cooled his overheated mind” — to explain the salve. The idea Spiegel seems most excited about is that VR might be as potent as conventional anxiety, sadness, or stress-relieving treatments, including drugs, but without the side effects; that it might augment, or potentially replace, certain talk, pharmaceutical, or wellness therapies; that a digicopoeia might one day replace the pharmacopoeia; and that, despite the VR set possibly working like a powerful sedative, surely it would be less addictive than antipsychotics or antidepressants, right?
Across many chapters, Spiegel muses on the existential half-life of a floating, near-death experience he had in VR and what it means for the edges of selfhood, thus joining a long line of people questioning the nature of R by asking, like a cartographer might, about its borders, including what it is like to be a bat (Thomas Nagel); what it might be like to be a brain in a vat (Hilary Putnam); whether one would choose to live in a virtual “experience machine” if it was convincing enough (Robert Nozick); whether thermometers, even those stripped of what we might call sensation, are conscious (Daniel Dennett); what the nature and limits to knowledge are (René Descartes); whether causal coincidences should count as useful data (David Hume).
More than once, Spiegel recounts how therapeutic breakthroughs can occur in patients mere minutes after trying VR, in contrast to the years it can take traditional talk therapy or the weeks to months needed for many drug-based methods. He tells the story of how one patient with bouts of depression and a deep pain in her stomach was able to discover its source — after all the medical tests came back negative. It was her brother’s stomach cancer, she claimed. She was afraid she had it too, she realized after only a few minutes swimming with dolphins in VR. Her explanation is recounted, verbatim: “A year on the couch wouldn’t have gotten me here. There’s something about these dolphins. They freed me up to think. They helped me figure things out.”
In another story, Spiegel tells the tale of a woman in visceral pain who was unresponsive to both morphine, a powerful analgesic, and ketamine, a powerful anesthetic. Within 10 minutes of trying VR, claims Spiegel, she reported zero pain, saying: “I’m ready to go home, as long as I can bring this thing with me.”
Now seems a good time to explain how I ended up snorkeling in the Atlantic Ocean, floating and staring down into the ocean, with its abysmal lack of depth cues, at the Necker Cube–like fish. In an excruciating process similar to molting, I had recently left a research position, my last in academia, and a romantic relationship — one of those compressed ones whose brevity belies the pain of their resection. I had escaped society to the small, coastal country of Belize on rumor that my father once had, too, and while standing on the beach thought more than once about walking into the sea, in search of a new SRD. I was so hungover the morning of the whale shark–spotting boat ride that I arrived an hour late and, within the first hour or so at sea, had also vomited over the side onto a dolphin coming up, graceful and curious, for air. All of us in the boat jumped into the water, the dolphins led us to the whale shark, and the whale shark led us to the memory of having met it and, all told, we swam around cetacean-like in their living room for almost an hour.
The point being that swimming with real dolphins, themselves swimming in real circles like bubbles around a real whale shark in the real ocean, was a divine experience, one of the best R can offer. But it took me years of actual, real talk therapy to achieve a lasting peace.
So what was it, exactly, about those virtual dolphins?
I found it intriguing that Spiegel doesn’t always qualify events as having virtually occurred but instead gives experiential ownership to the patient witnessing the events:
Then, after touring Venice, Italy, and visiting the Yosemite Valley …
Swimming with dolphins was just what she needed …
Next, they travel together to the depths of the ocean …
Off she went in the helicopter …
Perhaps this is all just a manner of speaking, as Spiegel also gives nonfiction the same power: “James Spiegel, the VR bioethicist we met earlier in this chapter…”
Did we really “meet” James (no relation) Spiegel? Did the patients really “go” to Venice or “swim” with dolphins? No. But in a way, yes? What is the meaningful distinction between a real experience versus a fictive one, if much of the brain can’t tell the difference? Is Spiegel making a statement about words, the stories they bring, and the blurring between R and VR, which may have profound consequences for identity, memory, and autobiography?
At one point, Spiegel compares VR headsets to a syringe: “The device itself is unimportant; it’s what passes through the device that matters,” he writes. (Virginia Woolf once also extended the experience-as-gauge metaphor, memory “run[ning] her needle in and out, up and down, hither and thither. We know not what comes next, or what follows after”). Put another way, the linguistic confusion around what counts as having happened highlights one of the long-term sources of actual confusion for VR experiences that are so rich they might be remembered no differently from ones in R.
If memories of VR can be confused with memories of R, then is this true for all kinds of fictions? And, if so, is that the point?
In 2016, during a panel discussion at the Sundance Film Festival, the screenwriter and director Charlie Kaufman said that if he were ever to make a VR film, the viewer would be able to walk up to its characters, rip open their skulls, and see inside their heads.
Kaufman’s desire to peer into another’s thoughts is an evolutionary, therapeutic, and artistic goal, deriving in part from the brain’s built-in escape mechanisms — empathy, theory of mind, and imagination — being limited, slow, and low resolution. But as fentanyl differs from heroin which differs from morphine only in their relative abilities to cross the blood-brain barrier, can the same be true for certain kinds of storytelling?
More than a philosophical curio, the question of potency is relevant to VR’s uses for treating trauma and mental distress. In one case, a man with schizophrenia and his therapist attempted to give a virtual body to the man’s inner hallucinations while the therapist, sitting in another room, also in VR, voiced the demon; in another, patients with social anxiety interacted in virtual worlds filled with avatars, their social alienation from the virtual population modified as if on a dial, like the volume levels on an amplifier, to help them read social cues; in another, an Austrian-accented Freud look-alike parroted a patient’s own words back to them from across a virtual desk; in “reminiscence therapy,” seniors with dementia were taken on tours of their old homes, 3D-generated from Google Maps, with their friends.
When I passed Kaufman’s comment by Blakey Vermeule, a professor of literature at Stanford and author of the book Why Do We Care about Literary Characters?, she responded like a neurosurgeon might. “If you could rip open heads in virtual reality all you would see is the dura mater. All you’d see is the brain,” said Vermuele. (The “dura mater” — translated in both Latin and Freudian as “tough mother” — is a thin piece of protective tissue between the skull and brain.)
Paul Bloom, a professor of psychology and cognitive science at Yale, answered less literally that the fidelity of VR might inveigle itself into the mind enough that it could, one day, interfere with the details of autobiographical recall: “There are people who have confessed to committing war crimes in Vietnam, and when you look into it you realize that, during the war, they actually were not sent overseas — they just saw a lot of movies and read a lot of things. If we are that pliable to stories and testimony, imagine what it would be if you actually have the clear sensory memory of VR,” said Bloom.
In other words, memories experienced as a kind of virtual tour through one’s own past can, like software or fiction, be rewritten. In a 2009 study, researchers at Stanford created 3D avatars of children of various ages and then had them watch, in VR, their own avatars swim with orcas, the largest member of the dolphin family — but only after being told, falsely, that they had actually swum with the orcas two years prior by presenting each kid with a personalized, algorithmic story:
[Child’s name], when I was talking with your mommy/daddy earlier, he/she told me that when you were [child’s age minus 2 years] years old you swam with two big fish named Fudgy and Buddy. Fudgy and Buddy were black and white and very nice fish. They liked swimming with you in the blue water. You liked swimming with them too. The water was blue with some small waves. There were green plants at the bottom of the water. After you swam with Fudgy and Buddy for a while you dried off and went back home to play. That is what your mommy/daddy remembers. Do you remember swimming with Fudgy and Buddy?
Some of the kids were told to wait and do nothing; some were told to visually imagine swimming with Fudgy and Buddy; some put on VR goggles and watched avatars of other kids swim with the orcas; some put on VR goggles and watched their own avatars swim with the orcas.
Five days later, the kids were asked what they remembered, and those of a certain age who had either imagined swimming with the orcas or watched their own avatars were more likely to have a false memory of swimming with them. Remarkably, those who watched other avatars swim with Fudgy and Buddy did not show the same effect. A sense of self, or at least the ability to “throw” its borders, like a ventriloquist does their voice, seems to be necessary.
One of Virtual Virtual Reality’s endings is an infinite loop, where no matter how many times you take off the virtual goggles you remain in the same spot. It reminded me of one of the endings to Julio Cortázar’s Hopscotch, which breaks the passive immersion of most literature by telling its reader upfront that its chapters can be read out of linear order, if they so choose; and that at the end of each chapter is a number corresponding to the next chapter in an alternate path through the story, which requires the reader to flip back and forth so many times until, at the end, like closure to what had been until then a hidden infidelity, the two final chapters lead only to each other.
Ernest Cline, author of Ready Player One — a dystopian, sci-fi novel about a VR-centered world, adapted to film by Steven Spielberg — told me that he wonders about its long-term effects: “We won’t know how the advent of VR will affect this generation for 10 or 20 years.”
I asked Cline whether seeing, for example, a painting in VR counts as “seeing” it, in the high art sense. “A purist will probably say, ‘No,’” said Cline, “seeing a painting in VR still isn’t the same as seeing it in reality, even if it’s a high-resolution scan, and you can examine it more closely than you could in real life. The same probably goes for the Grand Canyon or Niagara Falls. Seeing it in VR will never be the same as physically being there. But I also think that as time goes on, people might actually qualify their answers. ‘No, I’ve never seen the Mona Lisa IRL (in Real Life), but I have seen it IVR (in Virtual Reality).’”
Never being able to finish, in their alternate paths, either V-VR or Hopscotch, honors the link between story and R’s physicality, though not every experience translates well from R to VR. The pleasure of a well-struck golf ball, for example, is in the skillful domestication of chaos, where invisible differences in unprogrammable initial conditions lead to, in all but the best, different outcomes. Likewise, VR will never match the physicality of real wind, which wicks away real tears; of a real black hole, from which nothing survives; of real dice, whose randomness can be trusted; of real pinball, which is as much a fight against gravity, Newton, and entropy as it is against a high score.
Some things, after all, like reproductive sex, drowning, Scotland, and auditory hallucination, must be done in person.