JUNE 25, 2018
VIRTUAL REALITY HAS FOUND its place in the sun. Newspapers now keep tallies of presidential lies, bots drive social media, and the logics of reality TV rule the nation. “Reality” is so contested that its alternatives have become plausible while remaining elusive. Virtual reality is no exception. A heavily marketed alternative reality, VR evokes much while specifying little, referring variously to a bundle of quite different technologies or the latest must-have media gadget. VR also functions as an epistemological assertion, and as an antidote to the “continuous partial attention” that has displaced distraction as the characteristic of our time. It promises the ultimate in realistic entertainment — and a contested set of psychological effects. What, precisely, is VR? Jaron Lanier, the VR guru whose VPL Research created the sector’s first commercial products, offers no fewer than 52 definitions in his book Dawn of the New Everything.
VR is thus something of an empty signifier, meaning different things to different people.
For some, it includes the 360-degree videos published by The New York Times that “simulate richly immersive scenes from across the globe.” Accessible, technologically stable, easy to distribute, and even DIY, this end of the VR spectrum takes the form of fixed video assets, panoramas within which the viewer can look but not touch. Like Dante’s travels with Virgil, the viewer floats through a strange world, witnessing without being witnessed.
For others, VR entails possibilities for movement and gestural interaction. Mid-tier systems such as the Oculus Rift, PlayStation VR, and HTC Vive, together with higher-end proprietary systems, require significant processing power, are often physically tethered to computers, and are more complicated to produce. Echoing the trajectory of earlier media forms, the big money is on “read-only” professionally produced experiences rather than VR as a means of everyday self-expression. But that said, this is also the sector that displays the state of the art to full advantage, as evidenced by projects such as Arnaud Colinart and team’s Notes on Blindness and Karim Ben Khelifa’s The Enemy.
The technologies behind VR are changing fast — not surprising in a sector shackled to a law as exponential as Moore’s. And these changing technologies only complicate the definitional problem, whether by splitting the difference between the ends of the just-described spectrum, as did the recently released mass-market Oculus Go, or by engaging next-generation eye-tracking and EEG navigational interfaces, complete with new possibilities for data-harvesting based not just on where we look, but also on our brain activity, pulse, and galvanic skin response. Regardless of which sector is of interest, image density and frame rates will increase, cables and the physical scale of the apparatus will decrease, and the long-promised experience of reality-in-all-but-implication will remain virtually, elusively, there … a carrot dangling just outside of our reach.
If technology offers a slippery definitional slope, might we pin down VR by considering its claims? In a world of ubiquitous images, the entertainment and advertising industries have to struggle to engage and monetize our attention. They have to lure us away from second and third screens by bringing us back to the primal screen. Immersion, the premise of VR, seems like a perfect vehicle for achieving those goals.
If, for a moment, we turn our backs on the future and the imminence of the next big thing and instead look to the past, we might see something rather surprising: precedent.
Lanier’s wonderfully autobiographical book, which tracks his involvement with, among other things, VR from the 1960s through the early 1990s, powerfully portrays the technological imagination. Exploring some of the same terrain mapped in Fred Turner’s From Counterculture to Cyberculture, he traces his own idiosyncratic encounters with that cultural moment’s technologically enabled vision. To be sure, the book is self-indulgent in places, but that hardly distinguishes it from other books of its kind. This book is actually compelling. It is rooted in an eclectic mélange of philosophy, code, and some of the late 20th century’s defining characters, and it engagingly teases out the possibilities of VR before the marketing department hammered it into a gadget.
The existing literature, by contrast, tends toward either marketing hyperbole or carefully circumscribed scholarship. Perhaps most importantly, Lanier’s book serves as a forceful reminder that earlier generations of media technologists and publics have long sought the Holy Grail of immersion. In the 1960s, for instance, efforts to reach it included Morton Heilig’s sound-, scent-, and motion-based Sensorama system (1962), Hugo Gernsback’s 3-D television goggles (1963), and Ivan Sutherland’s head-mounted display immortalized as “The Sword of Damocles” (1968). But immersion’s story is much older still.
The apocryphal reception accorded Auguste and Louis Lumière’s L’Arrivée d’un train en gare de La Ciotat, first shown about a month after the cinématographe’s December 1895 public premiere, reverberated internationally in cartoons and even other films at least until 1910. Images of patrons diving under seats or fighting back at the screen buttressed the myth of cinema as “lebende Bilder,” “actualité,” and the Vitagraph. Invariably, the sensational aspect was relatively short-lived. So, too, was the buzz around the mid-19th-century stereograph, the photographic cousin of today’s Google Cardboard and a mass medium in its own right. Millions of views were sold to users in search of immersive armchair travel. Early 20th-century dioramas and even earlier 18th-century panoramas and phantasmagoria (smoke and mirrors!) all carried on the good work of immersing their audiences into worlds both factual and fictional. Robert Barker spoke for generations of media technologists when he ended his 1787 patent for the 360-degree La nature à coup d’ œil (“Nature at a glance,” later called the panorama) with a declaration of his intent: “[T]o make observers […] feel as if really on the very spot.”
Surviving historical reactions to these immersive technologies are like previews of the exuberant user responses that adorn today’s VR promotions. Enthusiasm decayed quickly, however, which surely should serve as a reminder that people acclimate to the immersive, normalizing it, and raising the bar for the next immersive sensation. Rather than being dazzled by its illusions, anyone who works in VR, no matter how advanced the system, confronts the system’s limits and imperfections. The power of Lanier’s telling of the tale derives precisely from how he eschews the sensational and attends rather to the human and perceptual drives behind age-old desires and attendant buzz.
VR is encrusted with claims that draw from and go far beyond its mimetic visual character. Chris Milk, award-winning VR director and the entrepreneur behind WITHIN (formerly Vrse), calls it an “empathy machine.” Peter Rubin, senior editor at Wired and author of Future Presence, calls it an “intimacy engine.” And Jeremy Bailenson, professor of communications at Stanford, founding director of the Virtual Human Interaction Lab, and author of Experience on Demand researches such topics as “empathy at scale.” Brain and cognitive scientists armed with MRI scans tell us that exposures to film and photos activate one part of the brain, while VR activates another. The difference suggests a distinction between “representation” and “experience.” And if we cognitively process VR as experience, then claims like these and others might well find empirical support.
The research, however, is still very much in progress. Bailenson’s book charts the work of his lab as it researches the potential use of VR for PTSD treatment, pain mitigation, sport performance enhancement, understanding the basic mechanics of human empathy, and more. His lab’s methods go far beyond dependence on MRIs, and readers in search of an overview of applied VR will be well served by his book. It is ambitious and wide-ranging in scope, but his claims are invariably nuanced by the tentative state of research, and the knowledge that correlation, after all, is not causation.
This said, the wider claims for VR’s ability to create change, be it empathetically mediated or otherwise, echo claims going back to Plato. There are two camps: those who believe that media do things to people, and those who believe that people do things with media. The former camp has a long history of grant support and wishful thinking, particularly by advertisers and state propagandists, even though most scholars have long ago rejected its most deterministic claims. Still, it is surely telling that the very media industries that deny that violent depictions create violent behaviors are the first to sell advertising because it works. Terms like “engine” and “machine” suggest that VR has returned a crudely mechanistic notion of media effects to the debate. Not surprisingly, its mechanistic assertions tend to be overstated. Still, it would be foolish not to explore how people interact with the medium — by, for instance, calibrating the specificities of that interaction, and so learning how we might make better use of them.
Bailenson’s arguments, and, to a lesser extent, Rubin’s, may also be read as evidence of a recurrent cultural habit: using media technologies to model the workings of the human mind. Phrases like “photographic memory,” behavioral “tapes,” and “overloading” our brains with information point to how we metaphorically lean on the medium du jour. Today’s ascendant metaphor offers models for how our minds interact with the world. Like any good metaphor, it enables us to see certain relationships more clearly. It offers heuristic value as we interrogate human behaviors, and potential insights. The trick is not to take the metaphor literally, and it is a trick that as we as a society are still mastering.
Remember 3-D television? A decade ago, consumer-electronics trade shows and magazines were filled with anticipation regarding the next big thing in television. The hype is familiar, but the quiet passing of 3DTV is far more interesting. Sure, one could fault its reliance on glasses, or shifting and incompatible technological standards, or incidents of motion sickness, or uneven program content. But ultimately, the experience did not live up to the hype, and therein lies a lesson for VR.
Optimistic though I am about VR, today’s systems risk a fate similar to 3DTV — and not just because VR shares the same litany of complaints. To be clear, the larger suite of technologies and tactics of which VR is a part, “XR” (Extended Reality, including Augmented Reality and Mixed Reality), will doubtless continue to morph and find deployments in both niche and mass markets. But the uncertainty regarding what VR is, even at the most basic experiential and technological levels, combined with the many overblown and conflicting claims regarding its operations, can easily lead to disappointment.
Peter Rubin boldly declares:
VR is poised to upend every industry and transform every aspect of our lives. It will change how we work, how we experience entertainment, how we feel pleasure and other emotions, how we see ourselves, and, most important, how we relate to each other in the real world.
Try that with your Google Cardboard! Subtitled “How Virtual Reality Is Changing Human Connection, Intimacy, and the Limits of Ordinary Life,” Rubin’s Future Presence posits VR as a game-changer. Enthusiasm is an essential part of promotion, and there is much to be enthusiastic about in the case of VR. But in a culture with a short attention span, not to mention an uncertain sense of the topic at hand, one must also tread carefully lest the unintended consequence that befell 3DTV repeat itself. Part forecast, part scholarship, part hyperbolic industry discourse, Rubin’s book will make for interesting reading in a decade, once the dust has settled.
Okay, the industry’s penchant for hype going on humbug needs to be moderated lest it overwhelm technological possibility. But VR faces three additional existential threats that need to be considered as the field develops.
The first regards our cultural tendency to put old wine in new bottles. In VR’s case, this means shackling the medium’s immersive potentials to the task of storytelling. VR is much better suited for environmental exploration, for storyfinding, than for the more constrained notion of storytelling. It builds on the experiential narratives enabled by some game genres far more effectively than on the narrative structures inherited from cinema and literature.
The second regards VR’s greatest competition: our nightly dreams. They offer compelling, completely immersive, and pixel-free virtual worlds. What would happen if we approached virtual reality the “other way around”? In other words: Rather than focusing on the cognitive processes triggered by strapping a phone over our eyes, what would happen if we learned to shape and program our dream state? This is an invidious comparison, of course (although researchers are hard at work on it!). But it suggests that hardware solutions may be inherently limited, and that we need to take human wetware approaches more seriously.
The third threat involves the inherent isolation that currently defines VR. Isolation has distinct limits. Social VR, by contrast, shows that this still-nascent medium may well have the capacity to connect us with one another, rather than trap us in our own worlds. And in so doing, it may open VR to an entirely new set of possibilities in speculative world-building, while acknowledging the intrinsically social dimension of human reality.
The good news is that a strong investment climate, significant technological advance, and a “real” world that breeds escape fantasies may well combine to push VR to the next level. As creative teams draw on talent from the environmental arts, immersive theater, games, and even magic, more compelling entertainment experiences may well ensue. But advances will also be tempered by the shapeshifting that already plagues VR, by the eternal return of bigger and bolder claims, and by the reality effect that eventually descends in the wake of too much humbug.
Jaron Lanier’s 33rd definition of VR hits the nail on the head: “The ultimate media technology, meaning that it is perpetually premature.” Amen.
William Uricchio is professor of Comparative Media Studies at MIT and founder and principal investigator of the Open Documentary Lab. He, together with Sandra Rodriguez, taught MIT’s first course on VR.