The Future of Reality

If data is the “new oil,” then augmented reality is the opportunity to colonize (or perhaps recolonize) the whole wide world.

Writing about the future is dangerous, especially when your subject is the intersection of people and technology. Just a few years ago, for example, experts were sure that there would soon be fleets of self-driving cars swarming our major cities. “Up to a million people a day [could be riding in a self-driving car] every day by 2020,” The Atlantic’s Alexis Madrigal—one of our best tech journalists—wrote in March 2018. Tech industry guru Tim O’Reilly wrote a whole book in 2017 called WTF? What’s the Future and Why It’s Up to Us, offering a trenchant guide to how artificial intelligence and other emerging technologies were going to shape things, and never even mentioned the words “diversity” or “racism” as issues that would transform the industry. And experts like Dr. Anthony Fauci have turned out to be wrong about their expectation last spring that it would take at least 12 to 18 months, if not longer, for science to develop protective vaccines against coronavirus that could be deployed en masse.

Humility about the future doesn’t get as much attention as bold predictions, and in today’s hyper-powered “attention economy” there’s even less incentive to be cautious. But having hung around this world for a few decades, I’ve learned that the people getting the most attention for their ideas are often just good at getting attention. Tech and politics are both fields rife with wrong guesses. Remember Francis Fukuyama’s book, The End of History? How’d that work out? People, both as individuals and as larger agglomerations, are unpredictable. Will the new union of Alphabet workers take off? I don’t know. It’s hard to organize unions anywhere today in America, let alone in the tech industry. But maybe this effort will be different. Will there be a serious accounting of the Trump Administration’s abuses of power and corruption? My political journalist friends all scoff at the idea, but who knows, maybe after the Georgia runoff Tuesday, the Electoral College confirmation Wednesday (inshallah, as President-elect Biden might say) and then the inauguration, the public will demand a true house-cleaning. Maybe there will be a giant leak from inside the Trump White House that will change the dynamics of this issue. Maybe not. (More on that topic later this week.)

I write all of this both as a meditation on the New Year, and also to introduce one of my favorite futurists, Mark Pesce, who has a must-read new book out called Augmented Reality. I’ve been a friend of Pesce’s since 2008, when I invited him to shlep all the way from his adopted home of Australia to speak in NYC at Personal Democracy Forum on what he called “hyperpolitics.” It was a seminal speech, especially at a moment when many of us were dazzled by how being “hyper-networked” was transforming power politics, enabling a relative newcomer named Barack Obama to overtake a longtime political insider named Hillary Clinton in that year’s Democratic presidential contest. Instead of celebrating, Pesce warned the audience that hyperpolitics wouldn’t be anything like democracy, bur rather something more like Hobbes’ “war of all against all.” He said, “A hyperconnected polity – whether composed of a hundred individuals or a hundred thousand – has resources at its disposal which exponentially amplify its capabilities. Hyperconnectivity begets hyper-mimesis begets hyper-empowerment. After the arms race comes the war. Conserved across nearly four thousand generations, the social fabric will warp and convulse as various polities actualize their hyper-empowerment in the cultural equivalent of nuclear exchanges.”

A year later, we had him come back and he gave another brilliant keynote, using the battle then underway between the Church of Scientology and Wikipedia (over the former’s Wikipedia page) as a harbinger of the kinds of battles over the truth that we now live with everyday. I recall that his 2009 talk was the first time I heard of Anonymous (which had decided to go to war with Scientology partially for the lulz) and I’ve valued Pesce’s contributions since then. So while we should still be skeptical of anyone’s predictions about the future, Pesce’s ability to see the bigger picture and patterns with the right degree of focus make me trust him more than the average industry pundit.

Augmented Reality, which at a brief 148 pages is a very comfortable read, is Pesce’s clarion call. Like Shoshana Zuboff, he is a trenchant critic of surveillance capitalism, the model by which companies like Google and Facebook have gotten fabulously rich and powerful by mining our data. He also understands why we humans have fallen in so in love with these sensuous objects that we carry with us all day long. “Within a decade of its introduction, nearly half of all people alive – and a clear majority of those over 18 years old – owned a smartphone. No other tool has ever charted a path from invention to ubiquity in such a compressed timespan.” Half of us sleep with our smartphones next to us at night, he notes.

But Pesce is not entirely thrilled by this change. “In the second half of the 2010s, the human race disappeared behind screens,” he writes, even before the pandemic accelerated this transformation. And when we emerge from the pandemic, a new changed reality awaits us, one that all of the biggest tech companies are currently investing billions of dollars to control. That is the prospect of augmented reality, a world where the sensuous computer now in our pockets and hands has migrated to our faces as smart glasses or “mirrorshades” and lets us “see” the world with hyper-powered eyes.

This future, to be clear, is not the narrow one offered by Google Glass, which simply offered a minicomputer in a sidebar view that was mostly used by “glassholes” to surreptitiously video the people they were with, or the narrow-frame of reference currently available with a Microsoft Hololens, which only covers about 40 degrees of a human’s natural eye-span, but a fully functional real-time overlay on the whole world we see. And to make that kind of data stream possible, AR needs three things: it needs to know what we are gazing at, it needs to know where we are physically, and it needs an up-to-date map of the physical world surrounding us. Obtaining and controlling that data is the holy grail of today’s surveillance capitalists, Pesce warns. The market for our “gaze data” alone, he predicts, will be enormous.

“How much would we pay for reality? How much should we be paid to let others drive our view of the real?”

There is tremendous power already in the ability to know what we are paying attention to. Facebook discovered several years ago that it could manipulate its users’ emotions by tweaking the content of their News Feeds. Soon enough, it was marketing its ability to advertisers who might be interested in knowing when teens were most emotionally vulnerable. And there is tremendous power in enabling us to connect to each other. Tech evangelists like Facebook’s Mark Zuckerberg think they are changing the world for the better by making it so easy to “find the others,” but Pesce eviscerates that naïve idea in just a few sentences, noting that when we connect around shared beliefs those beliefs get reinforced, and our innate tendency to repress “shadow” or heterodox feelings within ourselves causes us to hate others who embody differences. “Connection has always carried with it the menace of mob rule,” Pesce notes.

Now fast-forward slightly to a world where the ability to augment reality, to ascribe meaning to places, becomes common. Pesce makes much of 2016 rollout of Pokemon Go, a smartphone-based game that sent thousands of players out into the real world in search of virtual rewards, causing overcrowding and near-riots in some places with hard-to-find Pokemon, and demonstrating the lucrative potential of AR. But not only did business owners discover that they could bring players to their stores by paying to place Poke-lures in them, Niantic, the Google spin-off behind Pokemon Go, discovered it could also get players to map the world for free, while they played. If data is the “new oil,” as some tech pundits have been saying for a while, then augmented reality is the opportunity to colonize (or perhaps recolonize) the whole wide world.

Unlike Zuckerberg, whose zeal for global domination – a word he literally used to chant at the conclusions early company weekly meetings -- is matched by his uneducated faith in the goodness of human nature, Pesce worries about when it becomes as easy to augment reality as it is to “friend” someone or “like” a post. In addition to the positive content that may be added, he warns that bad actors wanting to mark up their enemies will have a field day, and the Big Tech purveyors of these new platforms will be as overwhelmed by that problem as they are today by the toxic content that floods their sites.

There is a silver lining, though. Unlike the first decades of the Internet and the web, where the public and governments alike were gulled into giving our tech overlords the freedom to take dominion over us, we are in changed times. We have tasted the Apple of knowledge and also now know what sin is. Pesce offers a vision of a different kind of future, the “potential of a universal, revelatory informational transparency, each object illuminated from within by its digitally inscribed meanings.” The potential revelation of hitherto hidden facts or meanings can redistribute power.

But the question, he writes, is who will control the augmentation of reality. If it is the big tech companies, what happens when the power to ascribe is sold to the highest bidder? “How much would we pay for reality? How much should we be paid to let others drive our view of the real?” Pesce asks. These are not fantastical questions. “Facebook already has the capacity – demonstrated within its smartphone app – to generate a reality so engaging that users find it difficult to look away from it,” he writes. “The same will be true for the users of its AR products. Those users will be inserted into another reality, one of Facebook’s making, based on their carefully analyzed profile data.” And, he could have added, warped by the needs of Facebook’s actual customers, its paying advertisers.

We are going to need new forms of governance if we want to master this brave new world and not be mastered by it. Pesce proposes a new kind of Domain Name System, a hypothetical “Internet Corporation for the Assignment of Locative Metadata” (or ICALM) that would vest the power to ascribe locations in the hands of private individuals and entities that now own or control their physical spaces, and he rightly says this is a political issue, not a technological one. “We cannot simply rely on the good graces of businesses busily seeking to monetize space. Instead we must act together, through a mix of governmental, legal, regulatory and standardization mechanisms. Only when all of these are engaged in concert – regionally, nationally, and internationally – do they present enough of a barrier to the interests of multiple trillion-dollar entities.” This is a tall order, but still one we can meet in these early days of the 21st century.

To do so, we are going to have to add a slew of battles to the ones we are already fighting. We are going to have to file and back lawsuits challenging companies when they casually plop digital installations on public spaces, as when Snapchat placed a Jeff Koons sculpture digitally in Central Park without asking anyone’s permission, and artists, seeing the future ahead, protested. We’re going to have to question the so-called privacy settings of our appliances and not take for granted that our Roomba cleaning robot should be able to get on our home Wi-Fi so it can upload its map of our house’s floorplan back to iRobot company servers. We’re going to have to stop facial recognition tools in their tracks and support cutting-edge groups like the Algorithmic Justice League as they pioneer that work. We’re going to have to go further, and stop products like Amazon’s Ring from creating a civic-trust-undermining home surveillance network tied into police authorities, by passing new local ordnances regulating what video homeowners can collect and how they can safely use it. In other words, we are going to have to learn that our bodies, our places, our thoughts and our actions generate valuable data that others are greedily planning to colonize, and defend ourselves.

As I’ve been reminding readers, my goal is to keep The Connector free to all. To help me do that, please consider signing up as a subscriber. A bunch of you have done so already, which I really appreciate. And Happy New Year!