The Machine Stops: Science and Its Limits

February 22, 2021   •   By Henry M. Cowles

The Knowledge Machine

Michael Strevens

“IMAGINE, IF YOU CAN, a small room, hexagonal in shape, like the cell of a bee.” So begins a story by E. M. Forster from over a century ago, about a future that feels uncannily like our present. Called “The Machine Stops,” the story conjures a world of individual humans isolated in such rooms who pass the time by streaming lectures and videoconferencing as the titular Machine tends to their every need. It is a world in which everything — music, food, even your bed — is summoned by the click of a button. In place of work, the Machine offers material and intellectual sustenance. For the story’s main character, Vashti, it turns life into a reassuringly predictable cycle: “[S]he made the room dark and slept; she awoke and made the room light; she ate and exchanged ideas with her friends, and listened to music and attended lectures; she made the room dark and slept. Above her, beneath her, and around her, the Machine hummed eternally.” To Vashti, the Machine offers serenity and certainty in equal measure. In this imagined world of isolated togetherness, heaven is a place.


Until it isn’t. Cracks appear, first as heresies whispered by Vashti’s son Kuno — in person, since they cannot be shared through the Machine. “Cannot you see,” he asks, “that it is we that are dying, and that down here the only thing that really lives is the Machine? We created the Machine, to do our will, but we cannot make it do our will now.” Gradually, the cracks widen: in place of music, jarring noises echo; at night, beds fail to appear. Eventually, Kuno delivers a cryptic prophecy: “The Machine stops.” At first, Vashti finds this unintelligible. The Machine, omnipresent and omniscient, is reality, if not God. Units of its currency — ideas — are disseminated via lecture and shared via chat; like cryptocurrencies, their value is relational rather than a proxy for some relationship to reality. So when the Machine stops, proving Kuno right, it is not just the loss of material comfort that horrifies Vashti and the others. It is the loss of meaning, even of truth, as the ideas stop circulating.


If this feels too close to home, you’re not alone. Forster’s eerie prescience isn’t even new. Readers were returning to “The Machine Stops” long before COVID-19 and Zoom turned so many of us into supine hermits binging media pre-selected by artificial intelligence. What’s the best analogue for the Machine in today’s world? Some say the internet, others say social media or digital surveillance. Each captures the uncomprehending dependence at the center of Forster’s story. But these analogies ultimately fail: each of today’s technologies is criticized, whereas Forster’s Machine never was — until Kuno. Our equivalent must be beyond reproach, at least in polite company; it must shape daily life via hidden processes shrouded in myth. Where is our Machine hiding?


¤


In the laboratory. Or rather, in the journal articles that flow from laboratory work. These are the outputs of what the philosopher of science Michael Strevens, a professor at New York University, calls “the knowledge machine.” Strevens’s Machine, the engine of modern science, runs on a single principle, a kind of speech code that he dubs “the iron rule of explanation.” This rule doesn’t limit what or even how scientists think but how they argue: “[I]t directs scientists to resolve their differences of opinion by conducting empirical tests rather than by shouting or fighting or philosophizing or moralizing or marrying or calling on a higher power.” According to Strevens, the iron rule is the difference between science and non-science, and its introduction in the 17th century drove what is often called the Scientific Revolution. With the rule in place, the knowledge machine creaked into motion — and hasn’t stopped.


Strevens’s argument is breathtaking in its simplicity: control how people argue and you don’t just get better arguments, more agreement, and less bickering — you get the truth, or something close to it. The Knowledge Machine thus addresses two foundational questions in the history and philosophy of science — “Why is science so powerful?” and “Why did it take so long to arrive?” — with one answer: the iron rule. The rule sidelined centuries of debate about higher, often religious questions in scientific communities by narrowing the field of acceptable evidence. And, by applying only to public statements, it did not run afoul of the unavoidable biases of private thought. As long as scientists keep making observations that can — at least in principle — be contested, then they’ll keep producing interesting ideas and technological marvels. The knowledge machine keeps humming.


Until it doesn’t. But how will we be able to tell when it stops? Forster offers a clue. Like the Machine, modern science is both essential and obscure. It is all around us yet difficult to see, let alone explain. From reliance on smartphones to hope for a vaccine, we assume science works; when it doesn’t, we blame the operators, not the machine. This combination of power and opacity both inspired Vashti’s devotion and hid the spanner in the works. Could the same be true for us? Is science’s supposed insulation from society responsible for its triumphs and its travails, including not only organized denialism and declining public trust but also crises of replication and inclusivity? If so, getting a grip on the machine should help us troubleshoot the jarring noises it is emitting.


¤


The Knowledge Machine is not just an argument about modern science. Along the way, Strevens provides a pocket history of the field of science studies, starting with a comparison of two of its figureheads: Karl Popper and Thomas Kuhn. As in the parable of the blind men and the elephant, Popper and Kuhn, he argues, have identified two aspects of modern science but have failed to grasp the machine itself. Popper identified their skepticism, and Kuhn their allegiance to a paradigm. Both were onto something, Strevens argues, especially in their recognition of the degree to which scientific thought is alienating, sometimes brutally so: it is just plain hard to be so skeptical, or to adhere to Kuhn’s “normal science.” But what each failed to grasp, according to Strevens, is why anyone would spend their lives in a laboratory, desperately pursuing these alien lines of thought.


Enter Strevens and his machine. The problem with Popper and Kuhn, he says, was that they were looking in the wrong place: the minds of scientists. While Strevens is interested in what a scientist thinks, he doesn’t view their individual thoughts as explaining why science is so successful — or so recent. Rather, it is despite what goes on in scientists’ heads — despite their hopes and dreams, biases and guesses — that the knowledge machine hums along. Like Popper and Kuhn, Strevens is a “methodist,” a believer in rules. Unlike them, he thinks science’s rules limit writing, not thinking. A scientist can think whatever she wants — so long as she adheres to the iron rule when publishing.


Strevens (mostly) does the same, offering up historical examples of scientists checking their biases at the proverbial door. (The exceptions, like a lengthy retelling of Romeo and Juliet as a debate over the caloric theory of heat, can presumably be chalked up to his being a philosopher.) So we get the testing of general relativity by an avowed partisan, Arthur Eddington, as a case of the iron rule in action. For reasons explored by Strevens’s colleague, the historian Matthew Stanley, Eddington really wanted Einstein to be right but never said so, at least not in print. Why? Because of the iron rule, which requires that “the grounds of many of the experimenter’s crucial assumptions, being partially or wholly subjective, are cut away.” Would papers be more complete if those assumptions and other human dimensions were included? Yes. But would they be scientific? For Strevens, the answer is no.


Readers may wonder why useful information, such as spelling out a researcher’s desires or hidden bias, has no place in scientific publications. The answer, to paraphrase the Mandalorian, is that this is the way. This puzzling austerity explains both Strevens’s metaphor (machines don’t care if we find them intuitive) as well as his subtitle: “How Irrationality Created Modern Science.” What makes the machine irrational is that it would make a lot of sense for scientists to share their hopes and biases (beyond a disclosure of conflicts of interest) — but doing so would gum up its gears. The machine and anything we put into it must be “sterilized,” as Strevens puts it. This isn’t a defense of irrationality — nor is it a condemnation. The Knowledge Machine is neutral, a description of what’s in the box rather than a prescription for what to do with it. And what a brilliant description it is!


¤


But sometimes, description is not enough. Certain moments call for prescription, ethics, normative arguments, or aesthetic distinctions — iron rule be damned. Strevens is not a scientist, and thus does not have to follow the rule himself. But strangely, he does. The book mirrors its subject: rather than intervene, or celebrate, or worry, it offers explanatory data. It describes a rule that it also follows. Thus, even at the end of the book, in a final chapter on “Care and Maintenance of the Knowledge Machine,” Strevens seems loath to cross over from is to ought. Noting the existential threat posed by climate change, he limits himself to the claim that the knowledge machine is “our best chance of salvation” before offering tips for “ensuring that it purrs — efficient, responsive, dynamic, and strong.” Once again, the book is more a user manual than a sermon.


But we need sermons now. The dangers are real, and not limited to climate change. The knowledge machine itself is in trouble, facing at least two kinds of threat. The first is a familiar exogenous threat: denial or distrust aimed at the knowledge machine even when it seems to be running perfectly. We could call these Kuno Problems, since — at least early on — critiques of the Machine were not about its failure to provide ideas or food but about the pervasive, negative effects of such provision. Assuming that the misinformation peddled by merchants of doubt is just that — misinformation — then their assault on science has less to do with the machine than with the politics of expertise in an age of alternative facts. Solutions to Kuno Problems — to the manufacture of ignorance — aren’t going to come from tinkering with the knowledge machine, since these problems are less about how it works and more about its effects, or its limits.


The second kind of threat is entirely different. Endogenous to the machine, these threats are signs of systemic failure. Call them Vashti Problems, since even true believers can’t ignore them for long. Some of science’s problems (with reproducibility, say, or with biases that affect who is or isn’t seen as a researcher) are with the application of the iron rule — or with the iron rule itself. What if science actually suffers because, in publishing it, we omit what motivates us to pursue it? Might the sterilization Strevens identifies be part of the problem, even as it accounts for so many solutions? Answering these questions requires a hard look at the machine itself, either to diagnose and address the issue or to ask whether the knowledge machine ever worked how we thought it did. Either way — whether we’ve got Kuno Problems or Vashti Problems, whether the knowledge machine needs a tune-up or the metaphor needs the scrap heap — there’s enough at stake to reconsider, if not reimagine, how we talk about science.


¤


For starters: Does science produce knowledge? It’s a compelling idea, especially in the face of organized denial and post-truth cynicism. Doubling down on knowledge, not to mention truth, is attractive precisely because knowledge feels solid, even tangible — the kind of thing you could produce with a machine. But this material metaphor, embedded in phrases like “knowledge production” and “social construction,” should give us pause. While the roots of “knowledge production” run deep, it gained popularity through the work of the economist Fritz Machlup. In The Production and Distribution of Knowledge in Society (published in 1962, the same year as Kuhn’s work on paradigms), Machlup cast knowledge in terms of production and consumption, circulation and extraction. Like Machlup’s idea of “the knowledge economy,” this framing blurred the process of inquiry with its products, including the papers spit out by Strevens’s machine. Such metaphors, figuring knowledge as a tradable good to be produced and circulated, bolster our sense that science is solid — a stable thing that buttresses reasoned debate.


But what if it isn’t? What happens when the foundation wobbles, when the machine makes jarring noises or starts getting second-guessed — or ignored? If recent debates are any indication, this is the world in which we find ourselves, and our metaphors of solidity aren’t capturing what is ailing scientific experts. In an age of crises — epidemiological and climatic, political and cultural — the crisis of expertise may be the most urgent, not least because its resolution will affect the others. The knowledge machine offers us one path forward, explaining how science works and how to care for it; but to explore other paths, new metaphors might be needed. If science isn’t to be captive to the market, to the up-votes and down-votes of capitalism or social media, then we will need to rethink whether it is a machine for producing a tradable good. After all, the machine stops.


¤


Henry M. Cowles is a historian of science and medicine based at the University of Michigan. He is the author of The Scientific Method: An Evolution of Thinking from Darwin to Dewey (Harvard University Press, 2020).