Is There Anything “Artificial” About Artificial Intelligence?

Julien Crockett speaks with Blaise Agüera y Arcas about the various ways that LLMs keep surprising scientists and how our definition of intelligence should be more complex than people generally think.

By Julien CrockettDecember 1, 2025

• Science & Technology

This interview is part of The Rules We Live By, a series devoted to asking what it means to be a human living by an ever-evolving set of rules. The series is made up of conversations with those who dictate, think deeply about, and seek to bend or break the rules we live by.

¤

“AT LEAST AS OF this writing,” Blaise Agüera y Arcas begins his new book What Is Intelligence? Lessons from AI About Evolution, Computing, and Minds, “few mainstream authors claim that AI is ‘real’ intelligence. I do.” Gauntlet thrown, Agüera y Arcas lays out his thesis, which is simple—in the way that profound remarks or universal theories can be—yet with enormous implications: because the substrate for intelligence is computation, all it takes to create intelligence is the “right” code.

What Is Intelligence? is a wide-ranging defense of this argument. Agüera y Arcas takes us from the emergence of life to Paradigms of Intelligence, his research group at Google, where he studies biologically inspired approaches to computation. Importantly, given the rapid development and deployment of AI today, What Is Intelligence? makes us question what is so “artificial” about artificial intelligence.

In our conversation, we discuss definitions of life and intelligence, cultural attitudes toward AI, whether we should have been surprised by the success of large language models in the early 2020s, and the implications of AI on society.

¤

JULIEN CROCKETT: Let’s start with your definition of life. What is it, and why do you think there are such varied perspectives about what distinguishes life from nonlife?

BLAISE AGÜERA Y ARCAS: I define life as a self-modifying computational state of matter that can grow, heal, and replicate itself heritably. It’s a functional definition, meaning it’s really about what life does, as opposed to what its essence is. It’s also a computational definition. As John von Neumann figured out in the 1940s and ’50s, if you are able to follow an instruction code inside yourself to build more of yourself, or heal yourself, or make another copy of yourself, then that “universal constructor,” as he called it, is literally a computer. I think it’s important to take this functional approach to defining life because it avoids specifying exactly what you’re made out of. It’s the kind of definition that would let us unambiguously identify life on another planet, even if it weren’t based on DNA, proteins, or any of the other familiar ingredients of life on Earth.

But all this is controversial—ask two expert researchers for their definitions of life and you’ll get three answers! I think that’s because, even though we now have a lot of specific biological knowledge, biology as a whole lacks a coherent overarching theory. It’s a little bit like “information” before Claude Shannon’s information theory, or “force” before Isaac Newton gave us F = ma [force equals mass times acceleration]. Some biologists aren’t sure there is such a grand theory for biology, but to me that seems too pessimistic.

Is there not something special about the substance we are made from that allows for life?

I prefer to define life functionally—in terms of what it does—because this approach leaves open the door for life to be realized in multiple ways. And, in my view, function is exactly what defines life. If you look inside the body, what differentiates your organs, for example, from rocks or inanimate material is that if you break a rock in half, you have two rocks; you don’t have a broken rock. Whereas, if you break a kidney in half, you no longer have a working kidney.

Nature also substitutes things for others that are made differently but serve the same function or purpose all the time. For example, take insect wings and bird wings. They are made of completely different materials, but both have the same function of flight. Or if you look inside a cell and think about respiration, the function of respiration is to generate ATP molecules, our body’s “energy currency.” That can be done aerobically or anaerobically, which are two different chemical pathways. In other words, nature realizes things in different ways, and it’s that network of functions that defines life.

The next logical question, then, is where does “function” come from?

That is the million-dollar question, and the one that stumped Charles Darwin. How do you get function out of nothing? For a lot of people thinking about evolution in the 19th century, the answer was God. Even people like Lord Kelvin, who came to believe in evolution, thought the first spark that started life had to have been instilled somehow by something. Purposes can’t come out of something purposeless.

But that is what was so exciting about the experiments in artificial life that my team and I at Google did a couple years ago. We started off with thousands of completely random strings of code that didn’t do anything, but we found that just by waiting and having them interact with one another enough times, function or purpose emerged. And the reason it emerges is that when something develops the purpose of propagating itself, then that will persist in a way that something that does not have purpose won’t. In other words, there is a Darwinian selection for purpose right from the beginning.

A theme in your definition of life, and running throughout the book, is that life is broader and more active than we would understand from the strict materialism model of science developed during the Enlightenment. For example, you write:

When scientists castigate animist beliefs as superstitious, they typically appeal to the materialist discoveries of the Enlightenment, which show that the atoms that make up our bodies are no different from the atoms that make up rocks or air. This is true. Atoms are atoms; they all obey the same rules. […]

Yet as Schrödinger pointed out in 1944, our understanding of these laws—which he played such a central role in developing—remains incomplete. The laws as they stand do not account for the computationally complex, dynamically stable, symbiotic phenomena that comprise so much of our experience on Earth—indeed, without which there would be no such thing as experience at all. There would be no life, or purpose, or minds, or agency.

What are the limitations of the strict materialism model and the implications of the broader, more active view of life you propose?

In the 19th and 20th centuries, science went through a disenchantment. In the 19th century, the idea that there was a vital spirit that animates living things became undone because we learned that the chemistry that we’re made of is the same chemistry that makes up everything else. In the 20th century, there was a “reductive” push—the idea was that, since we understood the fundamental laws of physics, we knew how nature “really worked” at a deeper level, so everything above that could be explained as an epiphenomenon of physics. You still get thinkers like neuroscientist Robert Sapolsky who say that free will is obviously an illusion because everything that you’re going to do is determined by the physics of the particles that make you up.

I see things a bit differently. And it’s not because I’m somehow anti-science or irrational. It’s because when you reduce everything to elementary particles and interactions, you stop explaining all of the complex phenomena that we see. In fact, all the nouns in our language—table and chair and so on—none of that exists at the level of particles! But once you start to think about reality as being a predictive model—this gets more into the neuroscience side of the book—reality is, actually, something that you construct in order to better predict your environment. You’re certainly not predicting it by modeling all of the particles of everything you see and their quantum interactions. You’re predicting it by forming higher-order models. And once you look at those higher-order models, then understanding the behaviors of tables and chairs and what they do or you do with them becomes the right level of description. Physics is no stranger to this. The whole field of thermodynamics, where you model gases and their pressures and temperatures, is exactly the same kind of move. You can’t talk about pressures and temperatures without averaging over ensembles of particles and forgetting about their individual trajectories. There is no temperature or pressure when you zoom in on an individual atom bouncing around.

Do we have free will?

I think we have it. My view is very similar to the one laid out by neuroscientist Kevin J. Mitchell in his book Free Agents: How Evolution Gave Us Free Will (2023). Basically, it’s that the moment we start talking about people or selves—I sense that you’re a person, you sense that I’m a person, I sense that I’m a person, and you sense that you’re a person—what we’re doing is engaging in an act of modeling in the same way that you model tables and chairs and all the other things that are behaviorally meaningful in the world. Modeling yourself turns out to be really important, because you need to also be able to predict how you are going to behave in the future and how you are going to feel under different counterfactuals. For example, when you respond to an email asking for your meal preference on a flight in a week, you have to imagine yourself next week on the plane. You have to time-travel with a model of yourself and test out different possibilities to see which one feels better.

That is what we call free will in the usual sense of the term. You are testing out various possibilities, and you’re making a real choice. There’s the “you” that is making a choice based on a model of yourself. If you can talk about computers making if-then type decisions, then you can certainly talk about yourself making such decisions as well, acknowledging that, unlike a classical computer, there is a level of indeterminacy in the model that you have of yourself and of others. That indeterminacy is why we have free will. There are parts of ourselves that we don’t reveal to others and that we’re not even able to reveal to ourselves, because we can’t model everything precisely. It’s necessary to be fuzzy in our model to even have a model at all. You don’t need quantum indeterminacy or spooky forces to be at work in order to understand that there is real indeterminacy involved in modeling a “self.”

Alan Turing said that the moment you understand everything about how something works, it ceases to have agency. I think that is true, but it’s impossible for us to understand everything about all the particles in our brains.

What do you mean by free will comes from our own internal “indeterminacy”?

If you look at the physics of, say, planets orbiting our star, those physics are highly deterministic. Using Newton’s laws and Einstein’s laws of gravitation, you can predict millions of years into the future what the orbits are going to be; that’s because they’re massive bodies, they’re moving in a vacuum, and they’re in stable orbits. That is physics with a lot of predictability and stability at the macroscopic level. You still can’t predict planetary orbits a billion years in the future because they do become chaotic on some timescale, but at least for a good while, you can do a good job. But that’s not the case for Julien or Blaise. We’re bumping up against the real world in tons of chaotic ways. And in fact, we are “engineered” to be “on the edge of chaos,” so to speak. As a living intelligent system, you need to be responsive to the environment. If somebody just whispers in your ear, “Julien, go left,” that has to be enough to cause your entire body to pivot in an instant. So there have to be systems that are tuned to be very sensitive to tiny perturbations, starting with your ear and going through your brain in order to make that possible. It’s both what makes us predictable to each other and able to model each other and, paradoxically, it’s also what makes us unpredictable. You could whisper to yourself, “What if I did this?” or have a flash of intuition, and that might be unpredictable to me and even unpredictable to yourself.

What is the relationship between life and intelligence?

They are very closely related. I mentioned that life is computational. It is also inherently symbiotic, meaning that the moment you have living things, they start to work together and form larger living things. Intelligence has all of these same properties. I used to wonder, Why is the brain computational? How is it that we evolved this computational organ? But it becomes clearer when you realize that we’re actually computational from the beginning. As things start to work together—like how cells come together to make larger bodies, neurons come together to make brains—what you’re really doing is increasing the parallelism of computation. And that’s exactly what brains are: massively parallel computers. We can see this with artificial neural nets. As we build massively parallel computers and train them, they start to get brain-like behaviors.

Intelligence is all about being able to model the world and yourself and others, and to predict how they’re going to work so that you can get on more successfully and persist through time. Under that definition, the moment you have a living system, it has to be intelligent.

Are modern AI systems intelligent?

Under my functional definition, they’re clearly intelligent. I think common sense also says that they’re intelligent. I know that there are some people who are averse to the idea that AI is intelligent. It’s less problematic outside the West and among everyday people, as opposed to academics. There are reasons that we can probably imagine for that, but I think if you took any of the frontier AI systems that we have today and transported them back to the year 2000 and asked anybody on the street, “Is this intelligent?,” they would say, “Yes, of course.”

Why does the West have more of an aversion to viewing AI systems as intelligent?

I could get into trouble by trying to psychoanalyze the West here, but I think that part of it is a sense of threat that comes from, in my opinion, a misplaced idea that individual humans are at the top of some grand species hierarchy. In reality, humans are in complex webs of interdependence with each other, and with many other species, and even technologies; there’s no real “top” or “bottom.” I also think that a lot of pundits have a vested interest in the idea that they’re the smartest ones. But in my experience, when you ask regular people who don’t make a living writing code or writing academic papers, they agree that modern AI systems are smart.

You write about the surprise many of us felt by the success of modern large language models (LLMs) in the early 2020s. For example, you explain that none of the models you were working on before then,

regardless of size, seemed like it had the requisite machinery to be able to do complex math, understand concepts or reason, use common sense, develop theory of mind, or be intelligent in any other generally accepted way. To hope otherwise seemed as naïve as the idea that you could reach the moon by climbing a tall enough tree. Most AI researchers—me included—believed that such capacities would require an extra something in the code, though few agreed on what that something might look like.

Put simply: it seemed clear enough that a real AI should be able to do next-word prediction well, just as we can. However, nobody expected that simply training an excellent next-word predictor would actually produce a real AI.

Yet that was, seemingly, exactly what happened in 2021.

You write, however, that we shouldn’t have been surprised by how well LLMs began to work. Why not?

My team at Google was building auto-complete systems to power products like Gboard, the smart keyboard in Android phones, which tries to predict the next word and put it on the strip above the keyboard to speed up typing. I certainly did not, in my gut, believe that by just making those systems much bigger and training them with a lot more data, they would start to solve word problems and auto-complete sentences about the emotional states of people in short stories. So I was very surprised when this started to happen.

But then, I thought: The predictive brain hypothesis must be right. The predictive brain hypothesis proposes that brains evolved to predict the future, and the hypothesis has been around in neuroscience for a long time. In that sense, the brain is arguably “just” an auto-complete system. Of course, there is a lot hiding in that “just.” But what we were building at Google was a predictor, and a predictor in a somewhat special situation where the things that it predicts it then emits, and those become part of what it then bases its future predictions on. It seems to me that the success of these large language models is a very powerful vindication of the predictive brain hypothesis.

Do you disagree with the criticism that modern LLM systems are limited because they are just statistical models?

My disagreement lies not in the substance but in the “just.” For example, my good friend Ted Chiang wrote an article for The New Yorker titled “ChatGPT Is a Blurry JPEG of the Web.” I agree: compression and prediction are very closely related. It is kind of a blurry JPEG of the internet. But when you say that, the subtext is, “and therefore, it is not intelligent.”

LLMs are, in some sense, just statistics. But when you’re trying to compress the statistics of something, or make a compact representation of it, that forces understanding. Even if we talk about something as simple as the multiplication table, if you’re trying to memorize the whole times table, then you’ll do okay at the one-digit level. But by the time you’re trying to multiply three digits by three digits, you can’t memorize the whole thing, so what you have to memorize is something else that lets you reconstruct the whole. You have to, essentially, figure out an algorithm that does multiplication. So understanding and compression and prediction are all very closely related. They all involve models, and the question becomes, How generalizable is your model outside its original domain? This is the question machine-learning people ask themselves all the time.

What about the pushback that says, despite this modeling, LLMs lack common sense and can’t be trusted because they make mistakes that humans wouldn’t make?

The way that LLMs learn and are trained is very different from the way we learn and are trained. The miracle is that they work at all and converge with our own brains as much as they do. When you’re an LLM skeptic, you’re waiting for the “gotcha” moment. You’re waiting for the moment when the LLM says something ridiculous, like, two plus two equals five. Or if we look at logic puzzles like river crossing problems, an LLM may do a river crossing that doesn’t make sense because, for example, the goat already crossed. Although LLMs are making these mistakes less and less. When people argue that LLMs can’t solve river crossing problems, well, that may have been true six months ago, but it’s actually not true anymore.

LLM skeptics also do not apply the same standards to LLMs that they use with humans. For example, AI scientist Melanie Mitchell and I—we’re good friends and have had really fruitful, friendly debates about these topics—once had a public debate at Town Hall in Seattle, and she said something about how LLMs lack common sense. I then presented her onstage with three or four examples that apparently show a lack of common sense, and she got one wrong too! We all had a laugh. Melanie is very smart, and in a fair test I know I wouldn’t have done any better. But our standards are very different when we’re looking for the computer to get it right every time.

In fact, when a human makes a mistake, it tends to prove they are human.

Exactly. There are many weird double standards, and I think it comes from importing ideas about computers in a kind of HAL 9000 way, where we’re saying they’re not human and they should be perfect, especially at logical problems. But LLMs are not solving logical problems as logic problems. They’re solving them with language the way we do, so of course they make mistakes—just like us.

There is a wide breadth of benchmarks for testing the behavior and intelligence of AI systems, such as with chess games, standardized tests like the SAT, and the Turing Test, which actually measures a very human trait: sociability. Is there an intelligence test or benchmark you think is valuable for evaluating modern AI systems?

I don’t think that intelligence is a single quantity. I know about the history of Spearman’s “g factor,” where there is a single variable that emerges as something that is correlated among many subtests. There are correlations among a lot of subtests of intelligence, in part because if people have some kind of organic brain damage or deficit, that will tend to affect their ability to perform multiple kinds of tasks or solve many different problems. But the idea that intelligence is a one-dimensional quantity doesn’t make a lot of sense to me. We all know people who are brilliant at one thing and suck at another. And in fact, the more brilliant they are in a particular area, the less likely they are to be well-rounded, in my experience. And it’s exactly people like that that make the world go round. So, I don’t like the quantification with any single number.

One test, though, that would be meaningful and that we’re on the cusp of now is where LLMs come up with new stuff that no one has come up with and expand the frontier of human knowledge with new inventions, proofs, and unsolved mathematical conjectures. My team at Google has made some efforts along these lines. And once LLMs make these advances, I think the argument that they’re not intelligent becomes a lot harder to make. Physicist David Deutsch, for example, has said that that’s the threshold to intelligence. In my mind, however, that sets the bar for intelligence too high. Most people wouldn’t make it, and that doesn’t make any sense to me.

Can you give examples of these tests?

We’ve quietly done a few. Math is the easiest to test for because it’s formalizable and you can know without a doubt whether something is right or not. Models from Google and OpenAI have recently done very well on Math Olympiads, which are famously hard; I think the AI solutions to some of these problems show signs of originality, but since the problems were written and previously solved by human judges, it’s hard to be sure that those solutions weren’t somewhere in the training data. When it comes to unsolved problems, though, we can be sure that they’re not in the training data. So a test made entirely of unknown mathematical statements or conjectures is something we can start to define and think about as a “golden” math test. For an AI to get a nonzero score on that test would mean it has done something both correct and genuinely creative in the field of mathematics.

You make many analogies between the brain and AI in the book. What has been the value of looking to biology for your work on AI systems?

The fields of computer science, neuroscience, and artificial intelligence actually began as one field. They split off from one another as the years passed because we realized that the way we had imagined neurons work, which was essentially as logic gates, wasn’t the way real neurons work. So neuroscience went its own way. Meanwhile, computer science discovered that it was good for a lot of stuff like calculating missile trajectories and keeping spreadsheets, but was not making progress toward AI. So they had an amicable divorce. But there have always been people working in the middle. For example, the idea of artificial neural networks is totally brain-inspired. Convolutional neural nets, which were behind the big AI successes of the 2010s, were modeled specifically on the visual cortex. There is a lot more biology in neural nets than many people realize. Almost all of the innovations, at least before the transformer, came straight out of neuroscience.

Now we’ve started to see that transformer-like operations might be occurring in certain brain structures, so there is some work that is starting to go the other way, where we realize that engineering solutions for artificial neural nets actually exist in the brain too. We’re also finding that there are convergences between the emergent properties of artificial neural nets—the kinds of maps that they develop of the world, for example—and what we find in brain scans, which actually makes sense. Obviously, there are a lot of differences—neurons are wet and the mechanics of how they work are really different from how silicon does—but because they’re trying to solve the same functional thing, they tend to converge on similar functional solutions.

You end the book by discussing the implications of AI on society and noting that there is much anxiety around AI’s deployment. Is the anxiety warranted?

The anxiety is not baseless, although I think that it often finds irrational outlets. Now things could all fizzle out next year and the huge progress we have made could suddenly stall. But it certainly doesn’t look like it’s stalling. Rather, it looks like things are on a rocket. It reminds me of what happened between 1870 and 1970. When J. R. R. Tolkien was a kid, the cavalry charge was still a thing, and by the time he died, we had the hydrogen bomb. That is unbelievable. Whereas between 1970 and now, sure, things have happened, there are personal computers, the internet, and smartphones, but this is nothing compared to what happened in the previous century. And I think we’re now back in turbo and that can be scary. We’ve gotten used to a different, more staid pace of progress. And when so much changes technologically, when so many things become possible, there can be a lot of disruption, as there was in the 19th and early 20th centuries.

I do worry about the disruption AI may cause. I worry about politics and whether the way we do it still makes sense. I worry about our economic system and whether it can still work for us. I worry about social media and our ability to communicate with each other, and whether our brains are just fundamentally not well calibrated for it. But I think that placing the anxiety on runaway malign AI is more revealing about our psychology than it is about keeping our eye on things that are truly dangerous, like nuclear weapons and climate change.

Whether we’re going to be able to “align” AI with humanity sounds in theory like a scary problem, particularly if you read philosopher Nick Bostrom’s book Superintelligence: Paths, Dangers, Strategies (2014), or more recent scare-mongering books and articles in a similar vein. But Bostrom was imagining a kind of AI that is totally different from what we actually have. He was imagining AI as an alien general intelligence that was a super good game player in an adversarial way, a kind of superintelligent spider that thinks and acts in utterly inscrutable ways. And that’s not the way real AI works at all. Modern AI systems are actually very human, for good and for ill. We got general AI to work not by having it play adversarial games using alien strategies but by training it on lots and lots of human language. It’s not HAL 9000. Even its logical failures are surprisingly humanlike. Does that mean our AI is “aligned” with humanity? Yes and no. The diversity of human perspectives and of human values is a part of what makes us go. The idea that there is one “correct” alignment is antihuman and doesn’t make any sense. The idea that humans are aligned with each other is absolutely not true—and that’s okay. Recently, we’ve oscillated between “AI is too sycophantic and aligns too much with what a particular user wants” and “It’s too aloof and doesn’t want to be my romantic partner.” Which “alignment” is better? There’s no right answer, of course.

¤

Blaise Agüera y Arcas is a vice president–fellow at Google, where he is the CTO of Technology & Society, and the founder of Paradigms of Intelligence, an organization dedicated to fundamental AI research. He is the author of Who Are We Now? (2023), and his research has included work on privacy-preserving machine learning, on-device AI, large language models, and human identity.