How to Raise Your Artificial Intelligence: A Conversation with Alison Gopnik and Melanie Mitchell

This interview is part of The Rules We Live By, a series devoted to asking what it means to be a human living by an ever-evolving set of rules. The series is made up of conversations with those who dictate, think deeply about, and seek to bend or break the rules we live by.

¤

A GROWING FEAR and excitement for today’s AI systems stem from the assumption that as they improve, something—someone?—will emerge: feed large language models (LLMs) enough text and, rather than merely extracting statistical patterns in data, they will become intelligent agents with the ability to understand the world.

Alison Gopnik and Melanie Mitchell are skeptical of this assumption. Gopnik, a professor of psychology and philosophy studying children’s learning and development, and Mitchell, a professor of computer science and complexity focusing on conceptual abstraction and analogy-making in AI systems, argue that intelligence is much more complicated than we think. Yes, what today’s LLMs can achieve by consuming huge swaths of text is impressive—and has challenged some of our intuitions about intelligence—but before we can attribute to them something like human intelligence, AI systems will need the ability to actively interact with and engage in the world, creating their own “mental” models about how it works.

How might AI systems reach this next level? And what is needed to ensure their safe deployment? In our conversation, Gopnik and Mitchell consider various approaches, including a framework to describe our role in this next phase of AI development: caregiving.

¤

JULIEN CROCKETT: Let’s start with the tension at the heart of AI: we understand and talk about AI systems as if they are both mere tools and intelligent actors that might one day come alive. Alison, you’ve argued that the currently popular AI systems, LLMs, are neither intelligent nor dumb—that those are the wrong categories by which to understand them. Rather, we should think of them as cultural technologies, like the printing press or the internet. Why is a “cultural technology” a better framework for understanding LLMs?

ALISON GOPNIK: A very common trope is to treat LLMs as if they were intelligent agents going out in the world and doing things. That’s just a category mistake. A much better way of thinking about them is as a technology that allows humans to access information from many other humans and use that information to make decisions. We have been doing this for as long as we’ve been human. Language itself you could think of as a means that allows this. So are writing and the internet. These are all ways that we get information from other people. Similarly, LLMs give us a very effective way of accessing information from other humans. Rather than go out, explore the world, and draw conclusions, as humans do, LLMs statistically summarize the information humans put onto the web.

It’s important to note that these cultural technologies have shaped and changed the way our society works. This isn’t a debunking along the lines of “AI doesn’t really matter.” In many ways, having a new cultural technology like print has had a much greater impact than having a new agent, like a new person, in the world.

MELANIE MITCHELL: We use so many different metaphors to talk about LLMs. They have been called “autocomplete on steroids” and “stochastic parrots.” The science fiction writer Ted Chiang called ChatGPT a “blurry JPEG of the web.” Alison calls them “cultural technologies.” Others use metaphors that have more to do with agency and understanding. I’m not sure we’ve come up with the right metaphor because in some sense, LLMs are all these things. And maybe we have to say to what extent they are each of these things. There’s been such a big debate in the AI community about this, and it’s interesting that such smart people can so violently disagree on how to think about these systems. But it just reflects something that’s been constant in the history of AI: it’s challenging our notions of what intelligence means.

Back in the 1970s, a lot of people were saying, if a computer could play chess at a grandmaster level, that would require general humanlike intelligence. Clearly it didn’t. We thought the same thing about translation. Now we’re being faced with systems that are conversationalists and able to act like agents that understand conversation. Does that require humanlike intelligence? It doesn’t seem to. So maybe that’s just the next step in how AI pushes people’s intuitions about what intelligence is.

AG: There is an implicit intuitive model that everyday people (including very smart people in the tech world) have about how intelligence works: there’s this mysterious substance called intelligence, and as you have more of it, you gain power and authority. But that’s just not the picture coming out of cognitive science. Rather, there’s this very wide array of different kinds of cognitive capacities, many of which trade off against each other. So being really good at one thing actually makes you worse at something else. To echo Melanie, one of the really interesting things we’re learning about LLMs is that things like grammar, which we might have thought required an independent-model-building kind of intelligence, you can get from extracting statistical patterns in data. LLMs provide a test case for asking, What can you learn just from transmission, just from extracting information from the people around you? And what requires independent exploration and being in the world?

What do you think are the limits to what LLMs can achieve just by being trained on language alone?

MM: It’s always dangerous to say LLMs can’t do something, given that language is such a rich representative medium and there is so much data out there. But, for example, there’s a new push to use LLMs to help control robots. I’m not convinced LLMs are going to be the solution to get a robot that can do physical activities like fold your laundry or anything really that involves both understanding motor action and how the world works at a very basic level.

AG: If you look at, for instance, videos of the very best robots people are inventing, you’ll notice that there’s a little number in the corner that says 10x or 20x or something like that, and what that means is that the video has been sped up 20 times to make it look like the robot is doing something intelligent. If you were actually watching the robot in real life, it would just look like it was incredibly painfully slow and awkward and making mistakes all the time. This is representative of what’s called Moravec’s paradox: things that looked as if they would be really, really hard for AI and require a lot of intelligence, like playing chess, turned out to be relatively easy. And things that look like any two-year-old can do them, like picking up an object and putting it in a pot and stirring it, are actually really hard. LLMs have made that paradox more vivid.

But going back to the contrast between different kinds of intelligences, one type of intelligence is about what I call transmission: how do I pass information to and extract information from another person? A different kind of intelligence is about truth: I’m in the world and the world is changing—what AI people call a “nonstationary environment”—so how do I find the truth about something that we haven’t seen before? Science is the most dramatic example of engaging in this activity, but even very young children are good at this. LLMs are not trained to do this. We talk about how they “hallucinate,” but hallucination isn’t really the right word. To hallucinate would mean that you recognize the difference between the truth and just a bunch of things that people have said. LLMs are not designed to make that distinction.

MM: I like to tell people that everything an LLM says is actually a hallucination. Some of the hallucinations just happen to be true because of the statistics of language and the way we use language. But a big part of what makes us intelligent is our ability to reflect on our own state. We have a sense for how confident we are about our own knowledge. This has been a big problem for LLMs. They have no calibration for how confident they are about each statement they make other than some sense of how probable that statement is in terms of the statistics of language. Without some extra ability to ground what they’re saying in the world, they can’t really know if something they’re saying is true or false.

Melanie, this ties into what you’ve said is the biggest issue facing LLMs: that they rely on statistical associations rather than “concepts.” What is a concept and why is it a limitation for LLMs?

MM: A concept is a mental model of some aspect of the world that has some grounding in the truth. You might have a concept of something like a unicorn, and while it’s not true in the real physical world, it’s actually true in some different kind of fictional world. I know it’s fictional, but I still have a concept of it, and I can answer questions about it. I think these mental models of the way the world works, which involve things that cause other things, are “concepts.” And this is something that I don’t think LLMs have, or maybe even can develop, on the scale that humans do.

AG: Another really important point about what makes mental models different from just statistical patterns is that when you have a mental model, you also have an idea that you can go out and test whether that model is right or not. I can actively go out into the world, do an experiment, and get the right kind of new data to decide whether I’m right or wrong. And that, again, is something that two-year-olds do all the time (although we call it “getting into everything”). Certainly, intelligent nonhuman animals also do this. It’s not that, in principle, AI systems can’t do it, but it’s not what LLMs are doing.

How can we give AI systems the ability to construct mental models?

MM: I think it requires the ability to intervene in the world and do experiments and reason about things counterfactually, like “if I had done this, what would have happened?” Or “if this other thing had happened, how would that affect everything?” Animals who have very different bodies from our own do this kind of reasoning, but the particulars of their bodies and their sensory systems matter a lot for how they conceptualize the world. The ability to more actively interact with the world and learn, as opposed to passively sitting there waiting for more data to come in, that’s going to be really important. In machine learning, people use the word “curriculum” for how you shape training. Do you just throw all of Wikipedia at it? Or do you let it develop more in the way a child develops?

AG: In my research, we’ve been comparing children and various agents to ask how good each is at constructing its own curriculum. If I give you a task, like a video game level that you can’t solve, can you figure out the simpler thing that you should do? For example, here’s the simpler game I should play to become an expert and then, eventually, I’ll be able to solve a harder level. We found that kids are surprisingly good at coming up with curricula but current AI systems are not.

There’s an evolutionary argument that the time when “intelligence” shows up in evolution is in the Cambrian explosion. Before the explosion, you had lots of organisms like sponges living on the bottom of the ocean, and they had a wonderful life where food wafted over them and they extracted it. But what happens in the Cambrian is you start having organisms with eyes and claws, or what biologists call actuators and sensors. When you get actuators and sensors, you can perceive things and move, and that’s a really different niche for an animal. That’s when you start getting neural systems and brains because you need a brain to coordinate action and sensing. And when you get a bunch of these animals together, they start trying to find prey and avoid predators. You get a perceptual system that’s connected to the outside world and taking in information about the world, and a motor system that’s connected to the outside world and going out and changing the world. This is a foundational kind of structure for which you need to have the kind of truth-seeking intelligence we are talking about.

There are some interesting attempts within robotics and AI to use reinforcement learning to try and get systems that are motivated to find truth. Instead of just trying to get rewards like a higher score in a game, these systems are motivated to get information or to try to be more effective in the world. And I think that might be the right route to think about for something that looks like the intelligence that evolved in the Cambrian.

How important for this next generation of robots and AI systems is incorporating social traits such as emotions and morality?

MM: Intelligence includes the ability to use tools to augment your intelligence, and for us, the main tool we use is other people. We have to have a model of other people in our heads and be able to, from very little evidence, figure out what those people are likely to do, just like we would for physical objects in the real world. This theory of mind and ability to reason about other people is going to be essential for getting robots to work both with humans and with other intelligent robots.

AG: Some things that seem very intuitive and emotional, like love or caring for children, are really important parts of our intelligence. Take the famous alignment problem in computer science: How do you make sure that AI has the same goals we do? Humans have had that problem since we evolved, right? We need to get a new generation of humans to have the right kinds of goals. And we know that other humans are going to be in different environments. The niche in which we evolved was a niche where everything was changing. What do you do when you know that the environment is going to change but you want to have other members of your species that are reasonably well aligned? Caregiving is one of the things that we do to make that happen. Every time we raise a new generation of children, we’re faced with this difficulty of here are these intelligences, they’re new, they’re different, they’re in a different environment, what can we do to make sure that they have the right kinds of goals? Caregiving might actually be a really powerful metaphor for thinking about our relationship with AIs as they develop.

Alison, this concept of caregiving touches on a recent conversation you had with Ted Chiang about his novella The Lifecycle of Software Objects (2010). What interested you in Chiang’s work, and what are the parallels with your own?

AG: As always, novelists are better at conveying things than we scientists are. This is a lovely science fiction story about people who try to raise AI agents as if they were children and describes the very complicated dilemmas the human parents of these AIs experience and the ways the AIs try to both follow what their parents do and find paths for themselves. It’s actually the best description of human parenting that I’ve read. Now, it’s not like we’re in the ballpark of raising AIs as if they were humans. But thinking about that possibility gives us a way of understanding what our relationship to artificial systems might be. Often the picture is that they’re either going to be our slaves or our masters, but that doesn’t seem like the right way of thinking about it. We often ask, Are they intelligent in the way we are? There’s this kind of competition between us and the AIs. But a more sensible way of thinking about AIs is as a technological complement. It’s funny because no one is perturbed by the fact that we all have little pocket calculators that can solve problems instantly. We don’t feel threatened by that. What we typically think is, With my calculator, I’m just better at math.

MM: Often, when these technologies first come out, people worry a lot because they think they’re going to harm us in some way. When calculators first appeared, people didn’t want kids using them because they thought that would make kids not learn math. But as we figure out what these technologies are good for, we learn how to use them. That will happen with AI. It’s going to be a new kind of technology that’s going to augment us in many ways, just like other technologies have, but it’s not going to supplant us. It’s not going to take away all of our jobs, because it’s just not that kind of thing. It doesn’t have the kinds of qualities that are going to replace humans.

AG: But we still have to put a lot of work into developing norms and regulations to deal with AI systems. An example I like to give is, imagine that it was 1880 and someone said, all right, we have this thing, electricity, that we know burns things down, and I think what we should do is put it in everybody’s houses. That would have seemed like a terribly dangerous idea. And it’s true—it is a really dangerous thing. And it only works because we have a very elaborate system of regulation. There’s no question that we’ve had to do that with cultural technologies as well. When print first appeared, it was open season. There was tons of misinformation and libel and problematic things that were printed. We gradually developed ideas like newspapers and editors. I think the same thing is going to be true with AI. At the moment, AI is just generating lots of text and pictures in a pretty random way. And if we’re going to be able to use it effectively, we’re going to have to develop the kinds of norms and regulations that we developed for other technologies. But saying that it’s not the robot that’s going to come and supplant us is not to say we don’t have anything to worry about.

Are you surprised by the rate at which we’re adopting AI tools like ChatGPT?

MM: It depends what you mean by adopting. People use ChatGPT, but what do they use it for? I don’t have a good sense of the degree to which it’s being adopted as a big part of people’s jobs, for instance. I think it’s really useful for some things like coding. But it’s not perfect—you have to check it.

AG: Rather than the dramatic AI narrative about what’s just happened with ChatGPT, I think it’s important to point out that the real revolution, which passed relatively unheralded, was around the year 2000 when everything became digital. That’s the change that we’re still reckoning with. But because it happened 20 to 30 years ago, it’s something we take for granted. There’s a wonderful paper in the journal Psychological Science where they asked people how harmful they thought a fictional technology from a particular date was. The researchers very cleverly connected the technology’s date to the person’s birthday. It turns out that if something was invented more than two years after you were born, you’re much more likely to think it was harmful than if it was invented two years before you were born. Suppose I said to you, here’s a technology that we know kills millions of people every year, and is an existential threat to our existence on the planet. How do you feel about it? Well, that technology exists; it’s called the internal combustion engine. No one is out there thinking, as they look at their family car, This thing is so terrifying. It’s changed the world so much and we do have to do something about it, but it doesn’t have the same kind of emotional impact that thinking about artificial intelligence does.

But there are many people familiar with the technology, such as Geoffrey Hinton and Douglas Hofstadter, who are very worried about the direction AI could go in. What are they seeing in the technology that you don’t think is right?

MM: I wouldn’t say that you shouldn’t be worried about it. There are a lot of bad things that could happen with these technologies. We’re already seeing some bad things happen, like what Cory Doctorow called the “enshittification of the internet,” where there are tons of crap out there now and it’s just getting worse. There are also the problems of bias, privacy, and the ridiculous amount of electricity and water that are used in data centers. I think, though, that Hinton’s and Hofstadter’s concerns are different. Hinton is worried about the systems getting out of control, becoming “superintelligent,” manipulating us, and having their own agency. Hofstadter, on the other hand, is more worried about dehumanization—that the things he values most, like the creation of music and literature, will be taken over by computers.

I’m less worried about these more speculative risks because I don’t see evidence that they are going to happen soon. I wrote a paper called “Why AI Is Harder Than We Think,” where I explained how people underestimate how hard it is to capture human intelligence in these machines. There’s a lot of stuff making up human intelligence that these critics aren’t really taking into account as part of intelligence. I find it funny that people call Hinton an “artificial general intelligence” expert because there’s no such thing as AGI. No one knows what intelligence is, much less how it’s going to be captured in computers in the future.

AG: I think it is admirable that some of the pioneers in the field are taking responsibility for the technology and trying to be thoughtful about its effects on the world. It’s an admirable sort of Robert Oppenheimer impulse. And, as Melanie said, there certainly are bad things that could happen. But I also think it’s true that the people who are designing the systems have an interest in saying that these systems are really powerful and have something like general intelligence. It is also striking to me that you see much more existential anxiety among people who aren’t cognitive scientists, who haven’t studied human or animal intelligence. I think it’s fair to say that the consensus among people who study human intelligence is that there’s a much bigger gap between human and artificial intelligence, and that the real risks we should pay attention to are not the far-off existential risks of AI agents taking over but rather the more mundane risks of misinformation and other bad stuff showing up on the internet.

Are either of you worried about the extent to which AI systems are already subtly influencing our decisions, particularly online and through our phones?

MM: In some areas of our lives, yes. We trade our agency for convenience. For example, I like to use GPS in my car to navigate. It’s very convenient and I trust that it’s going to work. But it means that I don’t get as good a mental map of the places where I go. That’s one of the problems with a lot of these systems—they make things very convenient for us, which comes at a cost, and we only sort of understand the cost. There are other examples in history where we’ve given up certain abilities for convenience, such as books, where we don’t have to memorize. Having a smartphone means that I can just look stuff up rather than ask people or go to the encyclopedia. Handing off our decision-making to algorithms has hurt us in some ways, and we’re starting to see the results of that now with the current state of the world.

AG: The political scientist Henry Farrell has argued that we’ve had artificial intelligences before in the form of markets and states. A market is just a big information-processing, decision-making device. So, in a funny way, anytime I see that something costs $4.99 and I pay it, I’m giving up a kind of autonomy to the force of the market, right? I’m not acting as I would if I had lived in a foraging culture, for example. We have these large-scale information-processing devices, and markets and states and bureaucracies are really good examples of this, where we give up individual decision-making. Legal systems are like that too. I’m not deciding whether I’m going to cross the street; the traffic light is telling me whether I should cross. And, again, those things have benefits in coordinating people across very large social communities. But they also have downsides too. They can take on a life of their own, independent of all the individual people who are making decisions about them. So when we say the country has decided to invade or we say the market has collapsed, that’s actually a lot of individual people making decisions, but these superindividual decision-making systems have had a kind of life of their own as long as we’ve been around as human beings.

MM: This reminds me of Nick Bostrom’s paperclip apocalypse where a superintelligent AI system behaves in a psychopathic way: it’s given a goal and doesn’t care about the consequences of its actions as long as it is able to achieve that goal. Ted Chiang wrote a piece where he argued that we already have entities that act like that now: they’re called corporations and their goal is maximize shareholder value. I think that’s why Silicon Valley people often worry about what AI is going to do. Corporations maximizing shareholder value is the metaphor they’re using to think about AI systems.

AG: I think Melanie is absolutely right. It’s about these metaphors. Often the metaphor for an intelligent system is one that is trying to get the most power and the most resources. So if we had an intelligent AI, that’s what it would do. But from an evolutionary point of view, that’s not what happens at all. What you see among the more intelligent systems is that they’re more cooperative, they have more social bonds. That’s what comes with having a large brain: they have a longer period of childhood and more people taking care of children. Very often, a better way of thinking about what an intelligent system does is that it tries to maintain homeostasis. It tries to keep things in a stable place where it can survive, rather than trying to get as many resources as it possibly can. Even the little brine shrimp is trying to get enough food to live and avoid predators. It’s not thinking, Can I get all of the krill in the entire ocean? That model of an intelligent system doesn’t fit with what we know about how intelligent systems work.

How has your work on AI changed the way you understand yourself?

MM: John McCarthy, one of the pioneers of AI, and one of the people from the 1956 Dartmouth workshop who thought we would make great progress in just a summer, later said, “You know, intelligence was harder than we thought.” That’s just been the constant progression of my own view. And it’s made me want much more to talk to people in cognitive science, like Alison. I think that research in AI is going to have to get back to its original focus on the nature of intelligence as opposed to better optimization techniques.

AG: As a developmental psychologist, I spend a lot of my time with little kids, and they are infinitely smarter than we think. Working with AI has made me even more impressed with the kinds of things that every two-year-old is doing. It has also made the intelligence of octopuses, brine shrimp, and all the other creatures around us more vivid. On the other hand, I would not have assumed that we could learn as much just from text as we do. That raises the question about how much of what I think is my deep knowledge about the world is really just my being able to parrot the things I’ve read or heard other people say. LLMs have raised that issue in a really interesting way.

¤

Alison Gopnik is a professor of psychology and affiliate professor of philosophy at the University of California, Berkeley, where she has taught since 1988. She received her BA from McGill University and her PhD from Oxford University. She is a world leader in cognitive science, particularly the study of children’s learning and development. She is the author of over 100 journal articles and several books, including the best-selling and critically acclaimed popular books The Scientist in the Crib (1999), The Philosophical Baby: What Children’s Minds Tell Us About Love, Truth and the Meaning of Life (2009), and The Gardener and the Carpenter: What the New Science of Child Development Tells Us About the Relationship Between Parents and Children (2016). She is a fellow of the Cognitive Science Society and the American Association for the Advancement of Science and a member of the American Academy of Arts and Sciences.

Melanie Mitchell is a professor at the Santa Fe Institute. Her current research focuses on conceptual abstraction and analogy-making in artificial intelligence systems. Melanie is the author or editor of six books and numerous scholarly papers in the fields of artificial intelligence, cognitive science, and complex systems. Her 2009 book Complexity: A Guided Tour won the Phi Beta Kappa Science Book Award, and her 2019 book Artificial Intelligence: A Guide for Thinking Humans was short-listed for the Cosmos Prize for Scientific Writing. Melanie is the recipient of the Senior Scientific Award from the Complex Systems Society, the Distinguished Cognitive Scientist Award from UC Merced, and the Herbert A. Simon Award of the International Conference on Complex Systems.

¤

Featured image: El Lissitzky. Proun 99, ca. 1923–25. Gift of Collection Société Anonyme. Yale University Art Gallery (1941.548). CC0, artgallery.yale.edu. Accessed May 25, 2024. Image has been rotated.