Automating the Professions: Utopian Pipe Dream or Dystopian Nightmare?
By Frank PasqualeMarch 15, 2016
The Future of the Professions by Daniel Susskind and Richard Susskind
The Susskinds don’t just foresee the robotization of accountants, actuaries, and architects. They anticipate the emergence of bot preachers, ministering to avatars in Second Life’s virtual cathedrals. They ride the “massive open online course” bandwagon, however stalled it may be at present. They argue that struggling newspapers may well use Narrative Science algorithms (now deployed to churn out earnings reports and sports news) to write up politics, business, and style stories. And they quote a study by Carl Benedikt Frey and Michael Osborne arguing that only one percent of tax preparation work is safe from computerization. The future of the professions is thus, for them, at the mercy of artificial intelligence–driven (AI) startups.
How realistic are such scenarios? The book argues that even the most complex, professionalized work is on the road to computerization. The logic is straightforward: machines will watch what professionals do, record their “outputs,” and reproduce them.
But, in fact, this logic is far too simple. The Susskinds cherry-pick the Frey and Osborne study, highlighting the parts they like while studiously ignoring its more optimistic conclusions about surgeons, nutritionists, clergy, teachers, and many other professionals. Indeed, the Frey and Osborne study is far less negative about the viability of the professions than the Susskinds’ book, assigning a “low” probability of computerization to myriad professional positions:
It is hard to disprove Frey and Osborne’s conclusions without exaggerating the quality of existing data recording, the power of current or near-future algorithms to integrate data streams, and the ability of engineers with little or no domain expertise in a given profession to code software replacing its human experts. Sadly, the Susskinds fall into these traps even where one would presume their expertise would be greatest — in law, where Richard Susskind has had some influence.
Legal Automation: Symbiosis, not Substitution
The Susskinds welcome the accelerating automation of litigation and transactions. They claim that the form provider “LegalZoom operate[s] reliably and to a high level of user satisfaction,” and they extrapolate from its success (and that of firms like it) to the rest of the profession. Unfortunately, they fail to credibly estimate how much of extant legal work is as automatable as, say, drafting a very basic will. Researchers Dana Remus of UNC and Frank Levy of MIT, who have done that work, project far milder effects on the profession. Many of the new software-driven “legal services” take on low-value tasks that were either rarely performed by lawyers or rarely generated much revenue for them (like uncomplicated contract drafting). Other software services might very well generate, rather than destroy, complex work for lawyers. For example, novel procedural moves accelerated by software may empower a whole new generation of plaintiffs’ lawyers.
Experts differ on the real likelihood of pervasive legal automation, and its effects. Frey and Osborne put the risk to lawyers at under four percent, and project that paralegals are in much more danger. But empirical research by economist James Bessen has complicated even that assumption: “Since the late 1990s, electronic document discovery software for legal proceedings has grown into a billion dollar business doing work done by paralegals, but the number of paralegals has grown robustly.” Like MIT’s David Autor, he observes that automation can create more jobs than it eliminates. Considering the role of text-mining software in law firms, the management consulting firm McKinsey says automation amplifies “the value of expertise by increasing an individual’s work capacity and freeing the employee to focus on work of higher value.” Researchers Michael Simkovic and Frank McIntyre reach a similar conclusion. All of this research taken together suggests that the Susskinds commit the classic “lump of labor” fallacy, assuming that there is some fixed amount of legal work to be done, and thus any cession of that lump to machines results in less work for extant professionals.
The Susskinds are not unaware of such research, conceding that “there is no fixed ‘lump’ of labour.” They nevertheless insist that, even if new tasks arise, “machines might be better placed than people to undertake” them. But their subjunctive mode here can just as easily be reversed: people might be better placed than machines. We should thus expect corporations, governments, lawyers, and programmers to struggle over the scope and intensity of automation in coming decades. Context matters, and the Susskinds are all too prone to dwelling on harmless or uncontroversial automation while downplaying its more sinister overtones.
For example, consider red light cameras implemented without a right to appeal or dispute a ticket triggered by them. “The Department of Motor Vehicles could attach to any driver’s license an “automatic debit permission” to routinize payment of fines.” Such a system could completely standardize the law of red lights, and would of course render obsolete some portion of what current traffic attorneys do. But it would also make a mockery of due process and other core legal principles. Indeed, so-called “smart cities” might make many forms of law-breaking impossible — but could do so at great cost to protesters and the marginalized populations already besieged by discriminatory laws and policing. We would need to reform the law wholesale before blithely committing ourselves to its perfect enforcement.
Visions of future sociotechnical developments always reflect contestable political aspirations. It is easy to press for a decline in professionals’ income or status by claiming that software could do their work. It’s harder to explain why the many non-automatable aspects of their current practice should be eliminated or uncompensated. That’s where the Susskinds falter, and where their book takes a rather nasty turn.
Repeatedly questioning the motives and competence of current professionals, the Susskinds insinuate that they resist change simply out of hidebound protectionism or technophobia. The values at stake in the human resolution of disputes — versus their automated dispatch to a computer system — are are flattened into a utilitarian calculus of speed and efficiency. To counter critics of automation, they treat “cost-cutting” as an all-purpose trump card. The general public should be grateful for more legal automation, they argue, because human lawyers are too expensive.
This is cavalier, not to mention naive. I would worry about any person who decides to file a tort or contract case against a major corporation using an app. If the claim is frivolous, they could be sanctioned. If the claim is serious, it will probably be outmaneuvered by a (human) defense lawyer. And if corporations don’t even need to deploy attorneys to deflect such interventions, but can even automate their own defense, then there’s little reason to believe this will constitute some great triumph for justice. Our legal system exacerbates inequality because of uneven access to resources for advocacy, not lack of automation. Digital projects to “democratize the law” rarely include the more sophisticated predictive analytics the Susskinds trumpet; instead, they remain the exclusive preserve of wealthy corporations. The Susskinds give us little reason to believe that automation will impede — rather than accelerate — inequalities in legal resources.
Why is it so difficult to turn legal disputes over to a computerized simulation of justice? Situations involving conflicting rights, unusual fact patterns, and open-ended laws will remain excessively difficult to automate for the foreseeable future. As the former General Counsel of DuPont Canada, Seymour Trachimovsky, said of an earlier Richard Susskind broadside, predictions of pervasive legal automation exhibit “little appreciation of the huge volume of indecipherable legislation and regulation that is promulgated every day.” The Future of the Professions argues that standardization of legal filings, briefs, and judgments is the way of the future, bringing the logic of interchangeable parts to the work of the mind. But, as Trachimovsky counters, “given modern trends in legislative and regulatory drafting, in particular the use of ‘creative ambiguity’ [...] demand for custom services will only increase.”
Note, too, that much of this legal complexity is not simply a question of redundant, easily simplified legal verbiage. It reflects, instead, deep-seated social tensions that are unlikely to be resolved any time soon. For example, had America united behind the goal of universal healthcare in 2009, the Affordable Care Act could have been a sentence or two long: “1. Everyone is eligible for Medicare, and 2. Eligibility starts Jan 1, 2010.” Instead, the Act went through myriad iterations as multiple stakeholders pressed their interests. It is now being interpreted in an epic series of rulemakings and adjudications on the federal and state level, which must address deep tensions between older models of healthcare regulation and the integrated delivery systems favored under PPACA. (For example: do we still forbid certain payments from hospitals to doctors as “kickbacks,” or do we let the industry and physicians recharacterize them as “gainsharing incentives”?)
Not only is the complexity of the Act becoming ever more obvious, but so too is the fact that human values and judgment will be critical to its implementation. As just one example: Is it too much to ask a policyholder to travel 25 miles to find a gastroenterologist? 50 miles? Questions like this will determine the fate of “narrow networks” (a key feature of many exchange plans). If the Susskinds have an algorithm to “solve” such legal questions, I would love to see it — particularly given the extraordinary variation in geography, car ownership, and public transit access in the US.
The fate of artificial intelligence in law is far more open-ended than The Future of the Professions suggests. Only specific, situated, technical, and sociological analyses of particular areas of law are truly valuable here. The Susskinds hastily sketch the power of “sophisticated diagnostic expert systems, which tackle highly complex, multi-jurisdictional legal questions,” but give us little sense of how their processing of tough legal questions is translated to clients, courts, or regulators. Presumably there are no direct brain downloads available yet, and each audience will demand more than a computational ipse dixit. Moreover, that demand for humanly intelligible explanation is important — no one should be denied access to benefits (or thrown into jail!) simply because a computer said yes or no.
Of course, technology has shaped, and will continue to influence, legal practice. But its effects can be checked or channeled by law itself. The Susskinds themselves finally arrive at this realization at the end of their book. Having extolled the virtues of technology at length, they hurriedly acknowledge the importance of law and policy in determining whether, say, automated life support systems should be able to terminate treatment for a patient who appears to be in a persistent vegetative state. They would like to believe that such ethical dilemmas are rare in order to retain the credibility of their central claim: that the vast majority of medical and legal scenarios can, and will, eventually be processed by software. But the medical field is rife with enduring legal and ethical dilemmas requiring difficult judgment calls and interpersonal communication.
Paging Dr. Google
Compared to the manufacturing, military, and finance sectors, the pace of automation has been sluggish in healthcare. Software has a difficult time mimicking the spontaneity, creativity, flexibility, and perceptiveness that are the hallmarks of good nurses, surgeons, psychiatrists, pediatricians, and so on. End-of-life care obviously requires that difficult decisions be made on the basis of imperfect information. The idea of mapping out every possibility in advance, and mechanizing the “termination of treatment,” is bizarre for anyone with personal or professional experience in an intensive care unit, nursing home, or hospice. The stakes are so high, the concept of what makes life worth living so ineffable and variable, the decision-making so intricately shared among family and providers, that routinization seems ill-advised at best, and deeply offensive at worst.
Moreover, even elementary medical apps can fail patients. Earlier this year, the Federal Trade Commission settled lawsuits against firms who claimed their software could aid in the detection of skin cancer by evaluating photographs of the user’s moles. The FTC argued that there was insufficient evidence to support such claims. The companies are now prohibited from making any “health or disease claims” about the impact of the apps on the health of users unless they provide “reliable scientific evidence” grounded in clinical tests. If algorithms designed merely to inform patients about their options aren’t ready for prime time, why presume diagnostic robots are imminent?
The Susskinds have another easy answer at the ready: Moore’s law. In other words, following Ray Kurzweil’s reasoning in The Age of Spiritual Machines, they predict that computer processing speeds will keep dramatically improving; difficult problems will become trivial. But even if we concede that Kurzweil’s controversial equation of human cognition applies to machine information-processing, the timeline of technological advance is necessarily fuzzy. Much of healthcare’s big data is also “bad data”; it needs to be verified, standardized, and connected before software can do a decent job of analyzing it. That process will demand greater professional engagement with patient records, not less — both from researchers contemplating novel issues in data science, and attorneys and security professionals grappling with vexing privacy issues.
For every hour that software may save a radiologist by corroborating a difficult diagnosis, many more hours may be needed to integrate new data streams into a diagnosis, track the results of therapeutic interventions, and discover underlying genetic and environmental determinants of disease. New professionals will be needed, too — and not just as coders. Understanding how the body itself works (and not just correlations among computerized representations of it) will be critical.
Having considered the informational and diagnostic aspects of medicine, let’s consider a third area: procedures. Yes, there are robotically assisted surgeries, albeit with a very limited scope of application at present. Automation optimists project that their early successes will quickly spread to other types of interventions. But MIT economist David Autor offers a general reality check about automation that applies with even more force here:
Most automated systems lack flexibility — they are brittle. Modern automobile plants, for example, employ industrial robots to install windshields on new vehicles as they move through the assembly line. But aftermarket windshield replacement companies employ technicians, not robots, to install replacement windshields. Why not robots? Because removing a broken windshield, preparing the windshield frame to accept a replacement, and fitting a replacement into that frame demand far more real-time adaptability than any contemporary robot can approach.
Of course, futurists can probably imagine a robot in a self-driving car that can navigate itself to your car, drive it to a garage, and order other robots to replace the windshield. But even that scenario depends on a chain of contingencies. When the stakes are higher — for instance, replacing a kidney instead of a windshield — then even more back-up systems and planning will be necessary.
Even if hospitals deploy robot surgeons en masse, the ultimate “back-up system” would be a skilled human surgeon with some experience, flexibility, and creativity. Our aim should not be to replace such individuals, but to aid in their efficiency and effectiveness. The sequence and shape of automation in healthcare cannot simply be dictated from on high by engineers. Rather, domain experts need to be consulted, and they need to buy into a larger vision of progress in their field. Perhaps more of medicine should indeed be automated — but let’s ensure that physicians themselves are lasting partners in that process. They should be helped, not replaced, by machines — both for the present (in order to override errant machines), and for the future (to develop new and better ones).
Expertise and Governance
The Susskinds address more than law and medicine. For them, education, architecture, journalism, management consulting, and even “divinity” all stand in automation’s crosshairs. They briefly grapple with concerns about empathy and other “soft skills” in professions, but they argue that such traits of mind and character can either be translated into algorithms, or can be stripped out of traditional professional roles. As with their perfunctory responses to other objections, their treatment of empathy and human interaction is “almost entirely an exercise in rational deduction and accordingly devoid of historical and cultural considerations,” as Howard Gardner explains in an insightful response to their work.
A persistent mistake undermines the The Future of the Professions. The authors conflate the professional role with the delivery of expertise. Thus they fail to seriously address two issues at the core of professional identity. First, there is some degree of self-governance among professionals. They primarily work with clients or patients, for example, and not for bosses or shareholders. Second, the main reason they enjoy this autonomy is because they must handle intractable conflicts of values that repeatedly require thoughtful discretion and negotiation. In isolation, these factors damage the Susskinds’ case; together, they prove fatal to it.
To start with the question of values: rarely, if ever, is a vocation simply a matter of conveying information. The duties of professionals do not end with an assessment of the relative likelihood of an intervention “working,” where “working” is defined by a clear, quantifiable metric. That’s most obvious in, say, elder care or education. A robot telling a shut-in elderly person, “Your friends and loved ones care for you” is not really a good substitute for visits. As for children in school, they require some guidance as to how to conduct themselves — as persons, not simply as absorbers and generators of messages. To think otherwise is to put society on a slippery Skinnerian slope to behaviorism.
There is no single measurement of success in the professions, and further complexities arise as soon as one lacks a single quantity or thing to be optimized. For example, attorneys, doctors, and teachers often face very difficult conflicts of values — between, say, zealous advocacy for a client, and acting as an officer of the court; or between extending a patient’s life, and assuring quality of life in the time remaining; or between attending to disruptive students, or simply ordering them out of the classroom to ensure others can learn better. We mark the importance of these decisions by insisting that a human be directly responsible for them. Routinized or robotized approaches do not respect the dignity of the client, the patient, and the student.
Admittedly, both government and corporate bureaucrats will insist that costs can be cut by summarily resolving such issues. (Far be it from them to consider raising taxes on the wealthy to pay for the poor’s access to human expertise.) The Susskinds’ book is rife with technocratic language, and one can imagine it being invoked by big data and predictive analytics firms who promise to replace all manner of human judgments with “data-driven” scores. Unfortunately, both the data and code for such proprietary systems is rarely available to those outside the firms selling them. Black-boxed as trade secrets, such systems defy challenge, debate, and correction, which hardly constitutes a step toward the democratization of expertise promised by the Susskinds. Rather, such proprietary systems constitute a regression to intellectual feudalism, where small groups of plutocrats arrogate to themselves ever more control over expanding areas of human experience. The Future of the Professions pretends to anti-elitism, but its attitude and program would be entirely at home in the drawing rooms of Davos, Wall Street, and Silicon Valley.
In the Susskinds’ hands, professionalism is too often misconstrued as the protectionism of guilds, dismissed as elitism, lambasted as a source of inertia. But sociologists like Eliot Freidson have observed that professionalism serves as a social and economic logic that tempers the worst excesses of free-market competition and cutthroat, Amazon-style managerialism. Professionals have been granted some degree of autonomy because they are charged with protecting distinct, non-economic values that society has deemed desirable. Their labor, in turn, reflects, reproduces, and is enriched by those values. Knowledge, skill, and ethics are inextricably intertwined. We cannot simply make a machine to “get the job done,” because frequently task definition is a critical part of the job itself.
Freidson lamented the frequent failure of professionals to “spell out the principles underlying the institutions that organize and support the way they do their work.” The Susskinds’ book deserves some credit for prompting professionals to better explain their roles and values. In the face of rampant and often reckless automation, the professions ought to reaffirm their own norms, highlight the importance of tacit skills and knowledge, and extend their status to other workers. The alternative is grim, and perhaps best described in Lawrence Joseph’s recent poem “Visions of Labour”:
… the question in this Third
Industrial Revolution is who owns and controls
the data. That’s what we’re looking at, labour cheap,
replaceable, self-replicating, marginal, contracted out
into smaller and smaller units. Them? Hordes
of them, of depleted economic, social value,
who don’t count, in any situation, in anyone’s eyes,
and won’t count, ever, no matter what happens,
the truth that, sooner than later, they will simply be
Joseph’s chilling lines are a sparer, more honest account of the politico-economic developments celebrated by the Susskinds. If we are to maintain a democratic society rather than give ourselves over to the rise of the robots — or to those who, like the Susskinds, bid them to rise — then we must spread professionalism from areas like law and medicine to information retrieval, logistics, elder care, marketing, and many other fields. Imagine a labor movement built on solidarity among those who realize that, like bad bosses, poorly implemented automation can crush human potential and innovation. If they succeed in uniting, they might project a vision of labor far more concrete and realistic than the feudal futurism of techno-utopians. They might foster automation that complements extant skills and labor, rather than accelerates a cheaper, faster, and more catastrophically unequal version of the present. Perhaps the Susskinds’ next book can explain how technology could advance the interests of all workers, rather than the narrow sliver of technologists, managers, and financiers likely to thrive in the future they commend.
When Pew polled hundreds of technology experts on the future of work, about half said that robots would render “masses of people […] effectively unemployable.” Guardian columnist Jess Zimmerman had a quick retort: “Stop predicting the future. Leave that to the robot who’s going to steal your job.” Concrete assessments of the real progress of automation in the professions confirm the wisdom of more sober voices. Computers assisting, rather than replacing, professionals is the norm, and will continue to be so in any recognizably humane social order.
Frank Pasquale’s research addresses the challenges posed to information law by rapidly changing technology, particularly in the health care, internet, and finance industries. His book The Black Box Society: The Secret Algorithms that Control Money and Information (Harvard University Press, 2015) develops a social theory of reputation, search, and finance.
Frank Pasquale’s research addresses the challenges posed to information law by rapidly changing technology, particularly in the health care, internet, and finance industries. He is a member of the NSF-funded Council for Big Data, Ethics, and Society, and an Affiliate Fellow of Yale Law School’s Information Society Project. He frequently presents on the ethical, legal, and social implications of information technology for attorneys, physicians, and other health professionals. His book The Black Box Society: The Secret Algorithms that Control Money and Information (Harvard University Press, 2015) develops a social theory of reputation, search, and finance.
LARB Staff Recommendations
Is America at risk of becoming Orwell's nightmare?...
Is AI a threat to humankind?...
Did you know LARB is a reader-supported nonprofit?
LARB publishes daily without a paywall as part of our mission to make rigorous, incisive, and engaging writing on every aspect of literature, culture, and the arts freely accessible to the public. Please consider supporting our work and helping to keep LARB free.