SEPTEMBER 13, 2015
THE SCHOLASTIC APTITUDE TEST was invented by an educational psychologist, Carl Brigham, in 1926, but did not catch on widely among elite colleges until the 1940s, when Harvard’s president, James B. Conant, began to take a serious interest in it. Conant wanted Harvard to serve the entire country, not just its traditional constituency. He wrote that the country needed a new kind of “American radical,” who disdained inherited wealth and truly believed in equality of opportunity. In 1943, he wrote that education would be responsible for “wielding the axe against the root of inherited privilege.” One of Conant’s deans, Henry Chauncey, had already successfully lobbied many elite colleges to use the test for admission of scholarship students. Conant encouraged Chauncey to create an educational testing agency to select the academically talented from a much broader pool than before. In 1948, the Educational Testing Service opened for business near Princeton University, with Chauncey as president and Conant as chairman of the Board. Neither used the term “meritocracy” to describe the group they hoped to identify — indeed, the term would not be coined for another 10 years — but that is the term, for better or worse, that came to be identified with their aspirations. The SAT became their vehicle.
The SAT attracted critics almost from its inception. In 1980, two Harvard professors, Warner Slack and Douglas Porter, published an explosive attack on the SAT in the Harvard Education Review, which set the tone for the criticisms that have been leveled again the test since that time. Slack and Porter made two important demonstrations that cut to the heart of existing justifications for the test. First, they showed that the SAT lacked predictive validity: in other words, it failed to predict the likelihood of a student’s success in college. It contributed little above high school grades and high school rank to the prediction of freshmen grades, and even less to the prediction of subsequent academic performance. Second, they showed that coaching could raise scores on the test by as much as 75 points. The ETS had steadfastly denied that coaching could improve performance, but as study after study piled up in support of Slack and Porter’s findings the company eventually relented. This admission eventually led, in 1993, to the “A” in SAT being relabeled “Assessment,” rather than “Aptitude.” (In 1997, the ETS announced that the SAT could not properly be labeled a scholastic assessment test, either; the initials now stand for nothing.)
Slack and Porter’s work helped to reignite the long-standing populist suspicion that the SAT favored the wealthy. It seemed to follow from their analysis that only the affluent had the knowledge and resources to send their children to test prep courses. In 1988, the sociologists James Crouse and Dale Trusheim launched a second cannonade against the SAT by showing that test scores were indeed correlated with family background. For individual test takers, they found moderate correlations between family income and SAT scores. Correlations with race were higher, suggesting the possibility of a stronger bias against minorities.
There’s another way to look at the relationship between family background and test scores that makes the relationship look much stronger than Crouse and Trusheim found. That is to divide families into income groups: for example, the group of families whose household incomes are $180,000 to $200,000, the groups of families whose incomes are $160,000 to $180,000, and so on. When family incomes are broken into broad strata like these, the correlation between family income groups and average SAT scores is sky high; there is a linear progression in average SAT scores as you move up income strata. Indeed, for every increment of $20,000 in income, The New York Times reported in 2012, students could expect on average a 12-point gain in their cumulative verbal and math score.
Around the same time that Crouse and Trusheim were arguing that the SAT was rigged, two celebrated psychologists, Robert Sternberg and Howard Gardner, went beyond statistical fault-finding. They denied that the SAT or any other test of academic aptitude could adequately identify the talents required for contributions to society. Sternberg’s theory, the more convincing of the two, focused on a model of intelligence linked to accomplishing things in the world. According to Sternberg, an intelligent person showed three qualities: the capacity for knowledge acquisition; the capacity to execute effectively on knowledge acquired; and the capacity to plan, monitor, and evaluate executive actions to improve future performance. Gardner’s rococo theory of multiple intelligence suggested as many as 10 domains of “intelligence,” including musical intelligence and kinesthetic intelligence.
Since 2005, several dozen selective colleges have adopted “test-optional” policies. The professed reasons for these changes in policy are concerns about the validity and fairness of the tests. (In some notable cases, such as Sarah Lawrence and Bennington, the change may have been motivated as much by the colleges’ need to attract more full-paying, but relatively low-scoring students.) Yet if we look at elite institutions as a whole, it is clear that the ETS fortress remains as imposing as ever. If anything, doing well on the SAT has become more important over time. Among the top 25 colleges and universities, average SAT scores are above 700 on each of the three tests (including the now-optional writing test). Students with 700 scores on the SAT are in the top three percent of test takers. Indeed, U.S. News and World Report’s top 25 national universities and top 25 liberal arts colleges can be predicted with good accuracy based on one factor alone: the average SAT scores of entering freshmen. Ambitious university presidents know that they are well advised to raise the SAT scores of their entering classes if they want to move up the ladder of academic regard. Between 1990 and 2001, the University of Southern California registered an increase of 240 points in the average SAT scores of its entering freshmen, and finally began to shed the nickname “University of Spoiled Children.”
In her recent book The Tyranny of the Meritocracy, Lani Guinier does not rehearse many of the criticisms of the SAT. Her heart is in finding an alternative to it. But there is one flaw in the “testocracy” (her term) that she finds particularly galling: the relationship between family economic resources and test scores. “SAT scores,” she writes, “are accurate reflectors of wealth and little else.” If we look at individuals, rather than broad income strata, this is a gross overstatement. One reason that the individual-level correlation is not higher is that some groups whose members are not wealthy do much better than they should on the tests. Many are the sons and daughters of East and Southeast Asian immigrants. It is true that immigrants from that part of the world often start out with greater educational advantages because of the importance placed on education in their homes. But they are also the ones who take the idea of improvement seriously. Their parents often speak little if any English. Like athletes who don’t mind drilling to improve their performance on the field, these disadvantaged students don’t mind drilling to improve their performance on tests and in the classroom. This is one reason why many of the best test prep schools on the West Coast are filled with children of Asian immigrants, with hardly a white or black student in sight. Sons and daughters of Caribbean immigrants also do better than we might expect on the tests. These groups follow a long line of immigrant strivers, beginning with the Jews who did so well on campus-based admissions tests in the early 20th century that Ivy League institutions set up quotas to protect themselves from “applicant flight” among their traditional WASP Eastern seaboard constituency.
The heart of Guinier’s argument is that “test merit” should be replaced by what she calls “democratic merit.” The definition of democratic merit is not consistent throughout the book, but it includes at least the following qualities: leadership, collaboration, resiliency, and the drive to learn. She writes that these, not SAT scores, are characteristics “that indicate a student’s potential for future success in our democracy.”
Although she rails against the testocracy, it turns out that Guinier may want to use a different test to discern these qualities. The test she has in mind (she calls it a “tool”) is the Bial-Dale Adaptability Index, or BDI, co-created by Debbie Bial, the founder of a non-profit organization, the Posse Foundation, that identifies students from disadvantaged backgrounds who can succeed in selective colleges. The BDI evaluates students in groups of 10 to 12 on several collaborative exercises. One exercise asks students to study a robot built of Lego blocks and then try to reconstruct it based on sharing their collective memories. Others ask students to run an impromptu discussion on a controversial topic, such as genetic testing, or work together to create a public service announcement. The goal is to identify those students who take initiative, collaborate well in the completion of the task, and are persistent.
The Posse Foundation claims that students who score high on the BDI tend to thrive in college and “contribute to the campus community.” Students scoring high also achieved better GPAs than students with low scores. Other organizations interested in identifying students who have the potential to be leaders in public life use similar group interaction procedures. One such organization is the Coro Foundation. Through a surrogate with whom she obviously agrees, Guinier claims that these are the kinds of people who make a difference in the world, whether they receive As or Bs in their courses: “What’s more important, receiving high grades or making a difference in the world?” she asks.
Guinier shows no aversion to dichotomies. One such dichotomy is that elite college students and collaboration do not mix. It would be interesting to see how students who are not from disadvantaged backgrounds perform on the BDI. Guinier likes to think that the wealthy kids who do well on the SAT are individualistic and hyper-competitive. My sense is that the extroverts among them, at least, could ace the BDI, and probably without coaching. Through their review of applicants’ extracurricular and volunteering experiences, elite colleges already build in many indirect indicators of the qualities Guinier admires. The difference is that the colleges also expect students to do well on written tests of reasoning.
In her emphasis on students’ ability to contribute to problem-solving groups, Guinier is aligned with the zeitgeist. Colleges have entered the Age of Collaboration. We can see this in the rise of group projects in undergraduate courses. We can see it in the craze for interdisciplinarity. We can even see it in the design of academic spaces, no longer emphasizing crowds of individuals staring at a teacher, but rather work tables around which teams of students put their heads together to solve problems. Among college administrators, I now frequently hear the phrase “collaborative leadership.”
In the Age of Collaboration, will low-ego problem solvers inherit the earth? There are many young people out there who have generous spirits and are good at knitting together groups. These people should gain a bit more prominence in the Age of Collaboration. I would personally welcome such a development. But individualism and collaboration are not the opposed concepts that Guinier believes them to be. Indeed, research I am conducting with Allison Cantwell suggests that elite college students may already be among the most avid collaborators in the land. Elite colleges typically sponsor one student organization for every 10 to 15 students. This compares to roughly one student organization for every 20 to 40 students in the leading public universities, and one student organization for every 40 to 60 students in state colleges. If collaboration becomes the order of the day, elite college students will likely find a way to stand out as collaborators. It’s not a tough adaptation for these kids.
Guinier expresses another belief that may be more central to her overall argument. It is that diverse groups are better at solving problems than homogeneous groups. “Today’s world demands problem solvers,” she writes. “Groups outperform individuals in problem solving, and…diverse groups — with diverse approaches — will yield the best solutions to modern dilemmas.” A literature has developed in social psychology that supports these contentions, and she cites some of it. These studies suggest ways diversity can contribute to group problem solving: it can lead to a wider range of fruitful perspectives; it can be an antidote to group think; those who have had challenging life experiences might be more prone to encourage inclusivity and therefore wider participation.
I personally believe that organizations should field diverse problem-solving groups as a matter of principle, whether or not they are more effective, and then help them to be effective. But the literature itself does little to convince me that diverse groups yield better outcomes. Many of the studies are conducted in laboratory settings with artificial problems that can be solved over the course of an experiment. In organizational life, problems often take a long time and a lot of persistence to solve. To tackle them, groups need leaders to organize and to take responsibility for performance. They need a reasonable scope. They need a division of labor. They need to collect good information and subject it to critical analysis. They need to have sound communications. The leader needs to follow up to make sure that responsibilities are being addressed in a timely way. They need a sense of mission. Groups often profit from people who can provide what organizational psychologists call “socio-emotional leadership” to complement task leadership. Most of all they need people who have really good ideas and the depth of commitment to realize them against resistance. Some diverse groups have these ingredients; some homogeneous groups do, too. Many groups, diverse or not, seem to trip over their shoelaces because they lack these basic principles of organization. When it comes to problem solving in scholarly and scientific circles, the presence of individuals who have the spark of imagination and the determination to follow through is, it appears, the most important factor.
A statistical average always leaves plenty of room for exceptions. In the case of college admissions, this includes exceptions for savants who score relatively low on the SAT but are gifted in some other way. Admissions officers recognize this and, indeed, many students who are admitted to elite colleges would not be admitted on their SAT scores alone. Colleges need to field athletic teams, orchestras, dramatic productions, and campus literary magazines. They also need to show that they are at least somewhat socially conscious by admitting a respectable proportion of students from under-represented minority (URM) groups. That portion has grown over time; it now represents about one quarter of the class at many Ivy League institutions, or roughly the same proportion of URMs as in the population at large. Students admitted under this elaborate system of exceptions and preferences are perceived to contribute to campus life or to the mission of the college or to the long-term stability of the institution (as in the case of family legacy admits). The most disadvantaged groups in elite college admissions are not URMs but students from low-income families; on most elite campuses fewer than 15 percent of undergraduates come from the bottom half of the income distribution. Partly that is because students from lower income backgrounds do not consider themselves competitive for admission to elite institutions, and partly it is because they lack the credentials, including the test scores, to be admitted.
Elite colleges are not meritocracies, then, if by meritocracy we mean that students are admitted solely on the basis of high test scores. In this respect there is no “tyranny of the meritocracy.” Most students at elite colleges score very well on the SAT, but that alone is not qualification enough to get in. They usually also show a raft of extracurricular activities — and, more important, effective follow-through on big projects. They show that they have concern for their fellow men and women by volunteering for good works in the community. They must pass interviews with alumni, demonstrating an adequate level of interpersonal skill in a charged environment. Ivy League institutions no longer take nude photographs of admitted students to prove that they are healthy physical specimens, as they did in the 1950s, but participation in varsity athletics is usually a plus factor for something like the same reason.
Even if we allow that admissions offices are not test score fetishists, the fact remains that only high school grades and class rank count as much. If the SAT can be coached, lacks predictive validity when compared to high school grades, and unfairly privileges the already-privileged, why then does it remain so important? My guess is that it is because of the faculty (reinforced, to be sure, by the U.S. News and World Report rankings).
Faculty members at elite college are, in the great majority of cases, hyper-intelligent and very hard-working people. They naturally want to be surrounded by other such people, both as a symbol of their distinctiveness and perhaps also for intellectual stimulation. Certainly they want the average level of performance of their students to be high. Teaching undergraduates at all is infra dig, as far as many are concerned. I recall the first words spoken in class by my one of my professors at Berkeley: “I am chair of the leading sociology department in the country. Can any of you tell me why I should be required to teach introductory methods to undergraduates?” As far as I know, no one has examined the SAT scores of tenured faculty members at elite colleges, but I am willing to bet they are high. What do these tests test, after all? They test the ability to study questions carefully, to absorb complex information, to analyze this information quickly, and to come up with sound answers. That is exactly what highly productive faculty members do every day of the year when they are at work.
Does it matter to elite faculty members that some level of efficacy in these activities can be coached? Not really. After all, successful coaching is also a measure of the coached person. Students do learn a few skills in test taking such as quickly eliminating clearly wrong answers and questioning why the seemingly obvious right answer might be wrong. But most of all they practice and are corrected, practice again and are corrected, and practice again. In coaching the question is: do students have the desire to dig in, even when they would rather be doing something else, and practice, practice, practice until they are nearly exhausted in order to improve their performance? These bouts of mental exertion resonate with the habits of elite faculty members, who push themselves in similar ways nearly every day of their lives. Coaching for tests is not that different from coaching for athletic contests. Yes, there is correction of technique and pointers about gaining a competitive edge, but much of it is drilling to the point where the awkward becomes tentative and the tentative becomes assertive and automatic. One of the arresting features of our supposedly “knowledge-based” post-industrial society is that many young people will spend hours and hours on the playing field honing their athletic skills, but won’t spend so much as an hour a day practicing their academic skills.
Does it matter to elite faculty members that the SAT tests lack predictive validity? I doubt it. Their goal is more likely to be identifying someone like themselves, and they don’t expect or hope for more than a handful of such people in any given cohort. If a test can help identify students who have the capacity to do outstanding work, however infrequently that promise is realized, the identification problem is considerably reduced. The questions become more focused: Does the person also have the drive for excellence? Does s/he have the resilience to overcome frustrations? Can s/he stand the isolation of research and writing, or does s/he instead thrive in it? Studies of predictive validity are looking for the power to explain variation in samples of ten thousands. Elite faculty are looking for singularities — individual students who remind them of themselves at a younger age. Those are utterly different projects.
Does it matter to elite faculty members that the tests are biased in favor of the affluent? Now we come to the rub. Professors at elite institutions have embraced the idea that social inclusiveness is part of the mission of undergraduate education. Affirmative action is the compromise they have accepted to bridge their elitist cultural interests and their liberal political preferences. Women outperform men in secondary school, and the incorporation of women, from a strictly academic point of view, has become a moot issue outside of the quantitative fields. Racial incorporation is, of course, another story. The current compromise is to put a thumb on the scales in favor of diversity, provided that candidates are able to show other outstanding characteristics. This has become a win-win situation for candidates and institutions. Most studies show that under-represented minorities gain more intellectually and economically from an elite college experience than other students do.
So much for what elite professors want from their students. A more fundamental question remains: If you want to be a leader in American society, how much does it really matter where you go to college? A book like Guinier’s assumes that admission to an elite college is very important, because elite colleges produce (or reproduce) the next generation of society’s leaders. She writes, elite colleges provide “enormous opportunities” and are oriented to “reaching the top of [the] hierarchy.” Or, as the old Harvard cheer goes, “That’s alright. That’s okay. You’ll all work for us someday.”
It turns out that Harvard’s athletic competitors may be working for graduates of the Big Ten. Sarah Yoshikawa and I recently conducted a study of more than 3,400 senior executives in business and government, the leaders of Fortune 1000 firms in 12 industrial sectors and top state and federal government officials. We found that about one in 10 had been educated as undergraduates at the 40 private college campuses that have dominated U.S. News and World Report rankings since the 1980s. That’s about four times better than chance, but it’s a long way from the common perception that elite colleges produce most of the country’s future leaders. In fact, in absolute numbers the second largest producer of future leaders (after Harvard) was the University of Illinois at Urbana-Champaign. The University of Michigan in Ann Arbor placed fourth (after Stanford). We found larger proportions of elite college graduates at the top of a few industries — notably, finance, internet services, entertainment, and government (in the latter case only when we did not count the House of Representatives) — but no more than a scattering in others. The top producers of executives in the energy sector, for example, were the University of Texas, Texas A&M, Colorado School of Mines, and the University of Oklahoma. Elite college grads were also rare in the aerospace, chemical, construction, and health care industries. We see the outlines here of a divided ruling class: elite colleges are fairly prominent in the symbol-producing industries, but public universities supply much of the leadership in industries that manipulate the material world. (This could be because they offer more undergraduate majors in business and engineering fields.)
There’s also the likelihood that elite colleges have a more sizable market share in parts of the occupational structure outside of business and government. I would not be surprised if a large portion of America’s cultural elite — I mean the top sliver of scientists, economists, policy people, writers, editors, directors, composers, journalists, pundits, gallery owners, museum directors, philanthropy executives, and, above all, professors — came from schools that select on the basis of written tests. These schools, after all, are home bases to members of the cultural elite; you would guess they would be good at identifying promise in the next generation.
But the assumption that attendance at a prestigious college or university is all but essential for entry into the American business and government elite fails to take into account the very different selection principles at work in elite college admissions as compared to top positions in business or government. Elite college students are selected for their capacity to master complex symbolic media, as indicated by scores on standardized admissions tests. They are also selected for their conscientiousness in their studies, as indicated by secondary school grades. They are rewarded for outstanding cultural and service accomplishments, more than for their promise as entrepreneurs and managers.
A different set of experiences and skills is relevant to success in corporate life. These include a strong interest in pecuniary matters; taking on, and succeeding in, big, visible organizational projects; creation of value in units for which one is responsible; impressing one’s superiors; well timed career moves; and careful maintenance of networks with other upwardly mobile executives. Personality characteristics such as extroversion and risk tolerance have been found in some studies to be correlated with success in business. To put it mildly, risk tolerance has not been established as a common characteristic of students who attend selective undergraduate colleges.
In truth, elite college attendance does not matter that much. And it’s not just because top corporate and government positions are open to those who graduate from non-elite institutions. Among the predictors of economic success in early career, graduation from an elite college is much less important than completion of a demanding quantitative major or a graduate degree. (Grades, interestingly, don’t seem to matter at all.) Two very clever economists think even the small contribution that elite college graduation makes to the prediction is overstated. Stacy Dale Berg and Alan B. Krueger have argued that the better predictor of success in life is the highest-ranking college to which one applied and was rejected. If they are right, having the confidence (and the baseline credentials) to apply to an elite college is more important to later life success than whether or not one is actually admitted.
In any event, the action in leadership preparation is shifting to the graduate and professional schools. Attending Harvard Business School is already much more important as a gateway into the executive suite than attending Harvard College as an undergraduate. More than 20 percent of the executives in our study attended a narrow band of top business or law schools, twice the proportion who attended an elite college as undergraduates. Even so, the most common path to the top was the humblest one: one quarter of the execs attended a non-ranked undergraduate institution and obtained no graduate degree. Another quarter graduated from a non-elite undergraduate institution and subsequently from a non-elite business or law school. In other words, more than half of these corporate and government titans had no experience of an elite higher education institution, either as undergraduates or as graduate students.
Why, then, pay eight Toyotas worth of tuition, room and board (not to mention uncounted thousands of dollars on cultural enrichment and test prep) and buck 35,000 other applicants in the scramble to send junior to the Ivy League? There’s a lot of pride involved in this, no doubt, but I think it has mainly to do with insurance. Highly selective private colleges and universities are less important for their role in producing future leaders than they are for providing insurance against falling out of the upper middle class. Many of the advantages they offer are geared toward producing this result. The great majority of admits are very able students. The campuses foster strenuous academic and co-curricular competitions. They provide ample opportunities to develop interpersonal and communications skills as well as valuable cultural capital. They open access to alumni networks, and they offer the imprimatur of a well known and respected institution. Large public flagship universities, like Illinois and Michigan, enroll four or five times as many undergraduates and, almost by definition, cannot provide the same level of opportunity for a push ahead into the upper middle class or the same insurance against falling out. Elite colleges are far less likely to advertise higher education as a strategy for shoring up students’ prospects for remaining in the upper middle class than they are to say they are training the next generation of leaders, but that is what, in most cases, is really happening.
If elite colleges want to improve their performance in leadership development, they will want to look at the reforms adopted 15 years ago by the engineering profession, because these represent an intelligent approach to rethinking preparation. The Accrediting Board for Engineering and Technology (ABET) looked at the qualities that seemed lacking in engineering training. Technical preparation seemed adequate, but engineers were frequently criticized for poor interpersonal skills, limited acceptance of different perspectives in problem solving, and weak communication skills. ABET set about to augment the strong parts of the engineering curriculum with required courses and experiences that compensated for these frequently cited weaknesses. Of course, many engineers did not want to change, but evaluations of the new engineering criteria suggest that they have made a difference. Their schools are no longer happy to produce mere technicians, however well trained. Something similar is afoot in medical training.
This suggests that, if elite colleges are unhappy with the outcomes of the educational experience they are providing, they will want to make a serious assessment of the weaknesses they observe in their graduates. Guinier’s book could be a useful resource in such an assessment, because of its emphasis on measuring a student’s potential for contributing solutions to national problems. But it won’t be the only source for thinking seriously about how to improve admissions. Elite colleges will also wonder why public flagship universities are successful in producing future leaders under much more financially constrained circumstances. There may be something in the exposed, bustling, research-oriented environments of public flagship universities — or the curricula they offer — that will prove to be of interest. They may also want to think even more than they currently do about how to detect sparks of imagination among teenagers who also have the discipline and drive to turn their ideas into productive actions.
But it may be, too, that leadership preparation will never be as central to elite college education as we assume it is, given the variety of pathways to the top in American society. The comparison with France is instructive. In France the top 7 to 8 percent of secondary school students study very long hours for two years at government expense to prepare for the concours examination, an examination that is the sole arbiter of entrance into one of the grandes ecoles. The grandes ecoles in turn prepare virtually all of the top leadership in French society. Something like this system would be necessary to reproduce the strong connection found in French society between elite colleges and top executive positions. I say, Vive la difference!
Because elite colleges are not as important to the production of future leaders in the United States as they are in France, it would make sense to stop obsessing about the allocation of spots at Harvard and Yale and focus instead on finding ways to support and invigorate our underfunded community and state colleges. That is an exacting national problem to solve, worthy of those who have democratic and test merit alike.