Why GRE scores might 'matter'

In a recent meeting I attended at Duke, the argument was made that we should abandon using GRE scores in our graduate school admission decisions (or at the very least allow departments to opt out of requiring GRE scores for graduate school applications). The reason for this is the now well-known finding by the Graduate School that there is no statistically significant relationship between GRE scores of incoming graduate students and their ultimate performance. After the meeting, however, I began to wonder whether that finding is not leading to more confusion rather than clarity.

I have no reason to doubt the extensive analysis conducted by the Graduate School and will simply take as given that the data reveal no apparent relationship between GRE scores and performance for graduate students at Duke (or at any other school, for that matter). Nevertheless, I believe the conclusion that this means that GRE scores should not factor into admissions decisions (or are somehow currently being overemphasized) may be deeply flawed. In fact, if admissions committees are using GRE scores as only one of many factors considered in admission decisions (as the Graduate School correctly urges departments to do), we should expect the ex-post data to show precisely no relationship between GRE scores of admitted students and their ultimate performance regardless of whether the GRE scores themselves predict graduate school performance.

The argument is subtle and most easily seen in a stylized example that generalizes to the more complex settings most departments face. Suppose, for instance, that GRE math scores are strong but noisy indicators of math ability which is in turn a strong indicator of the ability of a first year graduate student in economics to make it through an economics PhD. program. Of course there are other traits—such as motivation, creativity, perseverance, more general conceptual thinking skills, knowledge from particular classes taken as undergraduates and so forth—that also matter a great deal. A good admissions process takes all these into account when making admissions decisions even though some of these factors are difficult to quantify and require nuanced judgments on the part of faculty.

These considerations may then lead a department such as mine to admit someone who scored a perfect 800 on the math GRE even though the rest of the application showed some weaknesses elsewhere while at the same time admitting someone that scored a 700 on the math GRE but showed great strengths elsewhere. My department probably will not be able to enroll someone who scored both an 800 on the GRE and showed no weakness elsewhere because that student will end up going to a more highly ranked department. Thus, we end up (in “equilibrium,” as economists say) admitting a mix of students—some having very high GRE scores but less promising indicators elsewhere and some having lower GRE scores but more promising indicators elsewhere.

If my department makes just the right tradeoffs between GRE scores and other parts of the application, then the student who scored an 800 on the math GRE will be exactly as likely to succeed as the student who scored a 700 on the math GRE. Thus, after we observe both types of students go through our program, we should expect to find no statistical correlation between GRE scores and performance even though GRE scores themselves have a clearly positive role in the admissions process. The fact that there is no statistical correlation between scores and performance for those that are enrolled therefore has nothing whatsoever to say about whether GRE scores “matter.”

Were we not to have access to the math GRE score, we would end up rejecting students whose high GRE score tells us something positive about them even though the rest of their application looks less impressive. Often these would be students who come from undergraduate institutions that are not well known to us and for whom it is difficult to signal that they might in fact be very good graduate school candidates. More privileged students from well known undergraduate institutions like Harvard or Princeton would then have a much easier time getting our attention than less privileged students that possess talents they are not easily able to demonstrate within their undergraduate institutions.

I therefore applaud the Graduate School’s efforts to urge departments to consider applications in their entirety and not to make arbitrary cut-off decisions based on GRE scores. I am skeptical, however, of the notion that departments around the country have all somehow been duped by the creators of the GRE into placing too much value on the information contained in GRE scores. These scores are likely to matter a lot in some departments and very little in others—but there is no way to tell for which departments they matter from the data analyzed by the Graduate School. In fact, the very data the Graduate School is using to suggest that departments are making irresponsible use of GRE scores may in fact suggest that departments are using these in exactly the way the Graduate School suggests—and that abandoning GRE scores would make the process more arbitrary and less fair.

Thomas Nechyba is chair of the Department of Economics.