Title: It’s Time to Eliminate the GRE and PGRE in All Astronomy & Astrophysics PhD Programs: Motivation, Implementation and Outcomes (APC White Paper)
Authors: Adam Burgasser et al.
First author’s institution: UC San Diego
Journal: Bulletin of the American Astronomical Society (open access)
Over the last few years, a wave of institutions have dropped the GRE as a requirement for graduate admissions. In 2016, the American Astronomical Society recommended that astronomical science programs remove the GRE, as well as its Physics counterpart the PGRE, as requirements. Nearly half of Astronomy PhD programs have now dropped the PGRE, including 9 of the top 20 programs.
However, many institutions are still hanging on to the exams, despite evidence that the tests are biased and are poor predictors of success. Since grading standards and research experiences can vary drastically between schools, some professors argue that the PGRE provides the only uniform, quantitative metric by which they can evaluate applicants. In today’s bite, we review a “State of the Profession” White Paper for the Astro2020 Decadal Survey which argues that the PGRE still does more harm than good, and provides guidelines for its elimination.
What does the GRE predict?
Many studies have shown that the GRE is uncorrelated with success. A 2015 paper found that PGRE scores were poor predictors of winners of prize postdoctoral fellowships (check out this astrobite for a detailed summary). A recent 2019 study of students in 24 programs from 2000-2010 found that the GRE and PGRE are not effective at predicting PhD completion (Figure 1). Specifically, PGRE and verbal GRE scores do not show a statistically significant correlation with PhD completion. Quantitative GRE scores show a weak trend, with both women and men in the 90th percentile having completion rates around 10% higher than those in the 10th percentile. Other studies (for example, see here and here) have found no correlation between GRE scores and research activity, time-to-degree, or first-author publications.
So what does the GRE predict? Multiple studies have found weak correlations between GRE scores and metrics like first-year graduate course grades and qualifying exam scores. There is also a small (10-20%) correlation with long-term publication outcomes. However, these effects are dominated by correlations with demographic information—race, gender, and nationality—and programs rarely have the time and resources to correct for this effect.
Many studies have looked at these demographic correlations. One that the authors cite is a 2014 study that analyzed the relationship between demographic data and scores on the quantitative portion of the GRE. This work found that women score lower than men, and that African American, Puerto Rican, Native American, Mexican American, and Hispanic students score lower than white and Asian students. The authors of the white paper connect these results to other research that attributes these disparities to deeper societal issues such as socioeconomic inequality, stereotype threat, and imposter syndrome.
Further, the authors argue that the high cost of the exams and access to test sites are significant barriers. Many students limit the number of programs they apply to, or don’t apply to US schools at all, due to these factors. This biases the admissions process against low-income students and international students.
So you’re thinking of dropping the GRE. What next?
Many institutions are responding to these findings and re-evaluating the use of the GRE and PGRE. However, the majority of Astronomy and Physics programs still recommend or require the PGRE, including nearly all Physics programs. Check out this spreadsheet for nearly 200 institutions’ GRE policies.
The authors of the white paper present a strategic plan for eliminating the GRE requirement from graduate programs. I’ve summarized the steps here; for more details on carrying these out, check out the paper.
1. “Develop departmental consensus on what the goals of your graduate program are.”
This will require open discussion within the department, among both students and faculty, around what aspects of student preparation and performance are valued. These values will inform any changes to the admissions process. The authors point to recommendations from the AAS Graduate Taskforce Study for how to reach consensus, which include performing assessments to identify areas of need and supporting professional development on issues of inclusivity.
2. “Recognize that the GRE and PGRE are not the best diagnostics of physics knowledge.”
Both the material that is tested and the format of the exam are not conducive to assessing the physics knowledge that students gain in undergraduate programs. Better metrics of physics preparation include advanced courses taken, research experiences, and even undergraduate GPA (see Figure 1).
3. “Make sure everyone has access to the research.”
As scientists, we should be consulting the relevant research before drawing conclusions—yet scientists don’t always do this when it comes to social science and education research. The authors suggest equity discussion groups as venues for diving into the literature.
4. “Evaluate your own program’s outcomes.”
It is worthwhile for institutions to consider which applicants they are cutting out due to low test scores. For example, UT Austin did a self-study in which they found that the distribution of PGRE scores of prize fellows was the same as that of their original applicant pool, not UT’s accepted students with higher scores.
5. “Make an advanced plan for modifying admissions criteria and processes.”
By removing the GRE requirement, departments will be expanding their applicant pools! On the logistical side, this means more applications to read. Consider engaging all faculty in reading a first round of applications, and including graduate students in the process. For consistency, use rubrics for evaluation.
6. “Clearly advertise your GRE/PGRE policy and rationale for it.”
Be transparent about the policy to applicants, and explain how the score is used if it is allowed. For example, the PGRE is optional for the University of Colorado Astronomy and Planetary science program, and they provide a statement on exactly what this means and how it is used if submitted. They also include the fact that “Physics GRE scores are biased” and that applications won’t be adversely affected by not submitting the PGRE.
7. “Accept that the transition away from the GRE/PGRE may be a multi‑stage process.”
Some departments may need to ease into dropping the GRE. One option is to start with “GRE-blind review”; this is being done at the University of Illinois at Urbana‑Champaign Astrophysics PhD program.
In the words of the authors, “As scientists, we should embrace evidence-based experimentation in developing admissions procedures.” That is, astronomers should be open to a re-evaluation of their procedures in light of research from other fields, such as education and social science. In dropping the GRE and PGRE, the paper concludes, departments are taking an important step towards equity in Astronomy and Physics.
