By Louis DiPietro

There is no scientific evidence of gender bias in letters of recommendation written for women applicants in the historically male-dominated field of experimental particle physics, according to new research co-led by Cornell.

In a massive computational analysis of more than 2,200 anonymized letters of recommendation written for applicants seeking tenure-track faculty positions in experimental particle physics, psychology and sociology, researchers from Cornell and the Fermi National Accelerator Laboratory found no significant differences in words used in letters written for female candidates versus those for male candidates.

“The general result was that women are not described as less extraordinary or brilliant than their male counterparts,” said Sterling Chance Williams-Ceci, a doctoral student in the field of information science and co-author of “Assessing Gender Bias in Particle Physics and Social Science Recommendations for Academic Jobs,” which was published in the journal Social Sciences in February. “In fact, we found that letters for each gender of candidate did not have many differences in language, and the few differences that we found usually favored women over men.”

The team compared gender differences in letters of recommendation across academic fields where women have vastly different representation. Along with roughly 1,000 letters from experimental particle physics, where women represent just 15% of field faculty, the team analyzed another 1,000 letters for tenure-track positions in psychology and sociology, two fields where women Ph.Ds outnumber their male counterparts. The inclusion of letters from psychology and sociology (the bulk coming from social psychology) helped provide the team with a control group or “ground truth,” said Williams-Ceci. 

Crafting lists of words associated with gender bias drawn from previous studies, the team parsed through every letter of recommendation and categorized word choices from the letters’ authors, noting positive and negative words (for instance, “upbeat” versus “dour”), standout and grindstone words (“brilliant” versus “strong work ethic”), and agentic and communal words (“a force” versus “team player”).

In terms of word choice used in letters for applicants in the field of experimental particle physics, there were very few differences in the types of words used for men and women applicants. Among those differences: In the social sciences, words like “commit” and “success” were more frequently used in letters for women applicants, while “science” and “technical” were more common for men; in physics, words like “notable” and “brilliant” appeared more frequently when describing women, while “talent” and “dedication” favored men. 

“Given the striking underrepresentation of women in [experimental particle physics], we were surprised not to find weaker letters for women,” the authors wrote. 

Nationwide, women remain underrepresented in STEM fields, accounting for just over a quarter of STEM workers, according to U.S. Census data. Researchers have investigated possible causes including hiring and promotion biases, differences in career preferences, and possible gender bias in letters of recommendation.

Cornell co-authors from the College of Human Ecology, Stephen Ceci, the Helen L. Carr Professor of Developmental Psychology, and Wendy Williams, a professor in the department of psychology and director of the Cornell Institute for Women in Science, have done extensive research into bias in letters of recommendation and have found little statistical evidence that women receive more unflattering letters of recommendation for STEM jobs than men.

Still, the research team felt a larger, more robust and definitive study was necessary to rule out gender bias in letters of recommendation as a possible culprit in the underrepresentation of women in STEM fields, Williams-Ceci said.

“We have improved on past studies in many ways: We used a much larger data set of letters than most studies and we used a variety of different techniques, and we still arrived at the same conclusions,” she said. “Given that, we think it’s important for practitioners in the field, and people trying to mitigate the problem of underrepresentation, to focus on studying other potential causes.”

Along with Williams-Ceci and her parents Williams and Ceci, paper authors are: Robert H. Bernstein (lead author) of Fermi National Accelerator Laboratory; Michael Macy, the Goldwin Smith Professor of Arts and Sciences in the College of Arts and Sciences, and Christopher Cameron, computational scientist at Cornell’s Center for Advanced Computing.

This research was partly funded by the National Science Foundation.

Louis DiPietro is a writer for the Ann S. Bowers College of Computing and Information Science.