What is in a Name? Additional Sources of Bias in Citation Counts

8 March 2022, 0930 EST

Academics are rightfully concerned about gaps in article citations due to factors that have little to do with the quality of published research. There has been significant consideration of how gender bias has crept into citation counts and if this has improved over time. Less attention has been paid to other types of bias, although there has been some work on how factors such as race may influence citations.

An influential article in International Organization, “The Gender Citation Gap in International Relations” (by Maliniak, Powers, and Walter), shows clear, gendered citation patterns in leading international-relations journals. Such biases should certainly trouble us if citation metrics matter to important decisions about hiring, tenure, and promotion. Maliniak, Powers, and Walter contributed to essential discussions within the scholarly community about how to mitigate bias against women; the article has sparked interest in how other types of overt or implicit discrimination have impacted certain communities. Nonetheless, we still know comparatively little about how citation gaps affect other groups.

Simply having an uncommon (that is, foreign-sounding) name has a profound effect on citation counts

To address this, I examined the replication data from “The Gender Citation Gap.” Although these data are somewhat dated, I believe they are still instructive – at least for a preliminary analysis. Using information from the United States Social Security Administration, I created a variable for whether or not the name of the first author is among the 100 most common names for men and women in the United States over the last century. Common male names include Andrew, David, Matthew, and William, while common female names include Anne, Catherine, Kelly, and Suzanne. Uncommon names may simply be American scholars with unusual names (e.g. Bear), but are most typically foreign or foreign-sounding – Aseem, Chien-Pin, and Jurgen for males; Birgit, Mlada, and Xinyuan for females. In the Maliniak, Powers, and Walter data, 69% of articles have a first author with a common name, while 31% include uncommon names, reflecting in part the dominance of US academics in leading international-relations journals.

Simply having an uncommon (that is, foreign-sounding) name has a profound effect on citation counts. The mean number of citations per article with a first author with a common name is 29.5; that number plummets to 17.7 for those with an uncommon name (P<0.000 in a t test). For reference, articles by male first authors – regardless of name origin – have a mean citation count of 26, while those by female first authors have a mean citation count of 22. In other words, the gap for name origin (~12 citations) is three times larger than the gender gap (~4 citations). Looking only at the male sample reveals broadly similar results (30 citations/article for common male names vs 17.6 citations/article for uncommon male names); the gap is slightly improved among the female sample (24.8 citations/article for common female names vs 17.8 citations/article for uncommon female names).

NMeanStandard Deviation
t= -6.56
Table 1. T test of citation counts by common (US) first name of first author.

Scholars with non-standard anglophone names are more likely to work outside the United States, and thus be less integrated into U.S. academic networks. If this is the case, then we’re not looking at some kind of bias against scholars based on their names.

To test this possibility, I examined only those articles written by scholars based at a U.S. institution. The gap still holds: articles by scholars with foreign-sounding names receive on average nearly 10 fewer citations, a statistically significant drop.

Most scholars have good intentions and do not always recognize their own subtle biases

Every way that I parsed the data – and even when I used the full set of controls included in Maliniak, Powers, and Walter – I found that having a typically-American sounding name is associated with more citations per article. In substantive terms, this foreign-name effect is larger than the gender gap in citation counts.

So what may account for this difference?

  • We cannot rule out simple discrimination. Scholars may see colleagues with foreign-sounding names as less competent or intelligent – to pass it over due to an implicit (or explicit) notion that it’s substandard or that its English prose is poor.
  • Foreign scholars, and those who are immigrants to the United States, may lack access to academic social networks due to linguistic or cultural barriers. If friends cite friends, then exclusion from social activities — such as lunches or drinks at conferences — may hurt these scholars.
  • When writing literature reviews, anglophone scholars may hav an easier time recalling American-sounding names. For example, It may be easier for some to remember a name like “James” or “Amy” than it is to remember a “Seonjou” or “Mehrzad.” While this source of bias is less pernicious than some alternative explanations, it still has negative consequences for scholars with unusual names.

Whatever explains this gap, “typically-American” scholars seem to have a clear advantage when it comes it citations counts in the field of international relations. This advantage likely holds in other fields as well.

I’m not advocating that we should ignore citation counts or other quantitative metrics of scholarly impact. In many respects, subjective evaluations of scholarly merit are more prone to bias and discrimination. Instead, we should evaluate citation counts, H-indices, and the like alongside other pieces of information in order to develop a wholistic picture of a scholar. More importantly, I believe that most scholars have good intentions and do not always recognize their own subtle biases. By becoming more aware of implicit notions of merit or social acceptability, we can hopefully take active steps to foster a more inclusive academic community.