Hehman et al. (2018)

DECISION: Major Error

that affects a core conclusion

Reviewer:
Will Lowe, Hertie School

Author response:
Eric Hehman, McGill
Jimmy Calanchini, UC Riverside
Jessica Flake, University of British Columbia

Recommender:
Ruben Arslan, University of Leipzig

Decision & Recommendation

Hehman, Flake, & Calanchini (2018) “Disproportionate Use of Lethal Force in Policing Is Associated With Regional Racial Biases” was determined to contain a Major Error that affects a core conclusion. Specifically, the conclusion in the Abstract – “Results indicate that only the implicit racial prejudices and stereotypes of White residents, beyond major demographic covariates, are associated with disproportionally more use of lethal force with Blacks relative to regional base rates of Blacks in the population.“ — is unsupported upon reexamination of the assumptions behind the applied model. An intuitive reading would lead the reader to think that the relationship between White people’s scores on the Black/White Racial Prejudice IAT and police killings of Black people cannot be explained through demographic covariates. However, the use of a difference of proportions variable as the regression outcome did not effectively adjust for the Black population base rate. A reformulated regression model that accounts for base rates more effectively finds no significant relationship between Black/White Racial Prejudice IAT scores and police killings of Black people. Therefore, the results for racial prejudice did not, in fact, hold “beyond major demographic covariates”.

However, the result that scores on the Black/White Weapons Stereotype IAT predict police killings of Black people was still significant after effectively accounting for the base rate of Black people in the population. As such, the detected errors rise to the level where I recommend that the authors seek a correction notice. Following the ERROR project’s emergent guidelines, the recommendations associated with this decision are as follows:

• The report, author response, and recommendation have been posted on the ERROR website (https://error.reviews/reviews/hehman-et-al-2018/) and as a preprint on PsyArXiv (https://osf.io/preprints/psyarxiv/a7jhu). Their associated materials have been posted to OSF (https://osf.io/abyqs/).
• The authors are asked to recognize these errors in future discussions of the article.
• The authors are recommended to pursue a correction with the journal. Specifically:
  • The abstract should be updated to reflect the corrected conclusion.
  • The title should be rephrased to better reflect the main findings.
  • Analyses should be corrected to use an explicit adjustment for the Black base rate, for example using the binomial model suggested by Dr. Lowe.
  • As analyses need to be modified, the authors should take the opportunity to be clearer about the data sources and how they dealt with inconsistencies between data sources.
    • Clarify which version of the Guardian lethal force database was used. The authors have noted that it is difficult to reconstruct what version they accessed — in a correction, they should take care to make this clear.
    • Use and describe a principled approach to handle mismatched core-based statistical areas (CBSAs).
    • Report code or syntax to make analyses reproducible.
• The sentences “Black people represented 22.76% of all deaths” and “In contrast, the percentage of White deaths (77.24% of all lethal force)” need to be corrected (the reported percentages are of Black+White deaths, not all deaths)
• p values and regression coefficients that are reported as .001 but are in fact smaller should be reported as < .001 or 0, respectively.

Cite as: ERROR. (2025). Error review of Hehman et al. (2018). Version 1. https://osf.io/preprints/psyarxiv/a7jhu

Full Report

Materials

PubPeer Page