When should hate speech be restricted? As the prevalence of hateful political discussions increase, governments and tech companies face pressures to regulate social media platforms. Yet, hate speech is often used as a generic everyday term conflating the severity of the content with whom it targets, leading to a widespread perception that citizens cannot agree on what “hate” is. I offer a comprehensive test of when ordinary citizens want to restrict hate speech across two nationally representative samples in Denmark and the United States. I demonstrate that the public’s willingness to restrict hate speech almost exclusively stems from the severity of the content. Furthermore, I find little to no support for the partisan bias account, as differences in regulating hate speech stem primarily from sensitivity to severity. Across national differences and political lines, preferences for restricting hate speech is primarily shaped by the severity of the content, not who it targets or partisanship.