Just tell them "that's a slur for downs people" and they'll stop really fast with next to no resistance and all apologies. Its common use by brits is entirely accidental and not really reactionary. People here just genuinely have no idea is attached to anything bad. It's pretty much thought of like lampshade, wally or spanner in terms of severity, it's that casual because of the ignorance level and everyone is almost always happy to change when they're properly informed of why.