Hardcore gamer = someone who plays only cinematic grizzed white dude games and/or military fetishizing FPS
Casual gamer = anyone that is not a 15-25 yo male, and/or plays anything outside of the previously mentioned games, especially if those games are colorful.
So basically the gaming community is full of gatekeeping, misogyny, toxic masculinity and general chuddery. They make sure they're the loudest voice heard when anything about games is talked about, and won't be happy until all games a homogenous stream of bland, hyper-realistic but with a grey filter slog of mindless action with no heart or soul. And don't you dare force them to read any dialogue or story.
All mediated representations involve making arbitrary decisions, even if “reality” (whatever that really means in a game) is the goal. To continue torturing this metaphor: what kind of real ball and what kind of real cup are you simulating, and to what level of precision? These are choices and never have to be made just one way.
Of course you’re right, it’s not masked ableism to lose occasionally, that’s a normal part of properly adjusted difficulty. But it is quite another thing to make it impossible - that’s exclusionary and is probably not something that should be celebrated.