If I'm being completely honest, I dont really believe we have free will. Or we do, but in an almost entirely meaningless way on a civilizational scale. While you might be able to make small choices, material conditions and the flow of history ultimately decide the course of your life. We're all just products of our environment which none of us can change on our own, and as much as we can change our environment as a collective is decided upon by current conditions which are determined by oast conditions, etc. Basically i think the entirity of history was determined at the big bang, and were all just along for the ride.
I think free will is a matter of perspective. At the largest scale, as you say, I think we're all just along for the ride. Or rather, if time is just another spatial dimension of the universe, then everything we do is pre-scripted.
But we don't and can't ever have the perspective of the whole universe simultaneously, so from our perspective we do have some degree of agency. We don't have 100% freedom to do anything at any time, but we can make decisions from among however many options are available to us.
I don't know if these perspectives are even necessarily in conflict. Like, if the pre-determined shape of the universe at exactly X point in space and time is you deciding that you want pizza instead of tacos, does that temove the meaningfulness of your decision? Does the inevitability of a decision negate the fact the someone had to decide it?
So i definitely think you've got a point, and foe the most part, I would say I agree. With that said, I end up having trouble ever really blaming people for their actions. Which is not to say that their actions dont still make me mad or upset, etc. How can I fault,for example, cis men for the patriarchy when I think there's an inevitability to might makes right that leads to the typically stronger and more aggressive group subjugate the other (that doesn't say I approve in any way), or Europeans for colonizing for the same reason. And even me being very strongly against both is some product of me being a short, weak, "feminine" man who has had many negative experiences with the patriarchy and a combination of guilt, empathy and access to leftist spaces due to my time that made me hate colonization. I also can't feel good about people doing good things (or at least things I would classify as good) because that's also outside of their control.
If we're talking about personal responsibility, one thing that's helped me work through how I feel about others' actions is de-coupling the actions from the person.
Like, many cishet white men are the way they are because that's the way their environment shaped them. That's not their fault, but it doesn't make their actions any less harmful. You can still condemn those actions without laying existential "blame" for the actions at any one person's feet.
Plus, if their actions are inevitable, then so is your response. Maybe you can't "credit" yourself and others for pushing back against patriarchy/colonialism, but you can still appreciate that said pushback had to happen. We're all just part of the push-and-pull of history, and I've found there's a contentment that comes with embracing that.