I think he's referring to OP's dad's brain as having patriarchy brainworms. Your take on the situation is a valid interpretation.

It doesnt necessarily say anything about patriarchy brainworms in your brain, but since we are all immersed in this shitty culture from birth, we all have them to some extent.