It's already happening, the housing costs where I'm at have quadrupled in 4 years thanks to transplants from mostly CA, FL, and NY. There are barely any jobs here though, so they pay about half what they would in a big city while that half is about 20x the median wage here.

This leads to a vicious circle of them buying up even more property and converting it into short term rentals that are the equivalent of paying a mortgage that's 30x your salary.