A whole lot of them chose not to live in the south or southwest, until AC arrived.
Hell, before that it was even thinner on the ground, population-wise, before ice machines where invented.
You kids...my forbearers lived in SW Florida, central Alabama, coastal South Carolina...200+ years...