Mark Twain is famous for saying that a man with one watch always knows what time it is while a man with two never does. Unfortunately, when it comes to regional migration within the United States – one of the most important factors influencing local demand for everything from homes to schools and roads – there are at least three, sometimes dramatically different though all semi-official, data sources.
There are small differences in precisely what they measure and how they measure it, making drawing definitive conclusions about regional migration within the United States difficult to pin down. This already complex task has become even more difficult with the addition of various unofficial data sources on online search patterns, which are often referenced as a leading indicator of regional migration.
In two recent analyses we looked at the metro areas that are most attractive for home buyers and renters on Zillow looking to move from one part of the country to another. For housing markets, demand shocks – the increase in housing demand because of new residents moving into an area from elsewhere in the country (or the world) – are an important driver of price trends.
It has become increasingly popular to draw conclusions about national migration patterns based on online search trends, but it’s something Zillow has historically shied away from. Consumers’ online search behaviors often reflect aspirations or curiosities as much as any kind of actionable intent. Put another way: A Zillow user might look at homes in Seattle from their couch in New York not because they actually intend to move to the Pacific Northwest, but simply because they’ve always wondered what it’s like there or because they are viewing the home of a friend or relative. So while online search data certainly does contain some real, forward-looking signals, depending on the source, the signal is unlikely to be strong enough to merit extrapolating real-life trends without first modeling the relationship between the search data and some more authoritative truth.
On average, between 2011 and 2014 for the typical county for which we have data, the IRS estimates of net cross-county migration were off from Census official estimates by 417 people, a median absolute percentage error rate of almost 60 percent (table 1). For 17 percent of counties, the signs of the estimates were different, meaning one data source suggested the county experienced a net population inflow while the other suggested the county experienced a net population outflow. The error rates were higher when comparing official population estimates with estimates from the American Community Survey.
When comparing net household migration in the ACS to net taxpayer migration in the IRS data, the median absolute error rate between 2011 and 2014 was 1,050 households/taxpayers and in 40 percent of counties, the sign of the change in the two data sources was different.
The divergences between the trends implied by various data sources on internal movements in the United States suggest users should be cautious in drawing definitive conclusions from any single data source. Official data sources based on surveys and administrative records – much less online search data which are even noisier – can point to sharply different trends.