Top Seven Reasons the Vanderbilt Poll Is Fatally Flawed

 

There are at least seven reasons why the Vanderbilt Poll-Tennessee released on Tuesday by the Center for the Study of Democratic Institutions, co-directed by Vanderbilt Professor John Geer, is fatally flawed.

Those flaws come as no surprise to conservatives across the state.

“The Vanderbilt Poll has notoriously leaned left for many years in the eyes of most Tennessee conservatives,” The Tennessee Star reported on an earlier Vanderbilt Poll released back in April, and the latest poll is no different, local conservatives say.

Among its counter-intuitive findings was this cleverly worded claim:

Support for a path to citizenship for undocumented employees is the highest it has been since the poll’s inception: up to 56 percent. And that figure is not just driven by Democrats—40 percent of Republicans favor it too. An additional 20 percent favor the establishment of a guest worker program.

Even stronger is support for helping teens of undocumented parents raised in Tennessee and who attend a Tennessee public university. Two-thirds of Tennesseans say they deserve to be eligible for in-state tuition at public colleges.

“It is very obvious that Vanderbilt had the result they wanted and polled until they got it,” Tennessee conservative activist Judson Phillips, Founder of Tea Party Nation, says of the most recent Vanderbilt Poll.

“This is as much propaganda as anything you would read in the North Korea Daily Worker,” Phillips adds.

“This poll has the same legitimacy and accuracy as the one they did last October when their crack team of pollsters had Donald Trump leading Hillary Clinton in Tennessee by 11 points. He won by 26!,” political and media strategist Steve Gill tells The Star after reviewing the May 2017 Vanderbilt Poll data.

“If the methodology and political acumen displayed by this poll reflects what they are teaching at the Vanderbilt political science department, then a generation of political operatives will build successful careers running campaigns against Vandy grads,” Gill points out.

Conservatives around the state, in addition to Phillips and Gill, note the Vanderbilt Poll has had a consistently poor track record since it launched in 2011.

The May 2014 Vanderbilt Poll, for instance, reported “U.S. Sen Lamar Alexander holds a commanding lead in the upcoming Republican primary election.”

U.S. Sen. Lamar Alexander holds a commanding lead in the upcoming Republican primary election. According to the poll, Alexander, R-Tenn., has 64 percent favorability ratings among likely Republican primary voters, compared to 20 percent for his opponent Joe Carr. Even more ominous for Carr, 55 percent of those likely voters said they’d never heard of him.

Carr lost by only 9 points in the August 2014 Republican primary for the U.S. Senate in Tennessee. The May 2014 Vanderbilt Poll missed the mark by a whopping 35 points.

Carr vividly recalls just how inaccurate that poll was when he analyzes the credibility of the May 2017 Vanderbilt Poll.

“This is the Vanderbilt Poll co-directed by John Geer who famously predicted in the same worthless poll that Lamar Alexander would win his 2014 GOP Primary Race by 44 points, that Common Core was very popular and that illegal immigrants should be welcomed into our community’s,” Carr tells The Star.

“I don’t know anybody who takes Professor Geer and the Center for the Study of Democratic Institutions seriously. His record speaks for itself,” Carr adds.

The Vanderbilt Poll was also significantly off in its October 2016 poll that showed Donald Trump had only an 11 percent margin over Hillary Clinton in the presidential election held one month later:

Trump has the support of 44 percent of Tennessee voters and is especially popular with angry voters who want change. Clinton has the support of 33 percent of registered voters and does very well among those voters who are more optimistic about the future of the country. These results come from a poll of 1,000 registered voters in Tennessee.

On election day, Trump soundly defeated Clinton in Tennessee by a 26 percent margin, 61 percent to 35 percent, double digits beyond the Vanderbilt Poll’s stated 3.7 percent margin of error.

The Vanderbilt Poll described the methodology in its May 2017 poll released this week as follows:

The Tennessee Poll Spring 2017 obtained telephone interviews with a representative sample of 1,004 registered voters in Tennessee. Telephone interviews were conducted by landline (811) and cell phone (193). The survey was conducted by Princeton Survey Research Associates International (PSRAI). Interviews were done in English by Princeton Data Source (PDS) and Survey Sampling International (SSI) under the direction of PDS from May 4 to 15, 2017. Statistical results are weighted to correct known demographic discrepancies. The margin of sampling error for the complete set of weighted data is ± 3.3 percentage points.

 

Here are the top seven reasons this methodology is flawed:

1. They polled over an unusually long period of time (12 days).

“Rule #1 in polling: Any poll that takes over three days to complete, significantly increases the likelihood the poll is either inaccurate or useless. This is because polls are designed to take snap shots of public opinion during very specific time periods,” a Capitol Hill insider with more than thirty years experience managing political campaigns at the state and national level tells The Star.

News moves very fast these days, and someone’s opinion taken on Day 1 of a poll on a particular issue could be quite different than that same person’s opinion on the same issue taken on Day 12 of the poll because of events that take place between Day 1 and Day 12.

2. They interviewed “Registered voters” – not actual voters.

“That means at least one-third of the respondents will have no impact on the election, because they will not show up to vote. Their opinions are therefore irrelevant and useless in deciding who actually gets elected,” the Capitol Hill insider tells The Star.

“Based on the electoral history of all Tennessee elections over the last 40 to 50 years, it is rare that actual voter turnout exceeds 65 percent of registered voters,” the veteran political operative adds.

The actual turnout of registered voters in the four most recent contested primaries, as reported by the Tennessee Secretary of State is much less than 65 percent:

Primary                   Turnout

August 2016              14.05 %
August 2014             28.08 %
August 2012              18.63 %
August 2008             14.92 %

3. There appears to be an over-sampling of both the Nashville area and of Democrats.

A much more accurate sample would be based on actual population by county – not a region.

 

According to the methodology, the counties were divided into 4 regions, with percentages of respondents from each region divided as follows:

Region             % of Respondents

East                          37.1 %
Nashville area        22.9 %
Central                     20.8 %
Memphis/West      19.1 %

It doesn’t appear that they took any account of different regional weighting for GOP primary versus Democratic Party primary.

Nashville/Central account for 43.7 percent of the poll. Not surprisingly, Nashville/Central based candidates with the Nashville area media market reaching most of that area have higher name recognition than they might show in the other two sections of the state.

“The poll also includes 32 percent Republicans and 26 percent Democrats, making Tennessee–a state with a Republican super majority in both houses of the Tennessee General Assembly, a Republican Governor, two Republican Senators, and seven out of nine Republican members of Congress–according to the Vanderbilt Poll, a Republican +6 state,” the Capitol Hill insider tells The Star.

“The last time Tennessee was only a +6 Republican state was during Lamar Alexander’s first term in the 1970s,” the insider says.

4. The questions were inherently biased.

Example: Asking whether in-state tuition prices should be made available for “children of undocumented parents” vs. the more accurate “in-state tuition for illegal aliens” guarantees a higher level of support for the in-state tuition advocates. If the poll also accurately noted that providing in-state tuition for illegal aliens meant taxpayers would effectively be subsidizing the shortfall between in-state and out-of-state tuition, the level of support for in-state tuition for illegals would drop significantly.

Here is the actual tuition for children of undocumented question used in the May 2017 Vanderbilt Poll:

Now thinking about children of undocumented immigrants who are brought to this country when they are young… If these children attend Tennessee public schools, graduate from a Tennessee high school and are accepted at one of Tennessee’s public colleges and universities, do you think they should be eligible for the in-state tuition rate, or shouldn’t they?

Just as important as the bias involved in the way the questions were asked is the failure to include a question on the single issue that dominated the 2017 session of the Tennessee General Assembly and is likely to play a key role in the 2018 general election for both Governor and members of the Tennessee General Assembly: the 6 cents per gallon gas tax and 10 cents per gallon diesel tax championed by Gov. Haslam and passed by the Tennessee General Assembly.

One obvious reason for this bias is clear: There are no actual conservatives on the poll’s advisory board. Indeed, the members of the advisory board all have either a current or historical interest in advancing big government liberal causes and/or candidates.

According to the Vanderbilt Poll’s website, “Current VU Poll Board members are:”

• Samar Ali, attorney at Bone McAllester Norton PLLC and co-founder of the Lodestone Advisory Group
• Charles W. Bone, attorney and chairman of Bone McAllester Norton PLLC
• José González, instructor of entrepreneurship and management, Belmont University and co-founder of Conexion Americas
• Tom Ingram, political strategist with The Ingram Group
• Roy M. Neel, senior adviser to former Vice President Al Gore
• Bill Phillips, government relations consultant and former Nashville deputy mayor
• Bill Purcell, former Nashville mayor and partner at Farmer Purcell White & Lassiter, PLLC
• Lisa Quigley, Chief of Staff for Rep. Jim Cooper
• Anne Russell, special counsel at Adams and Reese LLP
• Chip Saltsman, former chief of staff for Rep. Chuck Fleischmann and political strategist
• Jamie Woodson, President and CEO of the State Collaborative on Reforming Education (SCORE)

Tom Ingram is a well known political guru and advisor to Senator Lamar Alexander and other “establishment” Republicans who are big government liberals.

Bill Phillips was the stage manager for the Republican National Convention 30 years ago, served as Deputy Mayor for Democrat Bill Purcell, former Nashville Mayor who also serves on the Advisory Board and joined a lobbying firm owned by a left wing Democrat.

“Chip Saltsman is an ‘establishment’  Republican known primarily for his political opportunism. No one in Tennessee politics describes Saltsman as a conservative, least of all his current client, Gubernatorial candidate Randy Boyd, who calls himself a ‘moderate,’ but whose policies are those of a left wing Democrat,” the Capitol Hill insider says.

5. There was no breakout of primary voters.

This is especially important in a state where every single statewide elected official of the last 25 years, save one (Bredesen), was elected by the person who won the GOP primary.

6. About halfway through the poll they changed one of the questions to include Mark Green among potential Governor’s candidates.

Thus only about 846 rather than the total sample of 1004 registered voters were used for that question. Did they reweighs the geographic, minority, age, etc. weighting for that question? And it is extremely poor polling process to change questions in the midst of a poll, on of the reason you don’t poll for more than 3-4 days.

7. In polling name recognition of potential Gubernatorial candidates the poll question identified the names as being potential candidates.

Thus, respondents are more likely to claim to have heard of the “important” people running for office so as not to appear ignorant and uninformed.

The same thing happens when you ask people if they regularly vote. The number of “regular” voters always far exceeds the actual percentage of voters. One way to avoid the higher false positives of name recognition is to ask a control question, i.e. include a name that doesn’t actually tie to a real or prospective candidate. That gives you a baseline for numbers that are based on trying to make the respondent appear more informed than they actually are.