February 23, 2013

Did Question Bias Skew Results from James Madison Institute's Poll Showing 63% of Floridian's Oppose Medicaid Expansion?

Tallahassee, Florida-based James Madison Institute released a poll and press release showing that 63% of Floridian's oppose Medicare expansion, which Governor Rick Scott recently supported.

While the merits of the expansion can be debated, I was thrown for a loop by the 63% margin. The results flew in the face of another poll by the Florida Hospital Association (certainly with its own agenda) that showed the exact opposition, specifically that 62% favored expansion., as well as my own gut feeling.

I wondered if question bias - essentially a leading or loaded question - had anything to do with the results. Apparently, others did as well. Bloomberg’s Josh Barro questioned if it was a push-poll and Michael Cannon of CATO, who helped draft the questions, rushed to defend.

Let's look at the questions and responses from James Madison Institute:

Totals: 600 100.00%
QUESTION 5:
Medicaid is a program for low income people and families, operated by the State and
paid for, in part, with Federal matching funds. Medicaid coverage is now generally
available to those with incomes up to 100 percent of the poverty line. Under the Federal
Health Care Law, Florida and other states have the option to increase those covered up to 138 percent of the poverty line.

At 21 billion dollars, spending on Medicaid currently represents about 30 percent of
Florida's budget. If Florida should implement the Medicaid expansion, Medicaid would
become an even higher percentage of overall state spending. Does this fact make you
more likely or less likely to support expanding the Medicaid eligibility requirements in
Florida?

1=More Likely 180 30.00%
2=Less Likely 337 56.17%
3=Unsure/No Difference 79 13.17%
4=Refused 4 0.67%
Totals: 600 100.00%

QUESTION 6:
Experts disagree on the exact cost, but predict that if Florida expands Medicaid coverage
to those up to 138 percent of the poverty line, it will cost the State of Florida an  additional $3 billion to $20 billion dollars over the next 10 years. Knowing this, would
you be more likely or less likely to support the Governor and Legislature expanding
Medicaid coverage?

1=More Likely 166 27.67%
2=Less Likely 359 59.83%
3=Unsure/No Difference 69 11.50%
4=Refused 6 1.00%
Totals: 600 100.00%

QUESTION 7:
To expand Medicaid coverage in Florida would require either additional taxes or less
State spending on things like education, roads and law enforcement. Would you be more
likely or less likely to support the Governor and Legislature expanding Medicaid
coverage, if it meant higher taxes and less spending on other priorities?

1=More Likely 162 27.00%
2=Less Likely 363 60.50%
3=Unsure/No Difference 66 11.00%
4=Refused 9 1.50%
Totals: 600 100.00%

QUESTION 8:
Under the Federal Health Care Law States can choose to expand Medicaid to cover
everyone below 138 percent of the Federal Poverty line, up from the current 100 percent.
To help pay for the increased cost the Federal Government will cover almost 100 percent
of the newly eligible for the first three years. After the three years is up, Florida
Taxpayers will have to cover more than ten percent of the cost for the newly eligible.
Knowing this, which view do you agree with more?

1=The Governor and State Legislature should expand
Medicaid even if it results in tax hikes and spending cuts. 184 30.67%
2=The Governor and State Legislature should opt not to expand
Medicaid keeping taxes and spending on current trajectories. 356 59.33%
3=Unsure 54 9.00%
4=Refused 6 1.00%
Totals: 600 100.00%

QUESTION 9:
Would you support expanding Medicaid coverage if the effect would be to pull many
adults out of private coverage and into the Medicaid Program, where taxpayers would
pay for their coverage and they might have decreased access to doctors?
1=Yes - Support 110 18.33%
2=No - Oppose 391 65.17%
3=Unsure 89 14.83%
4=Refused 10 1.67%
Totals: 600 100.00%

QUESTION 10:
Should Florida expand its Medicaid Program or should it improve the private insurance
market to increase coverage to the uninsured?
1=Improve Private Market to Cover More Uninsured 334 55.67%
2=Expand Medicaid to Cover More Uninsured 159 26.50%
3=Unsure 97 16.17%
4=Refused 10 1.67%
Totals: 600 100.00%

QUESTION 11:
Even without expanding the poverty line for Medicaid coverage, many currently
Medicaid eligible individuals will be forced to enroll in Florida's Medicaid Program
under the Federal Healthcare Law's individual mandate. This will require Florida to
spend more on Medicaid even without poverty line expansion. Does knowing this make
you more likely or less likely to support the Governor and Legislature expanding
Medicaid coverage?

1=More Likely 177 29.50%
2=Less Likely 323 53.83%
3=Unsure/No Difference 91 15.17%
4=Refused 9 1.50%
Totals: 600 100.00%

QUESTION 12:
Proponents of Medicaid expansion point out the Federal Government will reimburse
Florida for 100 percent of the cost initially and then 90 percent of the cost. Which of
these statements best describes your viewpoint?

- I am confident the Federal Government will not change the program and has the
resources to always reimburse the State of Florida for 90 percent of its expanded
Medicaid costs.

- I am very concerned the Federal Government will eventually change the program,
reduce the amount it reimburses to Florida, and add additional expenses to Florida
taxpayers.

1=I am Confident 170 28.33%
2=I am Concerned 380 63.33%
3=Unsure 42 7.00%
4=Refused 8 1.33%
Totals: 600 100.00%

On the wording of the poll, Barro writes:

instead of asking for a straight yes-or-no answer, the pollster asked if respondents favored Medicaid expansion “even if it results in tax hikes and spending cuts.” This isn’t a poll designed to figure out how Floridians feel about the Medicaid expansion; it’s one designed to get them to say they oppose it, so the organization commissioning the poll can say it’s unpopular.

Cannon responded, after conceding there was an inaccuracy in the description of the terms of expansion, writes:

Actually, the poll ties the Medicaid expansion’s benefits to its costs, which include (but are not limited to) higher taxes and/or spending cuts. Medicaid expansion is not a benefits-only proposition. When a poll only asks voters about benefits, the results are meaningless. Yet to my knowledge, JMI’s poll is so far the only poll that has asked voters about both costs and benefits. All other polls—for example, the hospital-industry poll Barro cites—ask only about benefits, as if the costs don’t exist or shouldn’t influence voters’ evaluation of the expansion. Those polls are “push” polls, while JMI’s poll is the only honest poll in the field. Barro doesn’t complain about JMI’s representation of the costs. He’s just an economics blogger who doesn’t think costs should be part of the question at all.

Let's also compare two similar but differently worded questions and compare the two vastly different responses.

James Madison asks:

Proponents of Medicaid expansion point out the Federal Government will reimburse Florida for 100 percent of the cost initially and then 90 percent of the cost. Which of these statements best describes your viewpoint?

- I am confident the Federal Government will not change the program and has the resources to always reimburse the State of Florida for 90 percent of its expanded Medicaid costs.

- I am very concerned the Federal Government will eventually change the program, reduce the amount it reimburses to Florida, and add additional expenses to Florida taxpayers.

1=I am Confident 170 28.33%
2=I am Concerned 380 63.33%
3=Unsure 42 7.00%
4=Refused 8 1.33%
Totals: 600 100.00%

Florida Hospital Associations asks:

In order to cover more uninsured adults, the federal government would cover all of the increased costs to expand health care coverage through Medicaid for the first three years. The federal government would then cover 90 percent of the increased costs permanently with the state of Florida paying 10 percent. Knowing this, would you say the state should or should not accept the federal money to expand health care coverage through the Medicaid program to cover more uninsured adults?

The view that the state should accept federal funding and extend coverage is significant with virtually all “swing” voters sub-groups in the state: younger voters (76 percent), self-identified moderates (72 percent , women (69 percent), seniors (63 percent), political independents (61 percent), white women (62 percent), Orlando voters (67 percent) and Tampa voters (59 percent).

Even a majority self-identified “Moderate/Liberal Republicans” (52 percent) support accepting federal funds for coverage.

 

JMI points out that the federal government could indeed renege on its promise in the future. However, "concern" doesn't necessarily mean "oppose". FHA assumes the federal government will always pay 90%.

In looking at the questions from both camps, they do seem one sided, depending on the source. I would even say no to some of JMI's questions despite my overall feeling that expanding Medicaid is the correct thing to do for low income families and seniors who work and have worked their whole lives but lack access to healthcare or affordable healthcare.

That brings us to question bias. National Council on Public Polls states:

7. Can wording of questions bias poll results?

How questions in a poll are worded is as important as sampling procedure in obtaining valid results.

Most professional polling organizations and their media clients review the wording of questions as carefully as editors would examine a manuscript before publication. This process usually calls for a review of several drafts prior to fielding a poll. Questions are checked for balance: Are they worded in a neutral fashion, without taking sides on an issue? Does the question represent both sides of an issue fairly? Answer choices read to poll respondents must also be balanced; e.g., approve or disapprove, favor or oppose.

The order of questions must be logical. That is, general questions are asked before specific questions. For example, overall job approval of an incumbent must be asked before specific questions are asked that may remind respondents about the incumbent’s successes or failures. The same goes for questions asking respondents what side they take on an issue which may influence a later question about opinion of a candidate who takes the opposite side.

Questions are written using clear, unambiguous, concise language to insure that all respondents, regardless of educational level, understand them. And since most polls are conducted by phone, a writing style suitable for the ear is often adopted as opposed to a style more suitable for reading.

Therein lies the issue: the other side of the argument was nowhere to be seen in the poll: expanding coverage that benefits the working poor and the economic upside.

Mother Jones crystallizes the expansion of coverage to the working poor:

Today, a single parent with two children can't qualify for Medicaid in Florida if she makes more than $3,200 a year—one of the nation's lowest eligibility levels. Obamacare provides funding to raise that ceiling to $25,390 for a family of three. The federal government would pick up 100 percent of the cost of the expansion for the first three years, and 90 percent in later years—sending about $73 billion in new funding to the state in the next decade, with Florida's share of the bill totaling just $9 billion.

(View DCF's Medicaid income/asset eligibility chart here.)

And the Tampa Bay Times points out that Medicaid expansion could create 71,300 well-paying healthcare jobs and $8.9 billion in economic activity.

However, these economic aspects – both the upside up increased jobs and economic activity as well as easing healthcare expenses on the working poor - were left out of JMI’s questions, despite Cannon's own admonition that Barro “doesn’t complain about JMI’s representation of the costs. He’s just an economics blogger who doesn’t think costs should be part of the question at all.” Costs and benefits work both ways.

If economics is indeed an issue, why wasn’t both sides put forth. For example, why was not a question posed:

Medicaid is a program for low income people and families, operated by the State and paid for, in part, with Federal matching funds. Medicaid coverage is now generally available to those with incomes up to 100 percent of the poverty line. Under the Federal Health Care Law, Florida and other states have the option to increase those covered up to 138 percent of the poverty line.

At 21 billion dollars, spending on Medicaid currently represents about 30 percent of Florida's budget. If Florida should implement the Medicaid expansion, Medicaid would become an even higher percentage of overall state spending.

However, currently a single parent with two children can't quality for Medicaid if she makes more than $3,200 per year. If Medicaid is expanded that is raised to $25,390 for a family of three.

Considering these facts, does this make you more likely or less likely to support expanding the Medicaid eligibility requirements in Florida?

Or:

Experts disagree on the exact cost, but predict that if Florida expands Medicaid coverage to those up to 138 percent of the poverty line, it will cost the State of Florida an additional $3 billion to $20 billion dollars over the next 10 years.

However, expanding Medicaid could create over 71,000 jobs and $8.9 billion in economic activity over 10 years.

Knowing this, would you be more likely or less likely to support the Governor and Legislature expanding Medicaid coverage?

My point: you can have a poll show whatever you want and results should generally be seen through that lens, particularly when the polling is coming from groups pushing a certain viewpoint. To get a better understanding of a poll, look into the how the questions are phrased and explore the crosstabs. I'm guessing many decision-makers have felt the same, which is why JMI's poll landed with a giant thud and head-shaking and no one has rushed to quote it. 

Akins Creative

Partner with Akins Creative and let us help you build your brand and your business.
CONTACT US
envelope-ophone linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram