Dana Loesch at BigJournalism has
pointed out one problem with the study: that it was in part funded by George Soros. Soros is known for funding all sorts of projects and companies, including Media Matters, that further his own personal agenda and beliefs. Loesch discovered that the UMD study was run with funding from the Tides Foundation, which counts Soros as a major funder. Blogger
Warner Todd Houston found that the study was also funded by "such
far left-wing organizations as the Ben and Jerry's Foundation, the Ploughshares Fund, [and] the United States Institute of Peace."
Of course, merely having a left-wing billionaire involved in the funding of a study does not automatically make it flawed. What makes a study flawed is bad methodology. And this study has plenty of that.
Twelve of the study's 43 questions asked were not released to the public in the study's formal
Questionnaire with Findings and Methodology. Instead, each question was marked to be "released separately." It is unclear where or when those questions will be released. The missing questions make up over a quarter of the entire study.
Of the questions that were made publicly available, however, there are obvious flaws. The study partially addresses these flaws, but they are still problematic if we are to take the study seriously.
As an example, take the following questions:
Q15. How do you think economists viewed the TARP bank bailout[?]
Q21. What is your impression of how ECONOMISTS viewed the idea of stimulating the economy in this way?
Here, the study is asking for perceptions about economists generally. Yet, the study does not count economists generally: the study counts the official findings of government agencies as though they represent all economists. Examine the following from the study's overview (emphasis added):
Voters' misinformation included beliefs at odds with the conclusions of government agencies, generally regarded as non-partisan, consisting of professional economists and scientists.
- Though the Congressional Budget Office (CBO) concluded that the stimulus legislation has saved or created 2.0-5.2 million jobs, only 8% of voters thought most economists who had studied it concluded that the stimulus legislation had created or saved several million jobs. Most (68%) believed that economists estimate that it only created or saved a few jobs and 20% even believed that it resulted in job losses.
- Though the CBO concluded that the health reform law would reduce the budget deficit, 53% of voters thought most economists have concluded that health reform will increase the deficit.
- Though the Department of Commerce says that the US economy began to recover from recession in the third quarter of 2009 and has continued to grow since then, only 44% of voters thought the economy is starting to recover, while 55% thought the economy is still getting worse.
In other words, voters were considered "misinformed" if they held "beliefs at odds with the conclusions of government agencies." Just because the people behind this study may believe government agencies to be immune to partisan influence, doesn't mean they are. And even if they are, it is misleading to ask about economists generally, but then only count economists who are employees of government agencies.
Similarly, the study asked questions about scientists' views on climate change. Question 34 asked for opinions on whether "MOST SCIENTISTS" believe a) that climate change is occurring, b) that climate change is not occurring, or c) that views are evenly divided. Yet, the only scientists the study's authors counted were those with the National Academy of Sciences:
The National Academy of Sciences has concluded unambiguously that climate change is occurring. However, a substantial 45% of voters thought that most scientists think climate change is not occurring (12%) or scientists are evenly divided (33%). Fifty-four percent recognized that most scientists think that climate change is occurring.
It is misleading to ask those polled about what "most scientists" think about any issue, without clarifying that "most scientists" only refers to the scientists employed by the National Academy of Sciences.
As mentioned above, the UMD study does partially address this issue. The
official study publication includes a section entitled "A Note on the Question of 'What is True,'" which includes the following (emphasis added):
A study of misinformation raises the somewhat delicate question of what is true. When dealing with topics that have been highly politicized, it is common to default to the position that all perceptions are relative and treatment of any position as more or less true is itself inherently political. We believe that such a position is at odds with what is necessary for well-functioning democracy. It is indeed very important for a healthy democratic process to be open to a wide range of positions. At the same time, it is essential that there be means and institutions for achieving consensus about key factors that ultimately affect public policy decisions.
On a regular basis government economists come to conclusions about the state of the economy. Such conclusions influence key decisions in the private sphere, as well as government decisions. Such government economists should be, and generally are, open to input from experts outside of government in the course of coming to conclusions.
In the course of this study,
to identify "misinformation" among voters, we used as reference points the conclusions of key government agencies that are run by professional experts and have a strong reputation for being immune to partisan influences. These include the Congressional Budget Office, the Department of Commerce, and the National Academy of Sciences. We also noted efforts to survey elite opinion, such as the regular survey of economists conducted by the Wall Street Journal; however, we only used this as supporting evidence for what constitutes expert opinion.
Again: while the study asked about "economists" and "scientists" generally, the authors "used as reference points the conclusions of key government agencies." Any other surveys of "elite opinion" were used as mere supporting evidence.
Unless "the conclusions of key government agencies" suddenly are always in keeping with the conclusions of economists and scientists as a whole, this study is flawed. People should use caution before taking its conclusions too seriously.
