It is a constant criticism of politicians; they poll test or focus group everything before making a decision. If you are one to follow politics, you’ll know that new polls on everything from election horse-races to how the public feels about minute details of policy ideas come out daily. But policy-makers should be wary of using public polling to determine policy directions. The recently released PPIC Californians & the Environment Statewide Survey illustrates why.

The Public Isn’t Paying Close Attention: For the most part, the public doesn’t pay attention to the intricacies of most policy issues. This isn’t a slight on voters; they have their own lives, jobs, and interests, all of which rarely overlap overtly with public policy. Because they aren’t paying close attention, they probably don’t have much information on the issue before they are asked to state their opinion on it. For instance, 59% of likely voters say they favor the state law to reduce greenhouse gases (named AB 32). AB 32 set up a cap-and-trade scheme to reduce greenhouse gas levels to 1990 levels by 2020, but only 24% of likely voters have heard a lot about the cap-and-trade policy and 50% of likely voters oppose the cap-and-trade program. In other words, six-out-of-ten likely voters favor AB 32, but less than one-quarter actually know a lot about its central policy construct and a majority oppose that very policy. With three-quarters of likely voters (and 87% of all Californian adults) not really knowing a lot about the policy, their views on it aren’t very informative.

Polling Requires Simplicity: While focus grouping and deliberative polling can enable respondents to dive into the issues, those types of surveys are expensive and time consuming. Thus, the most available public polling is the less expensive phone or online type, which requires question simplicity. With limited space, word choices are very important and can lead respondents in a particular direction. While not in this survey, PPIC often uses the phrase, “I’d rather pay higher taxes and have a state government that provides more services, or I’d rather pay lower taxes and have a state government that provides fewer services.” This, of course, is a bit of a straw-man argument since it is possible with an efficiently run state government to have both low taxes and more services.

Also, it is difficult to get the whole picture in just a few sentences. For instance, only 38% of likely voters support building more nuclear power plants, while 71% of likely voters said global warming is a serious or somewhat serious threat to California’s economy and quality of life. Left unsaid is that with the decommissioning of the 2,200 megawatt San Onofre nuclear power plant, California is going to have to plug the electricity gap with a mix of inefficient and expensive renewables and gas-powered plants, which, on the whole, will make electricity more costly and less environmentally-friendly.

And when more detail is provided, the results can widely vary. Likely voters go from strongly supporting low emission fuels (70%) and requiring renewables to supply 1/3rd of California’s electricity (69%) to opposing it if it means additional consumer costs (57% and 51%, respectively).

Issue Saliency Is Often Overlooked: It is a common phenomenon in politics. An issue polls well with voters, but then the candidate who opposes it wins. It all comes down to issue saliency. Voters prioritize issues; so even though they may support one stance, if a candidate supports another higher prioritized position, voters will likely vote against something they like. Just because 71% of likely voters think global warming is either a somewhat serious or serious threat to California’s economy and quality of life doesn’t mean Californian voters are going to vote with only a candidate’s position on global warming in mind. Global warming may not be the most salient of issues for voters and indeed, the poll suggest it isn’t. When asked what the “most important environmental issue facing California today” is, Californians say “water supply, drought, reservoirs.” And even this doesn’t mean voters are only going to focus on the drought on Election Day. This is just the most important environmental issue.

Taking polls purely at face-value can lead you astray, especially if you are trying to formulate public policy around what is popular. Polling is often contradictory and usually has a little bit of useful evidence for all points of view. However, if you approach polling warily and know where the tripping points might lie, voter surveys can be both instructive and informative.

Originally posted on the Hoover Institution’sEureka

Follow Carson on Twitter: @CarsonJFBruno

Carson Bruno, a research fellow at the Hoover Institution, primarily analyzes California’s economic, electoral, and public policy landscapes.