Most people are familiar with the “non-debate” debate between the overwhelming majority of scientists who conclude that fossil fuels are contributing to disruptions to the climate system and the tiny but vocal few who deny it. Now, another “non-debate” debate is making its way into the news. This time, though, the “debate” is about climate communications—or how environmental advocates can best “sell” climate change policies to the public.

On one side of this debate is a large, interdisciplinary group of social scientists who have consistently found that Americans’ support for a more robust policy response to climate change varies significantly depending on how the issue is framed. On the other is a pair of Swiss researchers who conducted a single, flawed study and came to the conclusion that frames don’t affect policy support for climate change. Frames, they claim, don’t matter, and advocates, they argue, should not expend precious resources reframing climate change.

The media ran with the story, publishing pieces about the study in outlets like the Washington Post and Vox. No surprise there; reporters, we know well, love conflict. But the media’s coverage of this lone, outlying study has the same damaging effect that coverage of climate-change deniers has in the larger conversation about the environment: It creates an impression of legitimacy where none is warranted.

I’m here to set the record straight. As a social scientist, I’m all for a healthy exchange of ideas, and I fully support the researchers’ goals to explore how different messages affect Americans’ support for climate-related policies, like fuel-efficiency standards, and their willingness to take civic actions, like signing petitions, to protect our climate.

At FrameWorks, my colleagues and I investigate these kinds of questions in our research on how to frame a wide array of social issues, including climate and ocean change. We too want to know whether advocates should emphasize the risk of environmental catastrophes that could be caused by a changing climate or if it is more effective to base the appeal on the economic, social, or health benefits of climate policies. We also agree that we must find answers to these questions if we are to have any hope of building the political will to take the often hard-to-swallow solutions to climate change.  

FrameWorks takes a straightforward approach to answering these questions: We randomly assign study participants to read different messages and then ask them about their policy preferences and attitudes afterward. This allows us to determine which messages move people and by how much.  Sounds simple, but the devil is in the details. For such experiments to yield meaningful evidence, researchers must ensure that the only differences in the messages are the gist of the argument, or the frame. If there are differences in the messages’ length, sentence structure, style, or tone, it is impossible to isolate which variables cause changes in participants’ responses. And if the messages don’t accurately reflect the arguments that advocates are actually making, then we can’t make practical recommendations based on the findings.

We found these sorts of methodological problems in the Swiss study, conducted by Thomas Bernauer and Liam McGrath. For the study (conducted in two separate parts), Bernauer and McGrath divided participants into random groups and asked each to consider a framed message about climate policy. The baseline group was exposed to a message about the environmental risks of not taking action, and other groups heard alternative frames about the economic benefits of climate policy (technological innovation and a more prosperous economy); the social benefits (stronger communities); and the health benefits (increased physical activity and reduced pollution). They found that alternative frames did not generate more support for climate policies than the risk frame.

Close inspection, however, reveals that the alternative frames differed substantially in tone and style from the baseline frame. See if you can guess the first thing we noticed when we looked at the following messages used in the experiment, keeping in mind that for the experimental design to be sound, the messages should be as similar as possible. This is something like the exercise of spotting minor differences in different photographs of the same scene.

Frame #1: Risk

It can be confusing with all the science that comes out about global warming. Many people say that reducing carbon dioxide emissions, which are regarded as the main cause of global warming, will stop the planet heating up, and most scientists seem to think that. I think that because there is general consensus among scientists we have to accept that global warming is real. I’ve been thinking about the impact of taking action on global warming, and I think the effects on the environment will be obvious—it follows from the science. We’d stop sea levels rising, reduce the chance of extreme weather like droughts and floods, and save plant and animal species. We would be less affected by food and water shortages, heat waves, flooding, and health issues that come with high temperatures. Reducing carbon emissions would certainly reduce these risks from global warming, and that would be a good pay-off.

Frame #2: Economic Co-Benefits

It can be confusing with all the science that comes out about climate change. But it got me thinking about what the world would be like if everyone pulled together to reduce carbon dioxide emissions, which are regarded as the main cause of global warming. It might stop global warming—who knows? But I think one thing it would do for sure is affect our economy. People focus on the environmental effects of action on global warming, but that’s not how I see it. I think about how taking action would impact the economy, and on balance I think it would be good. Taking action to reduce carbon dioxide emissions would lead to new scientific breakthroughs and new industries, leading to new jobs and more sustainable economic development. Who knows what it would do against global warming, but I think it’d make for a more prosperous and successful economy in the long run, and that would be a good pay-off.

Frame #3: Health Co-Benefits

It can be confusing with all the science that comes out about global warming. But it got me thinking about what the world would be like if everyone pulled together to reduce carbon dioxide emissions, which are regarded as the main cause of global warming. It might stop global warming—who knows? But I think one thing it would do for sure is make us healthier. People focus on the environmental effects of action on global warming, but that’s not how I see it. I think about how taking action would improve our health, and on balance I think it would be good. Taking action to reduce carbon dioxide emissions would help us redesign our cities and towns to make it easier and safer to travel by foot, bicycle and public transportation. This will reduce the number of cars on the road and will reduce traffic injuries and fatalities. It will help people become more physically active, lose weight, strengthen their bones, and possibly even to remain mentally sharp as they age. And using cleaner forms of energy—such as solar and wind power—will reduce air and water pollution, thereby preventing many forms of illness. Who knows what it would do against global warming, but I think it’d make for a healthier society in the long run, and that would be a good pay-off.

Did you pick up on the different treatments? The economic and health benefits frames include the phrase “Who knows?”, but the risk frame does not. The benefits frames also ask the question, “Who knows what it would do against global warming?”, but the risk frame does not.  In terms of study design, this lack of “parallel treatments” is a fatal flaw. It means that the benefits messages have a sense of uncertainty baked into them, but the risk frame doesn’t.

Moreover, the benefits frames do not  reflect the messages currently in use by scientists, economists, and public health officials, so the results are of little practical use. Environmental and health advocates, for example, do not ask questions like “Who knows?” in their messaging, so we can’t tell how these frames would fare in real-world settings.

These problems call into question the credibility of the study and cast doubt on its conclusions. They’re also dangerous; Bernauer and McGrath advise advocates to stick to the risk frame when making their case to the public. But we know from our research—and from dozens of other studies on the subject—that benefits frames do build support for climate policy—and to a significant degree. On this, one of the most important frame contests of our time, there simply is no debate.

Julie Sweetland is vice president for strategy and innovation at the FrameWorks Institute, a nonprofit communications think tank in Washington, D.C.