Experimental research has expanded markedly in political science over the past 30 years.
It's a trend NYU Abu Dhabi Visiting Global Distinguished Professor of Political Science Ron Rogowski couldn’t miss. As editor of the American Political Science Review from 2007-2012, he was receiving more and more research papers based on new types of experimentation. And now Rogowski has written his own paper about this modern day research phenomenon.
“We were getting a lot of this stuff,” at the Review “and there hadn’t been much at all, in any journal, a decade before” he said.
In his recent article “The Rise of Experimentation in Political Science”, Rogowski explains the rapid growth of important new research methods, outlining the advantages and drawbacks of each one.
It’s a surprising subject for someone whose impressive career path includes no such work of his own. Rogowski’s paper was commissioned by the Center for Advanced Study in the Behavioral Sciences, based at California’s Stanford University. It will be part of a forthcoming book that’s aimed at a wider audience than most academic journals; this book, one of two, reports on recent developments in the social sciences.
And there’s plenty to report about experimental political science, a field that now has its own journal. Rogowski’s article notes that the well-established techniques of laboratory experiments (“conducted chiefly among undergraduate subjects at major universities”) now “seem to be fading from the mainstream journals.”
Instead, three other types of experiments are gaining ground, he writes: survey, field, and natural experiments. As always, he noted, the ideal is “fully randomized ‘treatment’ of some presumably representative set of subjects” but this can present some difficulties, especially in historical research.
Tricky Science
Survey projects, he noted “best exemplify the effects of newer technology” and it’s easy to get a random sample. “Technology has given us this wonderful cheap gift. On the national sample of YouGov” – an international market-research firm – “you can add in a question of your own for $500 US” he said. “And you can do that in many countries.”
Field experiments, often conducted in less developed countries but sometimes involving participation in first world political campaigns, can generate ethical concerns. In one Texas study, a candidate for statewide office agreed to use different TV advertisements in different cities, while daily polling measured the impact.
But ethical concerns, about deception for example, can arise is such studies, Rogowski noted, and “I’m not sure they all can be overcome … Sometimes, to make things work well you’re highly tempted to do something deceptive.”
Some scholars have been ingenious, he added, in designing experiments to avoid deception. But the norm, he said, is that a university’s ethics watchdog, known as its institutional review board, must “vet the study at every point to make sure there is no deception.” Some universities are more demanding than others in this, he said.
If political science remains a largely non-experimental science, it will continue to be treated with skepticism and stinginess.
Natural experiments, where something outside the researcher’s control generates data, can entail problems of “external validity” if more than one factor is found to be exerting influence. A famous natural-experiment study on land tenure in colonial India, he noted, reached conclusions connecting outcomes to certain 19th-century British political developments. But these were later challenged when other scholars identified a different factor that also appear to correlate.
In this field the gold standard is the 1969 United States conscription lottery to determine which young-adult male Americans would be drafted during hostilities in Vietnam. In that case, Rogowski noted, “history luckily just gave you what clearly was a random draw.” This made possible surveys, right after the lottery and again in the 1990s, showing that those with unfavourable draft numbers tended to become – and remain – more antiwar, and more Democratic in their voting habits.
Summarizing all this, Rogowski said, has led him to begin considering doing some experimental work in his own field, the politics of world trade. At NYUAD, where Rogowski expects to be teaching for a fifth year next autumn, he’s a colleague of Rebecca Morton, professor of politics and director of the Social Science Experimental Laboratory, and founding co-editor of the Journal of Experimental Political Science, launched in 2014. (Her co-editor was Joshua Tucker, professor of politics at NYU in New York; the current editor, Eric Dickson, is an associate professor of politics and psychology at NYU.)