Spreadsheets and Global Mayhem

NY Times
March 22, 2014


IN this age of fine-grained prediction, a variety of algorithms hover around us all the time to divine what we might buy, whom we might mate with, and whom we are likely to vote for at election time. Now social scientists are using some of these same tools to predict when we are likely to do horrible things to one another.

Australian researchers say they have developed a mathematical model to predict genocide. A Swiss sociologist has sifted through a century of news articles to predict when war will break out — both between and within countries. A Duke University lab builds software that it says can be used to forecast insurgencies. A team assembled by the Holocaust Museum is mining hate speech on Twitter as a way to anticipate outbreaks of political violence: It will be rolled out next year for the elections in Nigeria, which have frequently been marred by violence.

What makes these efforts so striking is that they rely on computing techniques — and sometimes huge amounts of computing power — to mash up all kinds of data, ranging from a country’s defense budget and infant mortality rate to the kinds of words used in news articles and Twitter posts.

None of this has yet produced a perfect crystal ball to foretell mass violence — and for good reason. “Events are rare, data we have is really noisy,” said Jay Ulfelder, a political scientist who is developing a web-based early warning system to forecast mass atrocities. “That makes it a particularly hard forecasting task.”

But social scientists are getting better at anticipating where trouble might start — or as Mr. Ulfelder put it, “assessing risks.” That explains why the United States intelligence community has been exploring the field for years. The government’s Political Instability Task Force, which Mr. Ulfelder helped to run for over a decade, tries to predict which countries are likely to witness civil unrest in the near term. Its data is not public, nor is information on how the government uses its predictions.

By now, of course, data tracking is pretty much embedded in our daily lives. Amazon tries to anticipate what we can be tempted to buy based on what we’ve bought — or even considered buying — in the past. Google tries to predict what we’re searching for. Political parties in the United States and abroad are devising new tools to predict who will vote for whom. And policeagencies worldwide are increasingly turning to analytics tools to forecast when and where crime is likely to occur.

Predicting mass violence is yet another frontier. Among these efforts is a2012 project funded partly by the Australian government in which a team from the University of Sydney looked at more than a dozen variables that could point to the likelihood of mass atrocities: Had there been political assassinations or coups; were there conflicts in neighboring states; is there a high rate of infant mortality? (Infant mortality turns out to be a powerful predictor of unrest, a signal that state institutions aren’t working.)

Read the Full Article