GDELT

Global Heat Map of Protests in 2013

My colleague Kalev Leetaru recently launched GDELT (Global Data on Events, Location and Tone), which includes over 250 million events ranging from riots and protests to diplomatic exchanges and peace appeals. The data is based on dozens of news sources such as AFP, AP, BBC, UPI, Washington Post, New York Times and all national & international news from Google News.

Mapped: Every Protest on the Planet Since 1979

Check out these incredible maps documenting every major protest from around the world since 1979.

The Global Database of Events, Language, and Tone (GDELT) tracks news reports and codes them for 58 fields, from where an incident took place to what sort of event it was (these maps look at protests, violence, and changes in military and police posture) to ethnic and religious affiliations, among other categories. The dataset has recorded nearly 250 million events since 1979, according to its website, and is updated daily.

John Beieler, a doctoral candidate at Penn State, has adapted these data into striking maps, like the one above of every protest recorded in GDELT – a breathtaking visual history lesson. Some events to watch for as you scroll through the timeline:

  • Strikes and protests in response to British Prime Minister Margaret Thatcher’s economic reforms.
  • Poland lighting up through the 1980s while Cold War-era Eastern Europe stays dark.
  • The escalation of apartheid protests in South Africa in the late 1980s.
  • The fall of the Berlin Wall and the rise of protests in Eastern Europe preceding the end of the Soviet Union.
  • Protests in Iraq coinciding with Operation Desert Storm in early 1991.
  • The explosion of protests in the United States since 2008 – think Occupy Wall Street and the Tea Party movements.
  • Iran’s Green Movement protests after the presidential election in 2009.
  • The Arab Spring, with protests stretching across North Africa and the Middle East starting in 2011.
  • The persistence of protests in perennial hot spots like Kashmir, Tibet, and Israel and the West Bank.

Data Mining Reveals How News Coverage Varies Around the World

Last year, the news media reported on 195,000 disasters around the world. The ones you heard about depend crucially on your location.

One interesting question about the nature of news is how well it reflects the pattern of real events around the world. It’s natural to assume that people living in a certain part of the world are more likely to read, see and hear about news from their own region. But what of the international news they get—how does that compare to the international news that people in other parts of the world receive?

Today, we get an answer to these questions thanks to the work Haewoon Kwak and Jisun An at the Qatar Computing Research Institute in Qatar. These guys have analyzed the news agendas in different parts of the world to see how the coverage reflects actual events in other parts of the world. And to visualize the different news agendas, they’ve created cartograms to reflect the coverage. These are maps in which the land area of a country is distorted by the amount of news coverage it receives in a given region (the image above shows how international news is viewed in North America).

Kwak and An begin with a database of 195,000 disasters that occurred between April 2013 and July 2014 and which were reported by more than 10,000 news outlets around the world. They noted the country in which each news outlet was based and then counted the published stories from other parts of the world. Finally, for various regions, they created a map of the world showing where the news was from.

The maps make for interesting viewing. They clearly show how the news agenda differs across the planet. Unsurprisingly, people in south Asia consume far more news about disasters in that region than people in North America, for example. And people in Latin America consume far more news from Argentina than Europe.

More interesting are the anomalies. For example, people everywhere consumed relatively large amounts of news from Egypt and Syria, mainly about the unrest in these countries and the accompanying humanitarian crises.

Kwak and An go on to investigate the factors that determine why people in one part of the world view disaster news from another. They found, for example, that population size is significant. People in all regions are more likely to see disaster news from other large countries, probably because there are more likely to be immigrants from those large countries who provide demand for that kind of coverage.

But by far the biggest factor that determines news coverage is whether an international news agency, such as Reuters, or Associated Press, covers the disaster. That’s unsurprising given that most news outlets have subscriptions to one or more agencies and are therefore able to use their stories easily. This is the primary mechanism behind the way news stories sometime snowball around the world.

Interesting work that reveals the way patterns of news coverage change around the globe.

Ref: http://arxiv.org/abs/1410.3710: Understanding News Geography and Major Determinants of Global News Coverage of Disasters

via http://www.technologyreview.com/view/532036/data-mining-reveals-how-news-coverage-varies-around-the-world/

Abstract: In this work, we reveal the structure of global news coverage of disasters and its determinants by using a large-scale news coverage dataset collected by the GDELT (Global Data on Events, Location, and Tone) project that monitors news media in over 100 languages from the whole world. Significant variables in our hierarchical (mixed-effect) regression model, such as the number of population, the political stability, the damage, and more, are well aligned with a series of previous research. Yet, strong regionalism we found in news geography highlights the necessity of the comprehensive dataset for the study of global news coverage.

pdf: http://arxiv.org/abs/1410.3710

The GDELT crew just announced the launch of their experimental Global Knowledge Graph (GKG), which “attempts to connect every person, organization, location, count, theme, news source, and event across the planet into a single massive network that captures what’s happening around the world, what its context is and who’s involved, and how the world is feeling about it, every single day.” Wow. It’s going to be fun to see how this gets used and evolves.

See here for Andrew Halterman’s R tools for GDELT and the GKG.

Are we getting better at looking forward, but compromising our ability to look backwards?

Last year the GDELT project published a paper on how crunching through the big data of history can help us spot patterns and work out where the world is heading next. The GDELT database monitors news (broadcast, print, web) from across the world in over 100 languages and uses complex computer algorithms to codify what’s happening throughout the globe. Certainly our ability to project the future will improve, as we apply more sophisticated ArtificialIntelligence to draw such predictions about the future

Earlier this week however, Vint Cerf, one of pioneers of Internet, in his recent conversation at the American Association of Advancement of Science expressed a different concern.  He said that the humanity could be headed towards a digital black hole as the digital objects are becoming unreadable as the technology evolves.  He said that we be putting misplaced confidence in the longevity of digital information.  The information GDELT and similar such efforts are trying to preserve is tip of the iceberg. The volume of data that is getting generated continues to grow exponentially, much of which (currently) of no obvious value. Much of this data is being stored with little thought on how it will be retrieved by historians of the future generations. We are at the risk of losing a lot of this information, compromising the ability of future historians to look backwards.

A few years back, I was chatting with a friend and client in the Aerospace industry responsible for archiving engineering data for the airplanes they designed. I was intrigued to learn that the company kept all their designs on printed paper, and not in digital format. My friend told me that that the ever-evolving digital world meant there was no safe way to store this information in digitized manner.

Till the time we are able to create what Dr Cerf calls a ”digital vellum”, it may be a smart idea for my friend to maintain the printed documents for their airplane design, and for you and I to print the digital photographs taken from our latest digital cameras

GDELT (Global Database of Events, Language and Tone) Project

CC Image by the GDELT Project

The GDELT (Global Database of Events, Language and Tone)project is a real time network diagram database that monitors broadcasts, printand web news on a global scale across 100 languages. It consolidates the information and identifies them according to people, organisations, themes, sources and event incidences. On the ethical front, GDELT has reflexively exploredits contributions to uncover human rights abuses as well as its own potential infringement on privacy.

Keep reading

Bordering Me (HackMIT 24 hour project)

  • Winner of the Google GDelt Data Challenge and awarded the Google Service Prize. The project will be posted on Google’s official blog in the coming weeks.
  • Winner of the nameCheap best domain name. 
  • Winner of Nod Ring

Summary:

Bordering.me is a 3D data visualization of worldwide marterial conflict and material cooperation between cities and how connected those relationships are to you. Material conflict includes things like the US refusing the continue to import natural gas from Canada through the Keystone XL pipeline while material cooperation might be apple opening up a new warehouse with Foxconn in China. The visualization is set up with a holographic projection while the user interacts with the globe using a gesture sensing ring. From the start the user inputs their location to show all the cities with which their area has had direct material cooperation with in the past month. With a swipe up they can then see cities with 1 degree of separation. With another swipe they see that after 2 degrees of separation, the material negotiations relevant to their location are connected to part of the entire planet. 

Story:

Last weekend I was flown out to Boston to participate in HackMIT. I didn’t have a team when I got there so on the morning of, I hunted down a couple of genius freshmen, Ian Macalinao and Akhilesh Yarabarla, saying they were looking into doing 3D data visualization. At the last second we picked up a senior math major named Feyman Liang.

We rigged up this holographic display with a spare monitor, some sheets of plastic, and a hand-full of Bloomberg stickers. 

at first we thought we would mess around with some fit bit health data.

However, after listening to a talk by the Senior Engineering Lead from Google Ideas we decided we wanted to use data that could actually change people’s world view. 

We ended up using Google bigQuery on the GDelt database to pull a list of all events of material cooperation and material conflict between cities in the past month. Material cooperation might be Apple(Cupertino) opening a new factory in Beijing. Material conflict might be DC blocking flights from Nigeria due to the threat of Ebola. 

Each of these events what associated with a location and an edge width(<1 being a conflict | >1 being a cooperation). We mapped this data onto a 3D globe in order by degrees of separation from the user interacting with the map. 

To further experiment with the idea of immersive 3d experience with data we mapped the interactions to nod ring gestures. Here is a badly filmed demo:

Here is the demo website. Press ‘f’ to flip it as you probably aren’t using an inverted display. All navigation is done with the arrow keys. Once you’re on the map page you can drag with the cursor to turn the globe. Press to left and right arrow keys to show conflict edges one additional degree of separation at a time. Use up and down to show cooperation edges. 

Informazione e conflitti

Secondo un gruppo di ricercatori statunitensi, però, questo genere di studi non tiene conto di uno dei parametri più significativi disponibili: la copertura mediatica dei conflitti. La maggior parte dei dati non include la quantità e la qualità delle notizie apparse sui mezzi d’informazione sui singoli eventi. In questo modo “le proteste del 2011 in piazza Tahrir in Egitto o quelle a Kiev nel 2013 contano come un singolo evento, e non c’è modo di distinguerle dalle decine di migliaia di altre proteste avvenute contemporaneamente nel mondo”, spiega Kalev Leetaru dell’università di Georgetown su Foreign Policy.

GDELT and ICEWS are arguably the largest event data collections in social science at the moment. During their brief existence they have also been among the most influential data sets in terms of their impact on academic research and policy advice. Yet, we know little to date about how these two repositories of event data compare to each other. Given the nascent existence of both GDELT and ICEWS event data, it is interesting to compare these two repositories of event data. We undertake such a comparison for fighting in Syria, and for protest behavior in Egypt and Turkey, from 2011 to the present. You can view the visualizations here.

This is what data from a world in turmoil looks like.”

Penn State doctoral candidate John Beieler has mapped data from the Global Database of Events, Language, and Tone (GDELT), creating images that include a stunning visual representation of worldwide protests since 1979. While there are limitations to the data and the mapping itself (as the linked Foreign Policy article does a good job of explaining), the map still shows some incredible trends.