In the last three years, several technology trends in have begun to intersect. First, social networks took-off, led predominantly by Facebook and Twitter. The data being created in real-time by these networks is exploding at exponential rates. While most of the world has been focused on how to harness “Social Data” for everything from building corporate brands online to monitoring social uprisings across the world, something else has taken place beneath the surface.
The technology and infrastructure required to manage and analyze this data is evolving in significant ways. Open Source Projects have spun-out of Yahoo, Google, Facebook, Twitter, and others that enable a community of developers the ability to store and analyze enormously large sets of data at relatively no cost. The corporate world has long used data and analytics to be more competitive but now this technology has matured and is available to the masses.
Concurrent with this emerging technology, the proliferation of inexpensive mobile devices in the developing world is on the rise (by some estimates there are over 5 billion mobile devices on the planet). As a result, a flood of data is created every day by the interactions of billions of people using computers, GPS devices, cell phones, and medical devices. Many of these interactions occur through the use of mobile devices being used by people in the developing world, people whose needs and habits have been poorly understood.
The net result: the amount of data generated by humanity is increasing exponentially and now the technology exists to do powerful things with it.
Researchers and policymakers are beginning to realize the potential for channeling these torrents of data into actionable information that can be used to identify needs, provide services, and predict and prevent crises for the benefit of low-income populations.
Open data, data for social change, data democratization, data philanthropy and Big Data for Development are all relatively new concepts that have emerged in the last few year to describe initiatives where companies, NGOs and Governments make use of their data for purposes of social change and international development.
While they may appear as nuanced monikers, each one will play a distinctive and critical role in the evolution of this space.