How Can Big Data Be Used For Social Good Or Innovation

Today, the main hype around big data concerns tracking and targeting consumers. Nonprofits and other social change organizations are lagging their counterparts in the scientific and business communities in collecting and analyzing the vast amounts of data that are being generated by digital technology. Social entrepreneurs and activists should seize on humanitarian opportunities that the current data deluge is providing and make that difference.

The term “big data” is used to describe the growing proliferation of data and our increasing ability to make productive use of it. A myriad of big data projects have been undertaken in scientific domains.

Data-driven intelligence has been used successfully in technical and business endeavors, but a very different situation prevails in the social arena. Large gap exists between the potential of data-driven information and its actual use in helping solve social problems. Some social problems can be readily solved using big data, such as using traffic data to help ease the flow of highway traffic or using weather data to predict the next hurricane. But what if we want to use data to help us solve our most human and critical social problems, such as homelessness, human trafficking, and education? And what if we not only want to solve these problems but do so in a way that the solutions are sustainable for the future?

Social problems are often what are called “wicked” problems. Not only are they messier than their technical counterparts, they are also more dynamic and complex because of the number of stakeholders involved and the numerous feedback loops among inter-related components. Numerous government agencies and nonprofits are involved in tackling these problems, with limited cooperation and data sharing among them. Most of these organizations have inadequate information technology resources, compared to their counterparts in the hard sciences who work on technical problems or in business who have ready access to financial, product, and customer information.

Beyond the infrastructural impediments that social sector users of big data face, data itself can be a problem. Oftentimes, data are missing and incomplete, or stored in silos or in forms that are inaccessible to automated processing. Then there are policy and regulatory challenges that need to be faced, such as building data-sharing agreements, ensuring privacy and confidentiality of data, and creating collaboration protocols among various stakeholders tackling the same type of problem.

Whereas there is no doubt that nonprofits, government, and other organizations will continue to invest in big data technologies and programs, questions still remain about how beneficial those investments will turn out to be. The value proposition of big data is clear for tackling complex technical and business problems, but the jury is still out on how well big data can tackle complex social problems.

Why Data Is Big

Data, or individual pieces of information, have been gathered and used throughout history. What’s changed recently is that advances in digital technology have significantly increased our ability to collect, store, and analyze data. Today, big data is used to refer to data sets that extend beyond single data repositories (databases or data warehouses) and are too large and complex to be processed by traditional database management and processing tools. Big data can encompass information such as transactions, social media, enterprise content, sensors, and mobile devices.

There are multiple dimensions to big data, which are encapsulated in the handy set of seven “V”s that follow.

  • Volume:considers the amount of data generated and collected.
  • Velocity:refers to the speed at which data are analyzed.
  • Variety:indicates the diversity of the types of data that are collected. Viscosity: measures the resistance to flow of data.
  • Variability:measures the unpredictable rate of flow and types.
  • Veracity:measures the biases, noise, abnormality, and reliability in datasets.
  • Volatility:indicates how long data are valid and should be stored.


Although all seven Vs are increasing, they are not equal. Consider volume. The world’s collections of data are doubling every months, presenting the public and private sectors with new opportunities to transform information into insight. As the volume of data increases along with the tendency to store multiple instances of the same data across varied devices, the science of information search and retrieval will have to advance.

The most challenging V for organizations is variety. Organizations have built information systems to tackle data elements in specific categories. The challenge for many organizations is to find economical ways of integrating heterogeneous datasets while allowing for newer sources of data  to be integrated within existing systems. Ensuring that the data collected are of sufficient veracity is also critical.

Failing to Use Big Data

Increasingly, traffickers make use of mobile phones, social media, online classifieds, and other Internet platforms. Data from these technologies could be collected and used to identify, track, and prosecute traffickers, but a few daunting truths remain: The illicit nature of human trafficking makes it difficult to collect primary data, primary data collected from some organizations may be unreliable, and we lack reliable indicators to measure anti-trafficking program and policy success. Furthermore, most information collected on human trafficking is stored in a manner that meets organizational needs, but not global needs. Because of data privacy and security issues, data held by various organizations are seldom shared in raw form, limiting the creation of global, and big, datasets.

Making matters worse, agencies combatting trafficking often compete with each other for scarce resources, whether grants and gifts or recognition from the press and the community. Because of this competition, data sharing between agencies—and even between agencies and the public—is rare.

Barriers to Creating and Using Big Data

There are four principal reasons for the relative lack of structured big data for social problems: Data are buried in administrative systems, data governance standards are lacking, data are often unreliable, and data can cause unintended consequences. The issues being tackled in the social sector are often more complex than they are in business or science, making the use of big data more difficult.

Data are buried in administrative systems | Most organizations collect data to meet operational needs, and those data are often buried in the organization’s administrative systems. To overcome this problem, organizations are trying to find ways to build large datasets that can be more widely used. This obstacle needs to be overcome before we begin thinking of connecting datasets across organizations.

Data governance standards are lacking | A second challenge in our ability to use big data for social problems is the lack of adequate data governance standards that define how data are captured, stored, and curated for accountability. As a result, large inconsistencies exist and the data being captured are often not readily suitable for analysis. In many cases data need to be transformed before they can be used, and transformation is costly. Analysts often struggle with integrating different datasets because they lack good metadata and the quality of data is poor.

Data are often unreliable | The abundance of data provides great opportunities to researchers trying to understand and solve social problems, but unfortunately much of the data is unreliable. Simply having a lot of data does not necessarily mean that the data are representative and reliable.

Data can cause unintended consequences | Big data users can find themselves facing the unintended consequences of exploiting big data with no regard for data quality, legality, disparate data meanings, and process quality.

The Promise of Mobile Phones

The rapid proliferation of mobile and Internet usage allows for the collection of unprecedented amounts of information. Most modern mobile phones contain global positioning system technology, which identifies the geographic location of the phone. In addition to location data, mobile phones contain a treasure trove of information, such as call logs, SMS messages, and social media postings. A mobile phone acts as an individual sensor collecting relevant information from its environment, which when aggregated and analyzed with information from millions of other mobile phones can lead to the discovery of important information, which can then be disseminated back to people on the ground via the same mobile phones.

Steps to Increase Use of Big Data

Big data has enormous potential to inform decision-making to help solve the world’s toughest social problems. But for this to happen, issues relating to data collection, organization, and analysis must first be resolved. The following four recommendations have the potential to create datasets useful for evidence-based decision-making.

With the proliferation of open data platforms, citizens are creating new ideas and products through what has become known as “citizen science.”

Building global data banks on critical issues | The global community needs to create large data banks on complex issues such as human trafficking, global hunger, and poverty. The data bank would have the capacity to hold multiple different data types along with metadata that describes the datasets. For this to happen, multi-sector alliances that promote data sharing on thematic issues need to be created. At the 2012 G-8 Summit, leaders of the world’s largest economies and four African heads of state met to discuss and commit to a new phase of efforts to fight hunger and food insecurity.

Engaging citizens and citizen science | Big data is not the sole province of professionals. Citizens can also be enlisted to help create and analyze these datasets. With the proliferation of data through open data platforms, more and more citizens are creating new ideas and products through what has become known as “citizen science.”

Build a cadre of data curators and analysts | Today, not only do we have a shortage of data curators and analysts who can tackle social problems, we have limited avenues for our existing personnel to receive the necessary training and build competencies. For the most part, we have left data science to the sciences and business. The social sciences have often equipped students simply with the basics of statistics. This approach is unacceptable if we are to take advantage of big data. We need to equip students and analysts with the necessary skills to curate data so as to create large datasets.

Promoting virtual experimentation platforms | To increase our understanding of how to use big data for tackling social problems, we need to promote more experimentation. Virtual experimentation platforms, which allow individuals to share ideas, interact with others’ ideas, and work collaboratively to find solutions to problems or take advantage of opportunities, can bring interested parties together to create large datasets, develop innovative algorithms to analyze and visualize the data, and develop new knowledge.

The Future of Big Data

Business and science have shown that big data’s merits are undeniable. Social sector organizations must now figure out how they too can incorporate this type of decision-making capability into their operations. The potential for growth and innovation exists, but there are serious obstacles to overcome. The issues that are being tackled in the social sector are in many ways more complex than they are in business or science, making the use of big data that much more difficult. In addition, greater attention must be paid to the rights, privacy, and dignity of their constituents.

In spite of these obstacles, progress is being made. Public sector agencies have made it clear that data are an important element of social innovation. Institutions such as the US government and the World Bank have made their data available to the public for mining and further use. Individuals are using the data to create innovations, mainly apps, to address a particular social problem.




N.B: The views and opinions expressed in this article are those of the author and do not necessarily reflect the official  position of the African Academic Network on Internet Policy. 

Related Posts