By Tim Haight

The worlds of Big Data and foundations have collided. IT at foundations had better get ready for it.

The theme of this year’s Technology Affinity Group (TAG) conference, to be held in Miami in November, is “Data: Gather, Analyze, Act!” TAG is the premier conference for IT professionals at foundations. It’s pretty clear that Big Data is on foundations’ IT agenda.

Beyond that, I see three trends indicate that using data at foundations is about to take off. First, the vision of how foundations can use data is getting clearer. Next, helpful resources are starting to become available. Third, Big Data technology is becoming less expensive. It may even become simpler to use.

There’s also special news for IT at foundations: You are going to be involved. Unlike social media, a recent trend that mostly occurred outside of foundation IT and was mostly handled by the communications department, Big Data is going to involve infrastructure choices and expenditures.

So much for the executive summary. Here are the details.

The Big Data Details

The first thing that has to happen in order to get the promise of Big Data down to a foundation’s size is to decide what you need data for. It’s easy to get hung up at this stage. Some kind of silly loop between “what we need” and “what’s out there” can induce a kind of paralysis. Fortunately, innovators are making suggestions on both fronts. Lucy Bernholz, who has made quite a name for herself in this area, did a recent op-ed piece in the Wall Street Journal, succinctly expressing a vision of what’s possible:

Bernholz wrote, “More, better and faster information can help donors channel dollars to the organizations that are most effective. It can help nonprofits identify the strategies that work best at fulfilling their missions. And it can guide philanthropies to the fundraising tactics that are most successful.” From here, it’s not hard to get to specific topics: monitoring and evaluation, sharing results, transparency, establishing reporting standards and analyzing the data from these. And, as Bernholz points out, nonprofits can be helped to use data to evaluate their own activities and to see how others are doing. Both nonprofits and community foundations need help with fundraising, which puts them close to the way commercial firms are using big data.

It’s gone further than vision, however. A webinar put on by the Stanford Social Innovation Review in February, entitled “Data-Driven Strategy in the Social Sector” attracted more than 3,000 participants. After the webinar, one of its panelists, Jonathan Sotsky of the Knight Foundation, blogged and mentioned some “mechanisms nonprofits can use to promote more data usage and learning,” including some practices adopted at Knight:

• Ensuring all new grant projects define clear outcomes and an initial set of indicators for measuring success
• Planning from the outset of assessments to create an external version of reports (and to share raw data openly when possible) to encourage broader learning in the field
• Setting aside time at each board meeting to discuss what we’re learning and implications for our ongoing work
• Holding staff sessions to discuss findings from assessment and research projects

Sotsky also mentioned several organizations providing resources for nonprofits and foundations to use to get started with Big Data. I put together a list of resources that may also be of interest in a previous CGNET blog post.

Big Data in Office?

As far as the promise of the price of Big Data coming down, this is still a work in progress. Recent comments by Satya Nadella, Microsoft’s new CEO, are encouraging, however. In a post on the Official Microsoft Blog, Nadella presented a vision of “when every employee can harness the power of data once only reserved for data scientists and tap into the power of natural language, self-service business insights and visualization capabilities that work inside familiar apps such as Office.” At the same time, Nadella announced the Analytics Platform System (APS), which “combines the best of Microsoft’s SQL Server database and Hadoop technology in one low-cost offering that delivers “big data in a box.”

Finally, the role that IT will have to play in this has not gone unnoticed. As Accenture put it in a recent article in The Guardian, “Big data has been adopted across the private, public and nonprofit sectors, but for many organisations the first real challenge is adapting their infrastructure to maximise their potential gains from this new resource.” And for that, foundations are going to turn to IT. Are you ready?