Skip to content

3 Analytics Challenges: Context, Endogeneity and Spurious Results

May 16, 2014

Donor Analytics ContextAn interesting visual depiction of spurious correlation (check it out here) reminded me of my grad school days and the rigor with which I would build hypotheses. Rather than let R, SPSS, or Excel correlate away and then proclaim some amazing finding, I started from the reasons and results I expected to validate with data. The difference is, all too often, that the former approach tells you very little due to endogeneity, spurious results, and the lack of context.

Some organizations–Google is known for this–will say “don’t worry about the why”. Some have referred to this approach as “theory-free“, a nice euphemism to indicate how little long-term value we might find in these correlations. Now, for consumer behavior where Big Data is truly present  perhaps this works. But, data points are rarely available for nonprofit analytics in the same way as, say, Target and Wal-Mart have data…although there are new options underway, like David Lawson’s newsci.co.

And, if you talk with a gift officer who’s been disappointed with predictive modeling results, you see a different picture. From that vantage point, the analytics results are frequently devoid of context. The result confirm what we already knew (“these prospects look rich! they live in a nice neighborhood!”) or reflect a pattern we already see (“they gave last year! let’s ask them again!”). Yet, modeling doesn’t typically improve relationships with prospects.

A big culprit: Context. Donor context is critical in building relationships. And, context is quite challenging to incorporate into modeling. The following are real examples of discussions about potential prospects surfaced by a context-free model:

  • “Sure, Jane looks promising, but we don’t have a phone number to reach her and no volunteer connection, so how likely is it she’s approachable?”
  • “Absolutely, Ed looks great, but did you know he just filed for divorce?”

The solution to this issue isn’t to cast off analytics. It’s to improve it. Start with and add in theory. Guard against spurious results. Don’t elevate an endogenous variable as meaningful. And, most of all, our industry needs resources that can actually add context to results. As a student of philanthropy, I am anxiously awaiting the time when our new science of analytics better delivers on the hype and improves our understanding of donor behaviors, while avoiding endogeneity and spurious results.

Advertisements
5 Comments
  1. Interesting take Chris. I would hope that few are promoting predictive analytics as improving relationships–its goal is to just narrow the suggested list of folks to try and build relationships with (form a major gift perspective)

    Some endogeniety is unavoidable–past giving is best predictor of future giving. Also most of our databases are transactional (only reason we know anything about someone–their contact info or interests, is because they gave $).

    I think there is a counter argument to made here–the value of lack of context. It can force organizations to examine current prospecting heuristics which can be even more spurious and endogenous (I know him well, he gave big $ elsewhere, she loves the football team).

    I contend the resources we need are better consumers of analysis, and application. Understanding its strengths, and its gaps (all strategies have them). Bad information isnt the issue–its what to do with it–when to use it, when to combine with other approaches, and when to leave it parked on the sideline.

  2. Thanks, Alex. Thoughtful as always (and I hope all is well…you’ve been missed). Your points are academically useful, but the real-world is where solicitations happen. Having asked folks to give and having interviewed hundreds of gift officers, I would say that bad (or missing) information is certainly the issue. And, too often, data analytics is producing bad data. That is, data point us in the wrong direction too frequently or simply remind us of what we knew before we added that Chi-square to a 10-year, $1,000+ donor’s record.

    And, while I agree that we make a lot of decisions for our donors (“oh, you know what, I bet this week is busy for Jane with her board meeting, I’ll call later.”) due to some of the context/bias we bring, our industry appears enamored with data science when we need to be making more phone calls and actually getting to know our prospects. Might a model help us know whom to call? Sure. Do I see many known, ripe portfolios that are thoroughly engaged? Nope. Survey data over the last decade or so show that gift officers are spending less time (i.e., fewer in-person visits) with donors than before as we slice-and-dice more data than ever before. As a relationship business, we are out of synch. As you indicated, certain applications for analytics may make sense. Modeling for direct response segmentation applied to really large membership organizations such as Make-A-Wish is an example. The challenge I am seeing is unvalidated and/or misguided analytics devoid of context leading to strategic miscalculations and tactical blunders at the campaign strategy and major giving levels.

    Your comment on the endogenous aspects of more simplistic assignment processes is an important one. You’re right that decisions are often based on context and bias alone. However, as we’re seeing most significant campaigns generating 80%-95% of their dollars from top donors (most of whom are well-known to the organization in advance of any modeling), I would rather ask my volunteers, physicians, curators, etc. whom they believe we could solicit for very, very large gifts. That’s endogeniety I can believe in! There is a pipeline to fill, etc., and analytics can do a decent job here, but the limitations are greater than our industry appears to accept.

    There is a place for analytics and I’d love to see these limitations surmounted. At the same time, I’d like to see the positioning of analytics adjusted and the science improved. Otherwise, we’ll just keep learning a) our best future customers are our current customers and b) people that live in really expensive neighborhoods and have lots of assets have the most money to contribute.

    Thanks again for posting.

  3. A great post, Chris. I found myself nodding my head through the whole thing as I read it. What you describe is exactly the issue I have had with some analytics reports I have seen: they just verify what I call “the Big Duhs.” As you say, that “ a) our best future customers are our current customers and b) people that live in really expensive neighborhoods and have lots of assets have the most money to contribute.”

    But: we have to remind ourselves that, even though analytics in fundraising has been around for quite a few years, we’re still taking toddler steps in applying business methods to the nonprofit sphere. We’re still figuring out how to match what excites the analysts (and what they can provide) with what motivates the front-liners to act on the information. We’re still balancing how to capture/rent/utilize data to answer the real question that needs answering: “How can we better engage with, inspire, and retain our donors?”

    Analytics shouldn’t tell us who to ask right now (or even in this campaign) for major gifts; as you point out, insiders and key stakeholders should already know that. The questions we need analytics to answer for us are: Who will be our best future donors? Whom should we engage with now for major gifts 5, 10, 15 years down the road? Who will leave a legacy? Which of our team is great at inspiring donors and what is it that they do so well? What are our constituents most interested in, and how do we convert that interest into donations? And so many more!

    Thanks so much for writing this, Chris. We have to challenge ourselves as a profession to offer better, more relevant – actionable – analytics. We’ll grow into it, but we’re all a little impatient for excellence now.

    • Helen,

      Thank you for the wonderful and thoughtful post. Analytics is too often viewed as an immediate solution for major gift prospecting–a proverbial “silver bullet”. I fear one of the main drivers of this belief is the erroneous inclusion of wealth screening under the umbrella analytics. Wealth screening does provide what could be described as “instant insight” (aha, Alex is in fact a secret millionaire etc).

      Wealth screening however it is NOT analytics–it is data/information acquisition, and fundamentally no different from an NCOA, You would be hard pressed to find anyone who claims an NCOA is “analytics”.

      Many providers package these solutions together–which further adds to the confusion. I think the fuzzy/opaque line with which people discuss these two important yet distinct business processes has caused a lot of confusion, misunderstanding, and misapplication. When I would discuss projects with skeptical major gift teams I would say “I know you know the 300 best names for this campaign today–but who are the next 300, and the 300 after that?”. I think analytics works best when properly framed (in the major gift content particularly) as a predictive and planning tool. I would often use a forrest analogy–asking a major gift officer to envision themselves in a dense forrest of constituents. They are in the middle of their mature fully grown major donor trees, but need to know which direction to head to next to find the cluster of trees. The analogy was important to frame the idea–where are the major gift saplings? It can be difficult, sometimes impossible, with the naked eye to see a group of young trees in a dense forrest with limited time in our day to hike, let alone discern whether or not they will grow generously.

      The analogy is much stronger when shared verbally–but I think you get my intention. There have been many “false starts” with analytics that I have seen, and even been a part of. This is a critical time in this field, and I hope people like Chris, and Helen, respected thought leaders and practitioners, can continue to have discussions like this. Together we can continue our understanding of these powerful tools and apply them in productive ways.

  4. Thanks, Helen and Alex. You’ve both provided thoughtful additions to this issue. I trust my writing isn’t seen as dismissive of the future utility of analytics and is instead a call for even more thoughtful work in this space. And, I suppose my obvious impatience is spawned by our industry’s tendency toward anointing analytics as essential (i.e., a “silver bullet”, despite Alex’s wise caution) to great fundraising. This could be the case, but too often the analytics aren’t good enough to be a genuine driver.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: