Author: dbeaton17

There’s a better way to evaluate campaigns (including digital) than traditional MMM

dbeaton17 Leave a comment

This post appeared originally on the Crater Lake blog, here

Marketers have yearned for the data to help them decide how much to spend, and where for many years, data that would give them answers to questions such as:

How much should I spend on advertising?

How do I best allocate this budget to channels, content, markets and over time? What is my real ROI and lift in sales volumes?

More than half a century ago, statistical modeling methods began to be applied to answer questions like these. Over the years, as marketing began to broaden its remit beyond media advertising, these methods began to incorporate other elements of the marketing mix. As a result, the term MMM evolved from Media Mix Modeling to Marketing Mix Modeling. Today, MMM is often, and fairly, criticized for leaving marketers grappling for answers that the techniques themselves are just not suited to deliver.

  • How does MMM help with the details of planning and buying online media?
  • Can MMM measure the long term effect of advertising?
  • Can MMM help us understand the effects of creative?
  • Can MMM incorporate the impact of customer experience?

In short, traditional MMM is too limited in its capabilities to be useful for today’s marketer. To try to tackle these new requirements, Crater Lake has chosen to use a wheel as a metaphor. Each spoke on the wheel represents a channel. In the hub, all channels come together.

If we want to build a model that measures the impact of all channels we are operating in the hub. In the hub, we need to describe each channel using variables that ALL channels have in common.

But this inevitably means some of the data available to an analyst in a spoke is not used.

In the Crater Lake approach, our omnichannel model (hub) uses a unit of analysis that is a combination of geography (small areas) and time (usually day).

In a spoke, the unit of analysis may be different. Consider an email targeting and measurement model; that would use a unit of analysis that is an email address combined with day.

This means that while the “hub” model can certainly be used to measure cross-channel effects such as total lift, and it can also be used to allocate budget between channels, it’s application within a channel (in a spoke) is limited to the data available.

In our view, we compensate by building complementary models for each major spoke, and then solve for optimization across the entire system. When we want to integrate the hub learning with a spoke, we are pushing hub data back out into a spoke model. This means a spoke model (representing a single channel) can incorporate data about the presence and effect of other channels.

Our aim is threefold:

  1. to learn what we can from the totality of the effort.
  2. to push that learning into the data we have available on each individual channel thus allowing us to improve how we optimize effort within a channel.
  3. to optimize, across all channels.

There are many individual hurdles we need to overcome, given the huge quantity of data available on some channels and the comparative paucity out there on others. We believe that there really isn’t any point complaining about the lack of data, or its inconsistency. Over time the situation will improve but for marketers the time is now, and the need is immediate.

We work on the basis that 90% right and on time beats 100% right but too late every time.

We work with what we’ve got, and what we can get.

Making Sense of It All: The Clear Performance Narrative

dbeaton17 Leave a comment

This post appeared originally on the Crater Lake website, here

How many times has the boardroom or hallway conversation at your company contained a variation on the phrase “we are just not on the same page”. It could be referring to a difference of opinion or insight between marketing teams, or between marketing and the C suite. Or it could be a deep debate about strategy or overcoming challenges. The problem is, as one exec puts it, most marketers now enjoy (endure?) a glut of information. What is missing is a narrative that pulls the pieces together and makes sense of it all.

That’s why we set out to create the Clear Performance Narrative, one of the key deliverables we provide to Crater Lake & Co clients.

What we quickly discovered is that, occasionally, the problem with creating a Clear Narrative is that some pieces of crucial data/insight are missing, but you only discover that as you knit the story together. The story needs more signal, and less noise.

To qualify as “Clear” the narrative must not only answer why we are getting the performance we are, but also, what we can do next to improve the outlook for the business.

We must sort out cause and effect, and quantify the impact and economic value of all marketing activities.

We must be able to validate our analytics, to provide evidence to all marketing disciplines and the C suite that the narrative we build around our insights is reliable.

We must be able to quantify incremental lift in a holistic way, measuring advertising alongside other drivers of the business, whether those drivers are controlled by marketing or not.

We must be able to balance short term and long term, which also means balancing risk and reward. We have the tools (or some of us have), but applying them is challenging for leadership without context. As one wag put it “facts tell, stories sell”.

A clear performance narrative is one readable by any marketing discipline; indeed, every discipline should see themselves in it, and find and endorse their place in the story.

We must be able to build a clear picture of our customers and prospects in ways that are not just analytical, but empathetic. The narrative is not just dry numbers but human touch and feel.

Think of the narrative as the story of your brand in motion. When done right, we can clearly see where we have been, where we need to go, and how to get there. The very best stories invite the listener and reader to dig in, get invested in the outcome and root for the heroes. Only in this narrative, the heroes are not mythological beings, but our team members, their ideas, and their work.

The story is one of progress towards a goal, one that all marketers endorse and believe in. There are obstacles to be overcome, opposition to be defeated or outmaneuvered, challenges emerging that were previously unseen. How will the story turn out? It very much depends on the characters that write it.

So why the emphasis on story, or narrative? Aren’t we just talking about yet another 70 page powerpoint deck?

No, because this is NOT another 70 page powerpoint deck. We have enough of those.

As Jonathan Haidt (social psychologist and Professor at NYU’s Stern School of Business) put it “The human mind is a story processor, not a logic processor.”

Not a surprise to advertisers, who of course do this every day in their creative work. So now we turn the techniques of story-telling to the task of synthesizing the work of advanced analytics, of research, and of marketing and business leadership.

Making sense of it all: it should be an exciting journey.

-The Partners of Crater Lake & Company

Bots Don’t Buy: The Model in Action

dbeaton17 Leave a comment

As we saw in our last blog post, bots are eating digital ad budgets to an unacceptable degree. Let’s do something about it.

We have developed a predictive model that processes the massive number of signals from the digital ecosystem to predict who will buy….not who (or what!) will click. Buying includes on AND offline purchases (in this case; subscribing to a new service and paying at least one invoice); so this model varies significantly from ones developed on digital data alone.

The model was tested for a telco client for a campaign designed to acquire new customers. Channels included both search and digital display.

We found a very large difference in the activation rates (number of customers divided by households) between groups that scored highly likely to buy (and therefore recommended) and those that scored much lower.

Targeting recommended higher scoring groups led to an activation rate that was 2.1 X higher than recorded for other parts of the audience. But the overall targeting for the campaign, outside of the model, and based on chasing clicks, saw 70% of the budget go to low, out of target groups.

The path to improvement is clear: focus on in-target, high scoring groups. Given the digital inventory available to us, the entire budget could be spent there to secure these much better results.

In the battle against bots, we can have our cake and eat it too. We can keep money out of the hands of unscrupulous bot owners and in the hands of reputable digital publishers. AND we can improve performance, selling to real people who have a real interest in what we have to say. Win-win.

To learn more about how Bots Don’t Buy can help your digital marketing, contact us here.

Bots Don’t Buy: A better approach to digital display ad planning and buying

dbeaton17 Leave a comment

The digital ad display market has been plagued with issues, but it should still be an important channel for all marketers, both to build sales and to build brand.

The challenge is to overcome 5 problems:

1. Bots eating ad budgets

· According to Google, 56% of ads are never seen by a human

· A study by Forrester found 69% of brands spending $1 million per month reported that at least 20% of their budgets were being lost to digital ad fraud

2. Lack of respect for people’s privacy

· an industry focused on surveillance has created the largest consumer boycott in history…ad blockers

· at the same time, new privacy regulations in Europe, the US and Canada demand transparency and restrict the use of personal data

· Google will deprecate the third party cookie in Chrome browsers in the near future and Apple’s approach to email and ad permissions will greatly restrict some common practices

3. Bad, unreliable measurement and attribution

· digital attribution tools impose a rule on clickstream data, vs using that same data to intelligently determine cause and effect.

· some digital attribution tools pretend offline media doesn’t even exist!

4. Targeting without context

· When marketers don’t develop media plans with synergy in mind…how channels can reinforce each other, vs cannibalize…budgets are wasted

· cross-channel impacts can add a double-digit boost to performance…added impact without added budget

5. No forecast capability

· common digital planning tools are not predictive at all, let alone predictive of the incremental lift digital can have in the presence of offline channels

Navigation ME is pleased to announce we are launching a solution to these problems. Since bots don’t buy, our models will act as filters to avoid ad fraud. These models focus on signals (plentiful in the digital ecosystem) that differentiate human traffic, associated with purchases, from click traffic from bots…that never buy. These patterns allow us to focus digital ads on target audiences that are disproportionately driving sales volume lift and avoid audiences that will spend little to nothing with your brand.

In addition, our new predictive and optimization models are designed to drive incremental sales from digital ad buys. Since we will use privacy compliant data, we avoid making the problem of invasive tracking worse. And since our models don’t need personal tracking to determine cause and effect, we can help you avoid regulatory problems…and keep a cleaner conscience. Most importantly, these models integrate data from offline media, so that we can quantify digital ad lift without making attribution errors. We also, in this way, take full advantage of the synergies that exist between channels that are hard to measure but powerful in effect.

We believe the digital display marketplace can be rehabilitated; and turned into a reliable growth engine for your brand. To find out more, contact us here.

Pushing the Envelope: How Custom AI turned Neighbourhood Mail into a Profit Machine

dbeaton17 Leave a comment

Every marketer must make a decision as to how much to spend on each channel of communication available to them.  In our experience, budget goes to…and stays with…channels that consistently deliver measurable results at a par or better than any other channel available.  But many marketing teams today are overworked and understaffed.  They need channels that are easy to plan and buy, delivering those results with less risk and less effort than competing alternatives. 

Digital platforms have done an excellent job at making it easy to plan and buy.  (delivering profitable results?  not so clear).  This post will focus on how a channel that has been with us for a very long time…Neighbourhood mail…was re-invented by adding custom AI tools, to become a profit machine. 

We will look at a case involving a telecommunications company, selling Home Phone, Internet Service, Mobile and Bundles (combinations of services at attractive prices).  The company’s competitive advantage is price; they offer services at a 20-30% discount to national major brands at comparable quality. 

Before using Neighbourhood Mail, the company relied on digital campaigns, especially search, to sign up new customers.  While the success rate of such work was acceptable, volumes were not. Neighbourhood mail was considered as a channel that could deliver reach to all serviceable households in their footprint, and sign up customers at an attractive cost. 

Often Neighbourhood Mail is targeted by demography or region, and used as a media to reach a lot of households.  But for this case, such an approach yields poor results; more precision in targeting was needed to generate a good result. 

To support this effort, Navigation ME built a set of models, each of which predicted, for a given time period, the activation rate (number of new customers divided by pieces mailed) for a given postal walk (the unit of NM delivery) for a given product. 

In building these models, we found that many factors affected activation rate: 

  • How many times we had sent mail about a given product to that postal area, and how recently (or how many times we had sent mail about another product) 
  • Impact of other channels used in the same areas at the same time 
  • Demography; including income, family structure, and age distribution 
  • The baseline of demand for a given product 
  • Pricing vs competitive alternatives 
  • The penetration of current customers in that area, and the trend; was new customer acquisition outpacing churn, growing penetration?  How quickly?  Or was it the reverse; we are having trouble keeping customers in that area and replacing them when they go 

The difference in predicted activation rate, and therefore cost-per-activation, was large; top deciles were often at 4x as high an activation rate as the 5th decile; and the bottom deciles typically delivered hardly any activations at all.  (note; the measurement of activations took into account multi-channel effects, isolating the impact of Neighbourhood Mail for better decision-making).  Model accuracy rates, from pre-campaign scoring to post-campaign evaluation, are consistently in the 92-95% range. 

The cost of delivering a given service depended on its footprint.  This meant that since we wanted to calculate the predicted profit of mailing a given postal area, we needed to know the cost of delivery in that area. 

We found that churn was not uniform in all areas; there was large variation with some neighbourhoods consistently churning at higher rates.  We needed to take this into account as well, since the value of a new customer from those areas would, all other things being equal, be less than in areas that showed lower churn risk. 

For each postal walk, we had 7 scores; one for each product/footprint combination. For a given postal walk, the score for one month would be different depending on whether we mailed the previous month or not. 

To avoid this level of complexity from overwhelming decision makers, we added our custom optimization algorithm, tasked to find the optimal distribution of mail: 

  • For each postal walk… 
  • …for each of the three months of the campaign… 
  • …for each product… 

This meant some postal walks could get Product A for months 1 and 2, and Product B for month 3.  Or a given walk might not get any mail at all, if the model predicted the costs would exceed the profits of the predicted number of new activations. 

Interestingly, about 1/3 of new activations came from existing customers, something the model picked up by taking penetration momentum into account.  So in this case, Neighbourhood Mail became not only a profitable new customer acquisition channel, but a good cross-sell channel as well. 

For planning purposes, we created an ROI curve: 

Each point on the curve was the result of the model doing an optimal allocation of available budget across waves, time, and product.  All the work of taking full advantage of the model’s learning and accuracy was done for the planners by the software. 

Since there is no free lunch, the curve flattens as more pieces are added.  This reflects what we know about all advertising; after a given point, additional touches add nothing to business outcomes.   

The ROI curve enables a planner to increase spend with confidence of the outcome (risk can be calculated explicitly for each scenario).  It also demonstrates the overall impact of this channel; for a mailing of 2.5 million pieces, for example, a profit after all variable costs and campaign fixed costs are paid of over $4.5 million is generated. 

We have been running this system for this client for many years; with results in line with model expectations each time.  Predicted vs actual is carefully tracked by decile so we know the model continues to track today’s market conditions. 

Taken together, the reach of Neighbourhood Mail plus the decision-making impact of Navigation ME’s predictive and optimization models has created a profit machine…high performance, with consistent results, at low risk and that is easy to plan for.