Let's chat! When’s a good time?
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
We all have heard the saying: “Change is the only constant,” which translates to the fact that everything changes, every time. While that is too philosophical, and since marketers – more so data-driven B2B marketers – are not really fans of philosophy, there is a need to “measure” change. And not just that, a business needs to “attribute”, “rank”, “predict”, “explain”, and hopefully even “bring about" change using data analytics. Since statistics is the mother of all real-world measurements, and probability is the grandmother thereof, the science of measuring change too lies somewhere close by.
Since surface (i.e., not deeply thought-out) definitions are easy to make but difficult to follow with real-world data, the real challenges are solved by taking support from the well-researched areas of data science and probability. One extreme is to discuss the topic of change with its extreme roots, but that could easily fill-up a book (which is under way), but we stick to measuring change w.r.t. marketing analytics.
The main questions from a marketing perspective are: “What main factors changed from last week to this?”, and “By how much did these main factors change?”. But before that, let us understand something even more basic.
Marketers take various targeting decisions on a daily basis – which audience to target, how frequently, and with what kind of marketing campaigns. They also have to keep track of various prospective customers, their journey, their overall statistics, and most importantly, reasons why a particular technique worked (or didn’t work) – be it a campaign or a strategy. And there are various goals when it comes to marketing. While some techniques could be motivated towards increasing reach (where it’s important to maximize the number of eyeballs a webpage could get), others increase conversions (at various points in a funnel – form-fills, email cold/warm calls, prospects, pre-sales, sales, etc.).
The foregoing marketing goals are achieved by measuring both static marketing performances (i.e., what happened today / this week), and also more dynamic, time-aware ones. While analyzing performance over time is a comprehensive view of marketing goals’ achievements / shortcomings and hence makes it cumbersome to digest multiple metrics of interest in a single frame, summaries of the same are preferred. And one of the summaries of a dynamic measurement of marketing performance comes in the form of change – that is, what changed, by how much, and why. In this article, we would focus on measuring overall changes, and would dig deeper into measuring the causes for the same in a follow-up blog.
Based on the business requirements, marketers focus on tracking and eventually improving relevant KPIs (key performance indicators). For the scope of this article, we would explain change analysis taking the example of one important metric marketers are concerned about – the number of leads they generate every week. They achieve this by driving relevant visitors to their website every week. Any change in the number of leads that they get as compared to what they expected calls for an investigation on the reasons for the difference between expected and actual metrics.
Since the first step into any such investigation is to measure and compare (with last week) some global performance indicators measuring the reasons for change, the same is the focus of the current article. Hence, keeping a webpage in mind, we take the example of measuring the number of visitors (those who reached the website), the number of leads (visitors who reached target), and the conversion rate therefrom (leads per visitor).
When we perform these measurements for two given periods of time (say consecutive weeks), we could compare them. Let V1, L1, and C1 represent the number of visitors, number of leads, and conversion rate of week 1 respectively, and let V2, L2, and C2 represent the same measurements from week 2.
The four simple change measurements one could perform are the following:
Going by the earlier example of website visitors, we could be interested in the factor change in number of visitors, (𝚫fV = V2 / V1), leads (𝚫fL = L2 / L1), and conversion rates (𝚫fC = C2 / C1).
Depending upon the business and the audience, there are multiple combinations of possibilities. We only cover some of them, summarizing overall (global) change. And as mentioned earlier, our next article on this topic would dive deeper into digging up the “reasons that drive this overall change.”
Let us start with a simple world, and slowly drift towards complex scenarios. Suppose our website had 1,000 unique visitors last week, and 1,000 new unique visitors this week (i.e., there was no change in the number of visitors). Of the 1,000 users last week, 20 signed-up for our newsletter, and the same trend continued this week as well. In other words, both last week’s and this week’s conversion rate was 2% (20*100/1,000). What does this tell us about this week’s performance over last? That it remained the same! In other words, there was a 0% increase/decrease in both reach and lead conversion rate.
Now, if we had more visitors (say 1,500) this week as compared to the last, with a proportionate increase in leads so as to maintain the 2% conversion rate, it would amount to a 50% increase in visitors and leads, but a 0% change in conversion rate.
On the other hand, if the total number of visitors had remained the same, and leads would have increased by 50%, this would increase the conversion rate by 50%.
If, however, the number of leads would have remained the same despite an increase in visitors (by, say, 50%), we would see a 33% fall in conversion rate.
It is also possible that the number of leads increased by a disproportionate amount, which led to an increase in conversion rate.
We just discussed how measuring overall change is straightforward: simply report the signed/relative/percentage difference or factor change in visits, leads, and conversion ratio between the two weeks. But this is only half battle won. What is ideal is to measure the “causes” for the change. For example, we know that a 2x increase in visitors (V2 = 2V1) and 1.5x increase in leads (L2 = 1.5L1) – and hence a 25% drop in conversion rate (C2 = L2/V2 = 0.75L1/V1 = 0.75C1) – happened. But why it happened is one of the most important questions change science has to answer.
A seasoned marketer can quickly understand the main factors that led to a drop or a rise in scale (#visitors) or conversion (leads/visitor). But where data analysis comes in is in short-listing such factors from the rest, and hence help the marketer with her weekly (or periodic) decisions. In this article we only give an intuitive idea of what causes change (and how we measure it). In part 2 of this series on Change Science, we discuss the exact procedures and methods to measure factors that cause change periodically.
The more we know about our customers, the more our analysis benefits. While measuring change, as it was mentioned above, we usually track the number of visitors (V), the number of leads (L), and the respective conversion ratio (C). Along with mere counts, one ought to measure the profiles of such visitors and leads – in both weeks. For example, with a 2x and 1.5x raises in visitors and leads, to know what caused it, one has to track how the properties of visitors and leads have changed. The properties we are referring to are usual user/event based properties – from simple demographic ones such as Location (country, city, etc.), User Agent (browser, OS, etc.), etc. to marketing-oriented ones such as Referrer (domain, campaign, etc.) – along with their values (a “property=value” combination looks like “country=India”, for example). In summary, in addition to tracking overall statistics (visitor and lead counts), we also track counts “grouped-by” their properties.
Needless to mention, such properties are too many to count, and property=value combinations are even more. This is where Factors.AI comes in. We track-down each factor, measure the change attributed to the factor, and surface top insights. And we repeat the process for any event pair combination (in say, a funnel), for every property, and for every consecutive period of interest. We urge you to follow-up with us in the next article that describes how exactly we surface the main change factors via our very own Weekly Insights feature.
Stay tuned.
Get the latest best practices in Marketing Analytics
delivered to your inbox. You don't want to miss this!!
