How to run content strategy experiments to boost traction?

By Tom Greenwood

Move as many people as possible with your digital content.

Experimentation guarantees impact

Experimentation is everything in digital media. Strategy is only ever our best guess. Maybe it will work, maybe it won’t, but with digital media we get instant feedback in the data. We can see if it’s working or not, and we can continually tweak things to try to make it work better.

Those tweaks are experiments. Again, we don’t know if they’ll definitely work or not, but if we keep making tweaks and checking to see how our audience reacts, we keep learning what our audience wants from us. It’s the best way to understand who we need to be to create impact in the world.

So how does it work?

Let’s take an example from our own content last year. We collaborated on a really cool research project that showed how Meta’s anti-porn policy prevents young people from getting important information on sexual and reproductive health. It was covered by the New York Times and it got 100+ million reads.

Meta censor SRHR content ads

Start with a hypothesis

Our own facebook post promoting it got 48 engagements and 2 reposts. That’s not too bad, especially relative to our usual levels of engagement at the time. But maybe it could have done better. Experimentation is how we find out.

Experiments always start with a hypothesis: “If we do X, then Y is probably going to happen as a result.” So let’s take a look at the original post and figure out what stopped it from engaging more people. We can’t know exactly but we can take a guess. And then that guess becomes our experiment hypothesis, and we can design an experiment to either prove or disprove it. Let’s start with this:

Our hypothesis

Overall though, we’re guessing the problem (if there was one) was probably that this didn’t connect with our audience’s current experience. The big headline on the NYT article is about Facebook ads being rejected, and that’s not going to jump out at people unless they work in social media marketing. It didn’t spark an emotion either – unless you already know about this issue (and feel strongly about it) there’s not enough in the NYT headline to make you care. And as we all know, emotion means everything on social media.

So there’s lots to test there. If we were purists, we’d only test one variable at a time, but that’s boring, and we’re smart enough to work out what actually makes the difference. And if it’s still unclear further down the line, we can always untangle it with more testing. So let’s start again.

Always audience first

Successful content always always always starts with the audience. Our goal is to move people – as many as possible if we want to create social impact. So how do we frame this in a way that people want to engage with? Well, first of all, who could be interested in it? Who has a need for it? Whose lives is it relevant to? 3 clear audience segments spring jump out right away.

1. Young people– for obvious reasons, they need information about sexual health issues. This directly actually affects them
2. Parents– they want the best for their kids (obvs), and good info is essential for health
3. Current audience– people who are interested in media, tech, society and young people

So what we’ll do is one version for young people, and one version for parents and we’ll target those two versions PLUS the original version at all three audiences.

Content for young people

Our first question is, what is this giving them? Why should they click on it? They already know sex is censored on insta, so that’s nothing new. Nothing new = no point in clicking. So what can we give them that’s worth anything?

How we add value

Maybe we can give them a new perspective. Maybe we can help them express their frustration at being patronized and talked down to. And if they never saw the harm caused by platform prudishness before, then maybe we can add something of real value to their understanding.

Talking their language

Next question, how do we make the visual and the copy meet our audience where its at? How do we talk their language, and add to their conversation (and not butt in with something that’s irrelevant or old news)? Above all, how can we be interesting for them? (Our assumption right now seems to be that our audiences want academic, emotionless language. That’s our unspoken hypothesis, but let’s find out if it’s true.).

Content for parents

Now let’s redo it for those young people’s parents (or just anyone who cares about young people and wants the best for them; anyone who prioritizes young people’s mental and sexual health over tradition or archaic ideas of etiquette, in other words).

Motivating attention

Let’s grab their attention. Let’s show them how what they might have thought, might not be the way it actually is. People like to learn new things, especially if they don’t think they get it yet, or if there’s something they might be missing. If it’s about what’s good for their kids, then it’s a must read.

Splitting the audience?

Oh and by the way. Does splitting the audience here matter? No, because the promise is the same for all the audiences. If they’re interested in this – if their interested in society, and if they’re motivated by prosocial causes, then they could get value from the rest of our content too, and that means they’re less likely to become dead weight (followers that never interact again and so drive our algorithmic ranking down).

A/B testing

Here’s how we do it. We take our original, let’s call it A. And we take our new version, let’s call it B. And we show the different versions randomly to different social media users and see which one gets more clicks. So, basically, A/B testing is trial and error, but you get data to show you what’s working or not.

Content AB testing

Testing moves people

Testing is the way to make sure that we’re really moving as many people as we can. Literally anything can be tested to see if it can be made more effective. We can test different audiences, designs (what makes people click more), writing styles, video editing styes, lengths, depths. We can test the intro, test first person or third person perspectives, test colorful storytelling vs dry factual– the list is as long as our list of creative choices when we’re producing content

Today, there’s not one major company or media outlet – from Nike to the New York Times – that’s not using experimentation and A/B testing as a major part of their business strategy. Why just choose a web design you think looks nice when you can choose the design that definitely drives the most clicks or the most engagement? Experimentation is how we get answers.

And let’s be honest, we never look at the numbers and say, “ok, that’s enough.” Because our opponents, the Andrew Tates and the Vladimir Putins of the world never stop either. We’re in a battle of ideas, and experimentation and data analysis are how we win.