Newsroom overview

The Narwhal has taken a test-and-learn approach to optimizing its digital platforms for membership. 

In recent months, the site has run experiments to grow its audience, develop more loyal users, and make it easier for readers to become donors. 

One of its most successful tests has been to optimize the “donate” button on its website with language that better reflects its values as a news organization and has helped lead to a bump in membership. Its focus on the donation button has been a significant win, but its continuing to expand its testing repertoire to include a focus on search engine optimization, email newsletters, and more. 

Why this is important

The number of things a digital news site has to know how to do to reach sustainability can be dizzying. Many of them, such as conversion optimization, are far outside the skills most journalists have. These big decisions can be de-risked by taking a “test-and-learn” mindset to them.

The Narwhal is constantly running tests. 

“It comes down to always thinking about how we can improve what we’re doing,” said Arik Ligeti, The Narwhal’s audience engagement editor. “Never getting too comfortable with how things are operating. Even if things seem to be going great, how can we take them up a notch? It can be easy to get bogged down in all of your daily tasks. It’s about making experimentation part of your routine as much as possible.”

The Narwhal is a small team of 10 people. This case study offers guidance on how similarly small teams can optimize their operations to enhance their membership programs.

What they did

Until earlier this year, The Narwhal’s main donate button on its website enabled only one-time donations. 

As part of its participation in the Facebook Journalism Project’s Membership Accelerator, The Narwhal changed how it approached reader support. (The Lenfest Institute partners with Facebook to administer its accelerators and helps share learnings from them.) 

The Narwhal made a conscious decision to change the default to monthly recurring donations. It made the change because members who give monthly tend to have higher retention rates and bring in more revenue over time than those who just give one time. (See the retention section for more on monthly giving.)  

As a result, it changed the language on its website from “Donate” to “Become a Member.” That change resulted in 26 new members within the first two weeks of making the switch, which was an 120% increase in revenue per site user compared to a two-week stretch with a similar amount of traffic. 

But it wanted to see if it could improve on the results. It decided to run an A/B test on its website using Google Optimize. Most testing decisions, including this one, begin as a conversation between Ligeti, Emma Gilchrist, the site’s editor-in-chief, and developer Chris Desjardins. “There’s not a lot of bureaucracy,” Ligeti said. 

Half of the site’s visitors saw the standard “Become a Member” language and the other half saw “Become a Narwhal.” 

Over the 18 days it ran the test, it was clear that “Become a Narwhal” was outperforming the previous option. “Become a Narwhal” had a 0.29% clickthrough rate compared to a 0.17% clickthrough rate for the original “Become a Member” 

“I’d boil that down to a curiosity factor, probably. Oh, what is ‘Become a Narwhal’? Then once we were pretty confident with the results we made the permanent flip.” 

The donation button test is just one part of a larger effort for how The Narwhal is thinking about how it can optimize every level of the funnel — from attracting new readers to converting regular readers into sustaining members. 

It also ran experiments in its email newsletters by including donor solicitations higher up in the newsletter. 

Using funds from the Accelerator, it has hired a freelance breaking news writer to boost the amount of coverage it’s able to produce, which it hopes will grow the top-of-the-funnel audience, which it can then convert into paying members. 

“We know that there’s a much bigger audience for environmental news in Canada than we’re currently reaching,” he said. “Even though we’re growing exponentially compared to where we were a year or two ago, we’re thinking on that level about how we can bring people into the newsletter, how to reach more people on search…and thinking about how if someone is just searching for environmental news that they’re finding us.”

The results

Over the course of nearly 400,000 unique sessions, the “Become a Narwhal” button had a 0.22 percent click through rate compared to a 0.14 percent click through rate. 

126 new members came from clicking on “Become a Narwhal” 

Since March, when they made the button switch, the site has added 519 new members, bringing its total to 1,500 at the end of June, exceeding its annual goals. The Narwhal’s new goal is to reach 2,000 members by the end of 2020. And it plans to continue to optimize its operations by overhauling its donation page by implementing a streamlined checkout process and adding in Apple and Google Pay. 

The Narwhal says it expects $100,000 in new customer lifetime value from the new members it has added since the spring when it joined the Accelerator program. Check out the retention section for how you can calculate CLV. 

What they learned

Capacity is key. Running a news site is hard, and trying to handle product-thinking decisions and run tests on top of them is even harder. Ligeti joined the Narwhal in March, and he’s been given the responsibility of thinking more strategically about tests to run and then execute beyond just getting the daily products out the door. (It’s still a relatively small team though and he still pitches in on things such as running social media accounts or editing stories.)

“They were throwing together the newsletter, but I get the chance to spend more time with it and think about things like: What if we tested sending it on Thursday versus Sunday or testing where a donate button might appear,” he said. “That’s not complicated, it just takes a bit of time. If you’re juggling a million things, that’s not going to be at the top of your agenda.”

Give your team the autonomy to test. The Narwhal was able to get the test up and running pretty quickly because there wasn’t a huge amount of bureaucracy. Ligeti said he had some discussions amongst the site’s editors, but other than that they just decided to go with it and see what would happen. “It’s a pretty open culture,” he said. 

The Narwhal also has a freelance developer on retainer, and he was a regular part of the team’s accelerator sessions, so he was able to quickly spin up tests and new features to get them in front of audiences to get immediate feedback — including with the “Become a Narwhal” button. By empowering the team to try out different things, The Narwhal has established a culture that encourages learning. 

Key takeaways and cautionary notes

No test is too small: Small tests can make a big difference. The Narwhal learned that from its test around the donation button. Even though it may seem trivial, minor upgrades to your website performance through elements such as the donate button can make significant differences in your conversion rates, which are critical to the success of a sustainable membership program. 

You have to be OK with a bit of risk. There’s always the chance that a test could lead to adverse results. Adopting a test-and-learn mindset requires suppressing the impulse to stop a test the minute results start to trend downward. The data you’ll gather from completing the test, even if it leads to a dip in member conversions, will give you important insights that you can use to recover lost ground – and then some.

You don’t need flawless results. With the donation button test, The Narwhal is fairly confident with its results, but there may be pieces missing. Due to the fact that it’s using an outside payment processor, it’s not sure if everyone who clicked went on to become a member. “That also highlights that even though we had the insight, and we were confident enough to make the switch, having the higher click rate alone…if you’re having more people alone visit, we were confident that was still helping convert more people.” 

By not focusing on perfection, The Narwhal was still able to get enough information to make a decision to make the switch on its donate button. 

Other resources

• Google Optimize