A digital magazine based in Berlin that focuses on explanatory journalism and collaborations with readers
Krautreporter editor-in-chief Rico Grimm and publisher Leon Fryszer are often called “the survey guys.” That’s because surveys are baked into almost everything the newsroom does, from asking for story ideas, to gathering feedback on a product, to sourcing the crowd’s knowledge and finding experts on certain topics. Everyone in the newsroom is responsible for writing and using surveys, and has been trained in basic synthesis and segmentation to inform their editorial work.
In 2019, Grimm and Fryszer mapped out their entire surveying framework, from what types of surveys they do to what kind of outcomes each achieves. They’ve probed how surveys can be used as a growth tactic, and even identified what surveys can’t do for them. And then they put all of that into a playbook, which MPP discusses here.
Why this is important
Krautreporter has taken the guesswork out of designing surveys.
When done well, surveys provide an abundance of knowledge and resources, including leads on stories, expertise from members, and feedback on products.
But incorporating audience member feedback as extensively as Krautreporter has can quickly overwhelm a newsroom if the process isn’t templated. Krautreporter’s survey practices are notable not just for the quality of the information they provide to the reporters, but how systematically and regularly they are done. As MPP has found, what gets routinized is what becomes culture – and if you want to become member-centric, you need a process for regularly serving them.
From the members’ perspective, filling out a survey is one of the simplest forms of participation by members. It’s valuable on its own, but it might also be the first step on a path to greater participation. You always need people to help you out by taking a survey or sitting for an interview. If someone asks you, “What can I do other than give money?”, the easiest answer is usually “Tell us what you think about this” or “Fill out this survey.”
What they did
At any point in time, Krautreporter might be running 3 to 5 surveys to collect everything from feedback on products to their members’ expertise on a specific topic. With eight reporters on payroll, this means half the newsroom is asking their audiences questions at any given time. They keep it simple, using survey templates they created in Typeform. In 2019, they took a step back and mapped out every type of survey they conduct, identifying why and how of each.
The result was the Engaged Journalism Playbook, supported by the European Journalism Centre, which shares how they do everything from designing their surveys to evaluating the results. Krautreporter’s preferred surveying tool is Typeform mainly because it can be easily completed on mobile and integrates well with their other tools like Airtable. MPP has pulled some of the highlights below.
Vote on topics: Use surveys to ask your audience to vote on the topic they are most interested in. The results of this survey will help guide editorial coverage and the ensuring engagement tactics around the most-popular topic. A Krautreporter “topic vote” survey includes five options for topics that they could cover, and invites audience members to tell Krautreporter which they’re most interested in. When Krautreporter publishes a story on the most popular topic, there’s a built in engagement cycle: We asked, here’s how you responded, and here’s how we delivered.
Example: Reporter Susan Mücke writes a column “A Manual For Everyday Life.” For each piece, she creates two surveys: one where she collects questions that readers want answered, and the second where she lets readers vote on the questions she collected.
Ask about the spin: Sometimes, staff will simply ask readers: what questions do you have about topic X? The answers can help them figure out what angle to take on a broad topic.
Example: Grimm did this when researching Bitcoin. He received several specific questions from audiences, but also comments like, “I don’t even know where to start,” which showed Grimm that audiences felt overwhelmed and confused by cryptocurrency as a topic generally. This feedback showed him he should first write a piece explaining Bitcoin.
Ask about experiences and knowledge: Reporters often struggle to identify people who can humanize a story. Kautreporter asks members if they’ve had any experiences with a topic they are covering.
Example: In response to a post in the Krautreporter Facebook group soliciting story ideas, a member wrote, “I want to understand why people eat meat even if they know animals are suffering.” Theresa Bäuerlein, the editor-in-chief, asked her newsletter subscribers (each Krautreporter journalist has their own newsletter) that question with a Typeform survey. Bäuerlein received about 200 responses, and categorized them, which is how Krautreporter typically synthesizes survey responses. She noticed five answers come up repeatedly, so she focused on those five reasons for eating meat in her article. (Read more about this particular story in Nieman Lab.)
Source the crowd’s knowledge: Your audience might reach out to your newsroom and ask for advice on the best way to do something, such as finding a job or studying for a test. Krautreporter will solicit their members for answers to other members’ questions, and then round up a fact-checked list of the best responses.
Ask what matters. Krautreporter is honest that they don’t always know what the most relevant information is for their readers. Sometimes they survey members to find out if they care about a particular topic.
Example: Before the 2019 European Union election, they surveyed members about which five policy areas they wanted to learn candidates’ stances on. The results gave them a clear roadmap for their election coverage: analyze each party’s position on the five top topics. They disclosed this process to readers.
In addition to answering story-specific questions and informing immediate editorial decisions, surveys also help Krautreporter develop a general sense on their members’ interests and why they read or support the newsroom, which helps them understand them as segments, rather than a monolith.
In an interview with the research team, Grimm and Fryszer described the audience segments like this:
Very engaged: The top 1 percent, the power members who “comment on the article, fill out every survey… we know them by name.”
Somewhat engaged: About 9 percent of their audience; the people who “join a conversation when they have something to say.” Grimm said these readers rarely comment online because “they don’t want their names out there… they have no interest in fighting.” But when they find themselves in a safe space, like a survey, and they know something about the topic, they will engage.
The rest: The remaining 90 percent of their audience; the people who have an attachment to the brand, but are members mostly to get access to the journalism (Krautreporter has a paywall). There’s also a group of members who rarely read and “just want to be around.”
What they learned
Surveys lead to an engagement boost with members. Krautreporter found that in the four weeks after a survey was conducted, members who participated in a survey tended to increase their reading frequency. This outperformed the newsroom’s other engagement efforts.
Surveys are a retention tactic. Krautrerporter found that readers who participate in at least 1 survey stay on as a member for roughly four months longer than a non-survey taker. Other touchpoints show similar, but weaker, patterns.
Surveying can fill in gaps in analytics. As many online publishers know, it’s difficult to develop a holistic sense of your audiences using metrics and analytics alone. Surveys allow the team to fill in their understanding of their different audience segments, while also allowing the reporters to test assumptions about what those segments of readers want.
Key takeaways and cautionary notes
Surveys can be incorporated across all stages of an editorial process. At Krautreporter, reporters are asked about their plans to conduct surveys with members before they even start working on a piece.
Surveys are great for engaging shier audience members. Most audience members don’t want to engage in comments or in public forums, but would welcome opportunities to be a part of the process less prominently. Remember to design for the less vocal audience segments, too.
Assume you’re not reaching non-engaged members with surveys. It’s hard to get surveys in front of people who aren’t already engaged, at least through your own channels. Krautreporter acknowledges that this is a major information gap. To do that, you’ll need to get creative about your distribution, perhaps by asking another organization to share the survey or posting it to other public forums, such as a neighborhood group.
- Krautreporter, report: Engaged Journalism Playbook.
- Krautreporter, article: Why do you eat meat?
- European Journalism Centre, event recap: Five insights that will strengthen your engaged journalism
- European Journalism Centre, case study: How Krautreporter uses data to commission articles that grow its membership
Disclosure: Membership Puzzle Project supported a separate Krautreporter project in 2019 through the Membership in News Fund.