Minimum Delightful To Maximum Impact: What Testing Via Email Taught Us About News Apps

In an earlier post, I wrote about what we learned from prototyping the BuzzFeed News newsletter in public. This post shares more of what we’ve learned and how we’ve applied those lessons to the forthcoming app.

From the beginning, #teamnewsapp responsibilities would include not only a news app, but also a daily morning email newsletter.

But why would a team with “app” in its nickname be responsible for an email product? There were at least two reasons.

  • First, the app and the newsletter have similar editorial goals: provide a quick and easy way for people to catch up with what’s going on around the world.
  • Second, we could use one to learn from while building the other (and second and half: email is fundamentally a mobile product.)

One of my responsibilities on #teamnewsapp is to be the bridge between these two products, bringing lessons from our newsletter to our app and back again.

Launch quickly, test, learn, repeat

Designing and building an email newsletter takes a lot of work. But compared with launching a native app, a newsletter requires a much smaller investment of time and technology. As a lower-stakes product, we could launch it much more quickly than we could an app and start testing with a real audience immediately.

How our newsletter has evolved: before launch, the day of launch, and on May 1 and June 1

Throughout the second week after launching the newsletter, we asked our readers to write to us — and they did. Their feedback helped us continue to answer some of those initial questions we asked during the prototype phase. Our hypothesis was that people don’t have enough time to keep up with the news and may miss interesting stories because there aren’t enough of what I like to call “entry points.”

Here’s some feedback on the first part of that hypothesis — that our newsletter is saving people time:

  • I’ve always tried to keep up with the news but your newsletter makes things so much easier. I love that the stories are straight to the point, simple, and short and I don’t have to read through three pages of an article to figure out what in the world is going on today. —Amanda Schulz, Notre Dame of Maryland University, Baltimore, MD
  • Love the quick and easy to read format of the email. Especially love the sub headings like the “more on this” and “what’s next” … As a college student this email is a perfect way to stay up to date on current events. —Grad Student, Worcester Polytechnic Institute, Worcester, MA

And here’s some feedback on the second part — that our newsletter provides a little extra context to big news stories:

  • My thoughts on the Buzzfeed news letter. I like it more then others I have read because you give details about the story in the email. Most others I have tried give very little detail so then I have to go read the full article if I want to know more. — Jessica Ann Goldade, United States
  • Your newsletter gives me all the important news in 5 minutes and I can dive deeper if I need/want to. So thank you so much, and keep up the great work! — Cyan Ta’eed, Melbourne, Australia

The connection between email and a news app

Email is inherently a mobile product. I wrote earlier in a post for Nieman Lab that email newsletters have a built-in (and widely available, since most people have an email address) distribution system, are tied to an individual reader, and should look great on mobile.

  • Brief, informative, easy to read on mobile devices and concrete. —Adriano Varona, Lima, Peru
  • I love buzzfeed news I’m 19 years old and it’s the only news I keep up with and I’m glad it can come to my email now. — Aaron Wilson, United States

So, how do we transfer what we were learning from the newsletter to the app? This relates to what you might have been hearing about how Quartz is thinking of itself as an API and BuzzFeed’s “distributed” strategy when making content for specific platforms.

“BuzzFeed wants to create content of all kinds that exists within specific networks and platforms. In many cases, that content doesn’t even contain a link to the company’s website,” Mathew Ingram wrote in Fortune.

This is where I’d note that “adapt” is a better approach than “transfer” because we’re not copying and pasting the newsletter into the app. The email experience is different from an app experience. It’s not platform agnostic, but rather, “platform orthodox” as a friend of mine, Tom Rosenstiel, said. So the real question was how we could adapt what we learned and align it with how users were interacting with us on each platform.

I’ll highlight just two of the ways we’ve improved the app by applying lessons from our newsletter. The first is our “Quickly Catch Up” section and the second is our short snippets of background info.

Use what you have, even if it’s not much, to test new ideas

The idea for “Quickly Catch Up” was a spur of the moment experiment. But it was derived from the idea that we wanted someone to know exactly what were the top stories of the day as soon as they open the app.

A Slack conversation between me and Laura from April 7, 2015

It began with us hacking a few bullet points, and eventually became a feature of the app. Here’s how the design and product evolved (thank you, Gordon and Dave):

In the same way that we were prototyping small parts of the newsletter in public, we tested small parts of the app with what tools we had and at the time, which were literally just bullet points.

Feedback from our astute beta testers on April 10, 13, and 14, via our private Facebook beta tester group

Adapt features to different platforms

Another experiment involved providing “a little extra”: additional background info or another angle on a story:

The BuzzFeed News newsletter from April 15 and a comment from one of our beta testers on how he liked the background info that we adapted from the newsletter to the app

In that example, we adapted our “Quickly Catch Up” blurb about what the U.S. State Sponsors of Terrorism list into bullet points, further shortened the sentences, and highlighted the most pertinent facts. Then, we kept experimenting with providing background on stories about Bill Cosby’s sexual assault accusations and the European migrant crisis.

Feedback from our beta testers, via our private Facebook beta tester group

How we’re continuing to adapt editorial lessons to different platforms like push notifications

We’ve also taken seriously the goal of providing context. That means two things: giving you a little bit extra in a very small space and deciding what’s worth interrupting your day for in the first place.

Feedback from our beta testers, via our private Facebook beta tester group and Twitter

You can read about our experiments with contextual push notifications and about how we’re distinguishing “news about the news” from actual news.

But the real lesson here is to give your app to a couple of 10-year-olds and ask them what they think. They’re honest.

10-year-olds looked at #teamnewsapp's beta today and gave us feedback. ❤️: "The breaking news is very breaking."

— Aaron Edwards (@aaronmedwards)

You can sign up for the BuzzFeed News newsletter here! Just enter your email address below.



View this embed ›

Check out more articles on BuzzFeed.com!

Connect With Us

We're hiring

We're looking for great editors and writers around the world.

Join the team

More BuzzFeed Blogs