Before we officially launched the BuzzFeed News newsletter, we made prototypes and shared them publicly across our social networks for a week as part of the #teamnewsapp commitment to transparency in process.
Our prototypes ranged from what would become sub-sections of the newsletter (we called these 'snippets') to the product in full. The most valuable thing about this exercise was that it allowed us to avoid getting too emotionally attached to any one idea early on and to keep tweaking and adjusting the product to be better.
While prototyping in public can be anxiety-inducing — sharing an unfinished product! — it made our newsletter so much better in big and small ways.
Figure out your audience and where best to reach them
First, we identified who we believed our audience would be and figured out how best to reach them.
Since our presumed audience were not necessarily news junkies, we turned to Facebook to ask our friends, especially those not working in media and who probably weren't on Twitter all day, for their opinions.
This is an important part of the "public" part of our prototyping process because we needed to mimic our 'real' readers as much as we could. Because while internal feedback groups are useful and sometimes less scary than going public — in the end, we, the BuzzFeed News team, were not our users.
Start small and test your hypotheses
We started off small — with a "snippet" experiment on the death of Argentine prosecutor Alberto Nisman. A modified version of this approach would eventually form one of the sub-sections in our newsletter, "And a little extra." The important part here was that we had something to share that our audiences could interact with and react to, even if it wasn't a whole or finished product.
Our hypothesis was not revolutionary: it was that people may miss interesting stories because there aren't enough of what I like to call "entry points."
I wanted to find out how people first started following the story, or what held them back from following. My intent here was less to test the actual format of the snippet and more about testing the way we were writing the snippet itself.
The feedback we received helped reaffirm our hypothesis and continues to help how we think about our editorial voice for the newsletter and the forthcoming app.
We integrated this thinking into the newsletter, including "a little extra" on the political unrest in Yemen and President Obama's executive action on immigration.
Even tiny changes add to a better user experience
Feedback from our non-media friends and family also helped us rethink small things that we had missed:
Comments like this one led us to start linking the "And a little extra" section more closely to the top story, and to be consistent in placing photos so they more obviously related to a given story.
Soliciting feedback on the prototype also reminded us of how something as small as a sentence about a map could cause confusion.
This example helped galvanize our thinking about why we include links in the newsletter: as a service to our readers who might want to know more, not because our lack of context is a point of frustration and they need to go elsewhere to understand. We want to help fight #clickfatigue.
Other comments led to us defaulting to being explicit about sources, and being transparent about who authors and reporters were. We experimented a bit and decided that we would always cite the news organization.
Each of these are tiny considerations but together they add up to a smoother experience for our readers. Every time someone has to stop and question something or click something out of frustration, it slows them down — and we want our audiences to enjoy the ride.
Sorting for statistically significant feedback and what not to change
Finally, with such a small sample size with a clear selection bias, we had to remind ourselves about statistically significant feedback vs. one offs and highly idiosyncratic personal preferences.
For example, something we decided not to change was our introductory blurb. Initially, we wrote a sentence with three main points of the newsletter and received feedback that this section would be better as a bulleted list because it was confusing as one long sentence: "reads kind of weird because I don't realize the second point is not part of the Ukraine stuff until the extreme juxtapositioning has hit me."
So, we tried it the next day and then received strong feedback demanding the return of the italics.
It turns out this wasn't a problem with the form, bullets vs. a narrative paragraph — it was an editorial challenge, with how we were writing it.
We started writing these blurbs in complete sentences and kept the italics. This tone and presentation felt friendlier to us (and our audiences, even some of the pro-bullet points camp, liked it).
Identifying the 'You're doing it wrong' feedback from 'It would be better this way' feedback
One of the most important lessons we learned was the difference between 'you're doing it wrong' feedback and 'it would be better this way' feedback.
The first is an assertion that whatever you're doing and however you're doing it, stop. The latter, to me, is about taking the essence of what you are doing and creating a better experience for your audience.
Luckily, since our experiments with our prototype and then launching the "official" product, we've received mostly 'it would be better this way' feedback. That's helped us make those many small changes that, added together, make the experience of reading the BuzzFeed News newsletter a great one.
As a bonus: Our 'Happy day' section started off as just another item and organically became its own section after our internal testers loved it.
If you want even more, sign up to be one of our beta testers for the BuzzFeed News App.
