Buzz Bot is offline now, but throughout the Republican and Democratic conventions this summer, our curation team of Sara Yasin and Brandon Wall worked side by side (virtually, in any event) with Wes and Amanda from the Open Lab to tease out conversations that would resonate with Buzz Bot users. We’ve already told Poynter and Nieman Lab a bit about what we learned -- that experienced reporters are much more efficient at confirming or debunking news reports, that building a network of sources at an event that is already swimming with journalists is especially hard, that our users want a bot that gives at least as much as it asks for, and, most importantly, that it helps to be really flexible when you’re trying something new.
Buzz Bot welcomed users with a series of questions that helped us sort them into some basic groups. That meant that we were able to direct questions to people who told us they were protesters in Cleveland, or delegates in Philadelphia who had supported Sanders in the primaries.
These basic intro questions let us sort users a bit and direct our messaging. BuzzFeed News hosted a live taping of No One Knows Anything in Philadelphia -- Buzz Bot was able to frame questions differently for different groups, and send out very different questions to folks who were in the audience that night and presumably had context that users at home wouldn't have. Of course, Buzz Bot didn't know who was in the audience, the bot knew who told us they were in the audience.
The questions also let us bring groups together -- we asked locals for advice on how to get a transit pass or where to watch the night's speeches, and shared that advice with protesters, most of whom had come from out of town for the convention and weren't going to be able to watch the speeches live.
We really wanted to experiment with using Buzz Bot to verify reports about events on the ground, but overall, our own reporters were much more efficient than Buzz Bot users at cutting to the chase. It helped that BuzzFeed News had a team of ~20 reporters in Cleveland and Philadelphia and a dozen more watching the live streams and doing background work checking facts and pushing out news updates. In a situation where we didn't have as many folks on the ground, the bot could have been a real asset.
As we tested the bot, the opportunity to compare what our users reported
with what our reporters found was very useful, but it meant we could
rarely credit Buzz Bot with a scoop.
Some of the kinds of stories we tried to verify: Twitter rumors that protesters were being arrested at a particular intersection (there weren't any Buzz Bot users nearby, but a BuzzFeed Politics reporter got there very quickly); reports of protesters burning flags outside the convention center (again, our reporters had the story much faster than the bot). On Day 2 of the GOP Convention we heard someone shout what sounded like "no Islam!" from the audience as a Muslim speaker led a prayer. By the time that Buzz Bot's users in the convention hall had told us they didn't hear the shout, BuzzFeed politics reporters had found a clip where the exclamation is audible, cut it and posted it on our live post for the day.
When organizers told BuzzFeed News that a particular protest march included a number of Sanders delegates, Buzz Bot was quickly able to put reporters in touch with one of those delegates. Lessons Learned:
- Our reporters are really good at what they do, and most of the the time they can do it faster than bot users.
- If you don't have reporters on the ground, a tool like Buzz Bot definitely can help verify (or debunk) reports.
Everyone's a Reporter
We already knew that Buzz But wasn't going to be a substitute for working reporters doing their job, but we definitely wanted to experiment with asking bot users to help gather photos or even conduct interviews for us.
We tried a lot of different approaches to tease photos and video interviews out of bot users, none of which turned up much. A few users were great about sending us photos (thank you!) but none of their photos made it into news posts.
We didn't manage to get a single video interview out of Buzz Bot. To be fair, most of the newsroom thought that we were nuts to ask for those interviews, but we wanted to at least try. Lesson Learned:
- We need to keep working if we want Buzz Bot users to contribute real reporting.
You Do Know How To Talk To Bots
We'd been warned that people don't know how to talk to bots, but we found that Buzz Bot users had a lot of perfectly reasonable ideas about how to talk to bots.
It isn't crazy to think that a reasonably smart convention bot would know things like who is speaking, or who is scheduled to speak, or if neither of those works, would at least respond to a cry for help.
We could easily have set up "speaking ... now" to send the name of the current speaker, especially for a short term event like the conventions where we absolutely knew the answer to that question. And we definitely should have had ~something~ to say when users asked for "help" -- which quite a few did.
And, as a backup, we should have baked in an easier way to add new trigger phrases and responses. Lessons learned:
- Your bot should have some kind of "help" response baked in.
- Let your users guide you towards other useful trigger phrases, and plan to listen for them.
But That's Not What I Do!
We built Buzz Bot to gather information. We wanted to see if we could build a street team, or a few street teams, that could help us tell the story of the RNC and DNC through their own eyes, whether they were there to protest or as delegates.
We didn't build Buzz Bot to dispense news and information, so we didn't build in responses to questions about things like the speaker schedule.
The only trigger phrase we’d actually baked in was “Surprise me!” which would pull out some fun bit of convention or politics coverage. It was clear from our logs that users really wanted Buzz Bot to keep them up to date. So every morning Buzz Bot told users where to find the day's BuzzFeed News live post, where our reporters were rounding up convention news. We offered to send out occasional news updates during the day, and did send breaking news links to everyone who asked for them.
And when the Democratic convention kicked off, we realized Buzz Bot could also push out a minimal version of the days schedule every morning. More users thanked us for this handy guide than any other feature. Lesson Learned:
- Check your logs -- they can tell you a lot about what your users expected from you.
- Even an information gathering bot should distribute some information.
- Make sure you have something to offer the folks who aren't your primary audience.
- You don't have to know exactly what that is going to be before you launch.
We tinkered with some emoji polls, which were novel until they weren't novel anymore, which is about when we asked users to tell us what they thought of Michelle Obama's speech, using nothing but emoji. We got some great hieroglyphic essays, so we tried again and got another round of great stories entirely in emojis.
- Everyone s to ✍ with emojis.
- Every audience is different. Know yours and keep trying.
Get Your Own Buzz Bot
Ultimately, what you can do with a messenger bot depends on your audience and the moment. But Buzz Bot is free software so give it a spin and let us know how you use it. 🤖