What It's Like To Work On BuzzFeed's Tech Team During Record Traffic

    It's all about the company culture, folks. Here's how it all went down (or didn't) last night.

    Dao Nguyen
    by Dao Nguyen

    BuzzFeed Publisher

    8:23 p.m.: Jay's sysops pager goes off. His wife says, "Uh oh, is it the dress?" He replies, "What dress?"

    Yes, those are automated bot messages in a dedicated Slack channel.

    8:30 p.m.: I put the kids into their pajamas, brush their teeth, and hand them to my husband who puts them to bed. He says, "BuzzFeed keeps tweeting about a dress and I don't understand."

    At first the only people in dev-chat Slack room are the Los Angeles team. I check stats and see that this is going to be an amazing night. I do an @channel. Jay, of course, is already ON IT.

    9:02 p.m.: Active visitors on the site reach 450,000, a new record. Jake, data science, delivers a screenshot of the previous record from February 2014.

    This was Feb. 13, 2014, the night of the states quiz mania.

    PR is already pitching morning shows. This is also happening.

    Chris Tindal @christindal

    Actual quote from a BuzzFeed server admin: "I AM THROWING SERVERS INTO THE INTERNET"

    Reply Retweet Favorite

    9:09 p.m.: When we hit 500,000 concurrent active visitors, I invite Samir, social media, to dev-chat to join in on the fun.

    Eugene asks him for his over-under on active visitors. Samir knows that we still have a good 90 minutes of peak internet usage and says 600,000, then revises to 650,000. He was SPOT ON — traffic peaked at 673,000.

    Samir explains how it happened.

    "People were discussing it in the office, so I figured I would tweet it, and saw a HUGE click-through rate. I posted it as a photo on Facebook because I wasn't sure if it was something where you had to see the whole photo or not, but no one was clicking through at all. So I reposted as a link, watched it for a few minutes, not much engagement, deleted it. Twitter was doing so well, I decided to just post it again and saw it immediately rise in the live stats and get tons of comments on Facebook."

    View this post on Facebook
    Facebook: BuzzFeed

    "People were discussing it in the office, so I figured I would tweet it, and saw a HUGE click-through rate. I posted it as a photo on Facebook because I wasn't sure if it was something where you had to see the whole photo or not, but no one was clicking through at all. So I reposted as a link, watched it for a few minutes, not much engagement, deleted it. Twitter was doing so well, I decided to just post it again and saw it immediately rise in the live stats and get tons of comments on Facebook."

    There's a lot of debate and joking about the actual color of the dress in the chat. Shaun, illustrator, uploads this gem.

    Shaun Pendergast / BuzzFeed

    A nod to our new Cute or Not app that was just featured in the App Store.

    There are dozens of people in dev-chat at this point because it's fun, not because they have to be.

    9:57 p.m.: Peak traffic. There are more than 670,000 active visitors on the site. The GIF below shows what off-peak and peak traffic looks like to a BuzzFeed server. WAIT FOR IT.

    Dan Meruelo / BuzzFeed

    Holy hell.

    10:30 p.m.: A couple of posts get stuck while publishing. Ben Smith, editor-in-chief, chats me and Amy, project management.

    Jay, Dan, Raymond, and Eugene (sysops), and Mark, chief technology officer, are spinning up servers as fast as possible.

    Remo Williams: The Adventure Begins / Orion Pictures

    We have a bunch of things going for us at this point. We have heavily invested in infrastructure provisioning and scaling. We know exactly how to scale fast from running drills. We have seasoned sys admins who have gone through this before. Only two people get to bother them for updates. Jay's pugs sat on his lap the whole time. And most importantly, we are calm and cool under pressure.

    Predator / 20th Century Fox
    The Supernatural / CW

    11:01 p.m.: The articles are published! By the end, we added over 40% more capacity, and readers never had any problems reading the posts or answering the poll.

    Adventure Time / Cartoon Network

    The good part about last night: BuzzFeed experienced a singular moment in global cultural relevance.

    Dan Meth / BuzzFeed

    All eight dress posts have a combined 41 million views. The original had 28 million views (22 million on mobile) and logged 2.7 million votes. It has been read from every country in the world in five languages. It shaped conversations at dinner, in bars, on couches, over text, all driven by mobile and the ability to show your phone to your friend. Same picture, same device, but different colors! In less than 24 hours, people from every corner of the world were looking at each other's phones at a post, on a site, run by a company totally optimized for social and mobile.

    The GREAT part about last night: working at an awesome company that appreciates tech.

    Ben Smith @BuzzFeedBen

    @catesish I think you may be buying sysops drinks

    Reply Retweet Favorite
    Christina DiRusso @CDiRusso

    @BuzzFeedBen @catesish i think we all are. bows down to @eventi @rayzorinc @jaydestro

    Reply Retweet Favorite

    ❤️❤️❤️❤️❤️❤️❤️❤️❤️❤️