At Buzzfeed we love our graphs. Since our posts get pretty popular we have some big ones to deal with. Even so; we want to know a lot of statistics about them right up to the second.

Our language of choice is Python. Most of the time when we work with graphs we use nice packages like igraph or networkx. These allow us to do complex ad-hoc reports and analysis. For real-time applications and APIs, however, loading the graph into memory on-demand is totally prohibitive, the data is too big.

We could use a specialized database, but it's still slower to compute an arbitrarily complex query than to do primary key lookup. In this post we'll start by scaling those in-memory computations with a low-level Python implementation of my favorite interview question. We'll end with computations that are fully online.

Given a simple tree (like the one below) and a root node (e.g. 0), write a function that lists all the descendants in the tree.

Giphy / Via giphy.com

Rooted at node 0, this bad boy is directed, acyclic, and each node has exactly 1 parent. Just like a good tree.

Andrew Kelleher

One approach to representing graphs on your pew-pewder is using the adjacency matrix. This structure totally rocks because scipy, for example, has mad sparse matrix representations.

If you don't need to know anything in particular about your nodes or edges (just that they exist) this structure is probably ideal as far as in-memory computations go.

Each row represents a parent, and each column represents a child. A one entry at i, j indicates there is an edge from node i to node j.

Since our graph is directed, you'll notice the matrix is not symmetric. Undirected graphs are reflected about the main diagonal. Although you represent more data; using a format like SKS allows you to use roughly the same amount of memory as the upper-triangular version. We're kind of getting off track.

Let's have a look at a first approach to the age old question.

Andrew Kelleher

]]>

Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.asdf

Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.Lorem ipsum dolor sit. Lorem ipsum dolor sit.... The quick brown fox jumped over the lazy dog.

]]>