Do you have all the data and none of the visuals? Do you just want a pretty, fast way to compare lots of data that centers around maybe just a handful of moments?

D3.js can help you tame all of your data and d3.rollup is especially useful if you have lots of data that you need to combine into just a couple of data points. All it takes is a couple of (pretty long) lines of code and you will have an awesome visual that’s very customizable.

Lets start with a really straightforward example of a rollup. In all of these examples, I’m using code straight from my HeatVote project, which requires me to pull voting data from our server API that I receive as a JSON blob. Here’s an example entry:

1
2
3
4
5
6

{ video_id: 'T-D1KVIuvjA',
  timestamp: 2,
  vote: 1,
  id: 1,
  createdAt: Sat Dec 21 2013 14:55:42 GMT-0800 (PST),
  updatedAt: Sat Dec 21 2013 14:55:42 GMT-0800 (PST) }

Now obviously there are a bunch of these, and technically there are easier ways to do this, but to show off the structure of a rollup, lets count how many entries we had in our database using a d3 rollup!

1
2
3
4
5

var total = d3.nest()
  .rollup(function(d){
    return d.length;
  })
  .entries(data);

1
2
3
4
5
6
7
8
9
10
11

var averages = d3.nest()
  .key(function(d) {
    return d.timestamp;
  })
  .sortKeys(d3.ascending)
  .rollup(function(d){
    return d3.mean(d, function(g) {
      return +g.vote;
    });
   })
   .entries(data);

Now there is a lot going on in this very compact few lines, so well go through them one by one, but the result is that averages is equal to an array of objects with the properties key (that is equal to each unique timestamp) and value (that is equal to the mean of all votes at that timestamp).

So lets break it down:

• .key(...) is just used to tell the function what our keys are, only grabbing unique values of that property.
• .sortKeys is just a prettiness thing, it sorts my keys into an order (when they’re pulled off the server the only order is by the time they were created on the database).
• and finally our lovely .rollup(...). Now instead of d being an array of the whole data, it’s now an array of only the data for each individual key (so all of the data with the same timestamp). The inner function d3.mean takes a specific property from all of the data for each key and averages them up.
  And that, is d3 rollup in a nutshell, it’s really lovely at coercing relationships out of your raw data and you can obviously do a lot more with it that just averaging things. The d3 nest docs are probably the next best place to look to get your hands dirty (.rollup is a property of nest).

The tiny bit of Hack Reactor/git humor above is going to be lost on most of my audience but it made me happy. This week felt a bit like the first week of the program. A bit of uncertainty, a lot of excitement.

We were turned loose this week. No more structured lessons, it’s the start of the projects phase. The first two days were devoted to a “hackathon” where I learned angular and firebase and created a bookmarking app that included full-text search and a companion Chrome extension. That was super exciting. It was awesome to build something with my own two hands and realize all that I’ve learned in the past six weeks. Even if something didn’t work right off the bat, I now have the confidence to read through the documentation and understand what individual pieces weren’t functioning the way I expected them to. One of my best strengths has definitely been my super-human ability to craft a google search query.

The rest of the week has been a move into group projects and learning a whole new skill-set. I’m a get-along kind of person by nature and it was hard at first for me to feel like I wasn’t stepping on toes when I wanted to work on a specific part of our project. After a day or so of planning and modeling what we wanted to do though, it became easier to split up tasks. I learned Asana (a project tracking app) and got really good at dividing up tasks.

My favorite part of our new app is that I got to deep dive into d3 (a JavaScript library for data visualizations) functionality and create a homegrown heat-map. Some of my notes are up on GitHub, but I plan on refactoring and creating a whole post on the heat-map process (spoiler: d3.rollup is my savior).

I am very content this week. I’m getting a taste of what the real world is going to look like for me as a software engineer (an unfortunate side effect is that I’m getting a real taste for coffee to keep up with the long Hack Reactor hours). Now though, I’m sitting in SFO waiting for my plane to PDX and home for Christmas. I’ll miss the beautiful (and non-rainy) sites of San Francisco, but I’m excited to visit family and show them how much I’ve changed.