Keeping JSON Endpoints in Check with json-weight

Nate Hunzaker, Former Development Director

Article Category: #Code

Posted on July 11, 2016

j s o n - w e i g h t h e l p s i d e n t i f y w h a t i s m a k i n g a J S O N e n d p o i n t t o o h e a v y .

Performance audits are an integral part of our development process. Good data is essential when doing this kind of work. It's too easy to neglect true bottlenecks; to chase after the wrong problems.

I've run into a couple of situations where JSON API endpoints were larger than expected. Without a solid understanding of the shape of a response, I've found it can be difficult to determine where to focus attention.

To help solve this problem, I wrote a script called json-weight.

Introducing json-weight #

json-weight visualizes the weight of JSON endpoints. It reports the size of each branch in a JSON response as it steps through the tree of key/value pairs. This generates a helpful analysis of the composition of the response.

Check it out on Github. More information on usage follows.

Installing json-weight #

json-weight is published on npm. Assuming you have NodeJS (really any version) installed, picking it up is simple:

$ npm install -g json-weight

Using it #

I've added a fairly large JSON response as a static file on the Github project for the script. It clocks in at around 430kb. Pretty hefty.

Let's figure out why:

$ curl -s http://code.viget.com/json-weight/people.json | json-weight 2

0: 69.14kb (30.95kb gzip)
  id: 0.03kb (0.06kb gzip)
  name: 0.02kb (0.06kb gzip)
  bio: 1.76kb (1.25kb gzip)
  friends: 67.31kb (30.03kb gzip)
1: 67.54kb (30.55kb gzip)
  id: 0.03kb (0.05kb gzip)
  name: 0.02kb (0.05kb gzip)
  bio: 1.38kb (1.03kb gzip)
  friends: 66.09kb (29.85kb gzip)

#... truncated for brevity

Here I'm piping the API response to json-weight and telling it to report 2 branches deep. From the report we can already identify that the friends field contributes the most weight.

But why? Let's dig deeper.

$ curl -s http://code.viget.com/json-weight/people.json | json-weight 4

0: 69.14kb (30.95kb gzip)
  id: 0.03kb (0.06kb gzip)
  name: 0.02kb (0.06kb gzip)
  bio: 1.76kb (1.25kb gzip)
  friends: 67.31kb (30.03kb gzip)
    0: 2.02kb (1.40kb gzip)
      id: 0.00kb (0.03kb gzip)
      name: 0.02kb (0.05kb gzip)
      bio: 1.99kb (1.36kb gzip)

    # ...

    39: 1.53kb (1.16kb gzip)
      id: 0.00kb (0.03kb gzip)
      name: 0.01kb (0.05kb gzip)
      bio: 1.49kb (1.10kb gzip)

Cool. We can see that quite a few friends are being pulled in for each person. 40 in this instance. There are a couple of natural follow-up questions:

How are we using the friends field?
Do we really need to include the full bio?
Should we consider paginating the friends field?

These are good follow-up steps; we've moved past "why is the page is slow". We can talk productively about the ramifications of changing this endpoint, and what kind of benefits it might yield.