Tuesday, December 3, 2013

Streaming JSON Parsing Performance Test: Jackson vs. GSON

A few years ago, when speed and memory became a concern in one of my Android apps, I implemented some streaming JSON parsing using Jackson.  At the time I chose Jackson over Gson because the benchmarks seemed to show it was significantly faster*; however, I've heard that recent versions of Gson rival Jackson for speed.  I want to re-examine the benchmarks again to make sure I'm choosing the right library.

The best benchmark I found is at json-benchmark.  There were a number of other benchmarks out there but they didn't appeal to me for various reasons, either because they didn't provide source, the benchmark didn't seem very fair, or it didn't test on the Dalvik VM.

Unfortunately, json-benchmark's last documented test is from two and a half years ago.  I decided to try to reproduce these tests against the latest libraries, Jackson 2.3 and Gson 2.2.4.  There were a few trials getting the test to run again:
  • The JSON benchmarking code has moved to libcore, so I checked out and referenced the latest benchmarks from there.
  • The ParseBenchmarks code to test Jackson and GSON directly was removed, so I had to re-add it (using the same old code, but with some package names updated).
  • vogar uses old assumptions about where "dx" (and related tools) are located; it still thinks they are in $ANDROID_HOME/platform-tools.  Rather than fix and recompile I just made my files match its assumptions for a bit.
Once I got it running, I ran the tests on a Nexus 7 (2nd generation), running Android 4.4.  Here's the results:

        document            api     ms linear runtime
          TWEETS ANDROID_STREAM  19.82 ===
          TWEETS JACKSON_STREAM  11.47 ==
          TWEETS    GSON_STREAM  17.98 ===
          TWEETS       GSON_DOM  39.00 =======
          TWEETS       ORG_JSON  30.86 =====
          TWEETS       XML_PULL  26.08 ====
          TWEETS        XML_DOM  85.32 ===============
          TWEETS        XML_SAX  33.83 ======
    READER_SHORT ANDROID_STREAM   4.60 =
    READER_SHORT JACKSON_STREAM   2.99 =
    READER_SHORT    GSON_STREAM   4.36 =
    READER_SHORT       GSON_DOM   8.07 =
    READER_SHORT       ORG_JSON   6.42 =
    READER_SHORT       XML_PULL   6.75 =
    READER_SHORT        XML_DOM  14.84 ==
    READER_SHORT        XML_SAX   5.54 =
     READER_LONG ANDROID_STREAM  38.24 =======
     READER_LONG JACKSON_STREAM  18.12 ===
     READER_LONG    GSON_STREAM  43.81 ========
     READER_LONG       GSON_DOM  72.34 =============
     READER_LONG       ORG_JSON  52.47 =========
     READER_LONG       XML_PULL  65.53 ============
     READER_LONG        XML_DOM 160.02 ==============================
     READER_LONG        XML_SAX  47.37 ========

Of course your mileage may vary depending on what exactly you're parsing, but with somewhere between 1.5-2x the speed I think I'll stick Jackson when I need my JSON parsing to be fast.

* Speed isn't everything; if Gson was only marginally slower than Jackson I still would have preferred it; I find their streaming parsing paradigm easier to use.

3 comments:

  1. Nice analysis, I appreciate that you use the Dalvik VM to provide a real-world comparison for the Android use case.

    I'm curious about what you find easier to use about the Gson streaming API. With just a quick look at JavaDocs (no hands on experience) the two seem roughly equivalent, modulo minor style differences, so I'd be interested in hearing what you think Gson does better.

    ReplyDelete
    Replies
    1. It's not a major difference, I agree, but Jackson makes you parse tokens before you read them, whereas Gson has a minor amount of look-ahead. This means that with Gson you can check if there is a next token before officially popping it.

      It's subtle, but I've often made the mistake with Jackson where I read one token too many because I should've been using the last parsed token instead of reading another one (or vice versa). With a bit of look-ahead the API is more clear - read until you find the right key, then parse.

      Were I to guess, I would think the look-ahead explains some of the performance differences.

      Delete
  2. Good Work! Kudos!

    It would very helpful in choosing right JSON parsing library. :)

    ReplyDelete