Tuesday, December 3, 2013

Streaming JSON Parsing Performance Test: Jackson vs. GSON

A few years ago, when speed and memory became a concern in one of my Android apps, I implemented some streaming JSON parsing using Jackson.  At the time I chose Jackson over Gson because the benchmarks seemed to show it was significantly faster*; however, I've heard that recent versions of Gson rival Jackson for speed.  I want to re-examine the benchmarks again to make sure I'm choosing the right library.

The best benchmark I found is at json-benchmark.  There were a number of other benchmarks out there but they didn't appeal to me for various reasons, either because they didn't provide source, the benchmark didn't seem very fair, or it didn't test on the Dalvik VM.

Unfortunately, json-benchmark's last documented test is from two and a half years ago.  I decided to try to reproduce these tests against the latest libraries, Jackson 2.3 and Gson 2.2.4.  There were a few trials getting the test to run again:
  • The JSON benchmarking code has moved to libcore, so I checked out and referenced the latest benchmarks from there.
  • The ParseBenchmarks code to test Jackson and GSON directly was removed, so I had to re-add it (using the same old code, but with some package names updated).
  • vogar uses old assumptions about where "dx" (and related tools) are located; it still thinks they are in $ANDROID_HOME/platform-tools.  Rather than fix and recompile I just made my files match its assumptions for a bit.
Once I got it running, I ran the tests on a Nexus 7 (2nd generation), running Android 4.4.  Here's the results:

        document            api     ms linear runtime
          TWEETS ANDROID_STREAM  19.82 ===
          TWEETS JACKSON_STREAM  11.47 ==
          TWEETS    GSON_STREAM  17.98 ===
          TWEETS       GSON_DOM  39.00 =======
          TWEETS       ORG_JSON  30.86 =====
          TWEETS       XML_PULL  26.08 ====
          TWEETS        XML_DOM  85.32 ===============
          TWEETS        XML_SAX  33.83 ======
    READER_SHORT ANDROID_STREAM   4.60 =
    READER_SHORT JACKSON_STREAM   2.99 =
    READER_SHORT    GSON_STREAM   4.36 =
    READER_SHORT       GSON_DOM   8.07 =
    READER_SHORT       ORG_JSON   6.42 =
    READER_SHORT       XML_PULL   6.75 =
    READER_SHORT        XML_DOM  14.84 ==
    READER_SHORT        XML_SAX   5.54 =
     READER_LONG ANDROID_STREAM  38.24 =======
     READER_LONG JACKSON_STREAM  18.12 ===
     READER_LONG    GSON_STREAM  43.81 ========
     READER_LONG       GSON_DOM  72.34 =============
     READER_LONG       ORG_JSON  52.47 =========
     READER_LONG       XML_PULL  65.53 ============
     READER_LONG        XML_DOM 160.02 ==============================
     READER_LONG        XML_SAX  47.37 ========

Of course your mileage may vary depending on what exactly you're parsing, but with somewhere between 1.5-2x the speed I think I'll stick Jackson when I need my JSON parsing to be fast.

* Speed isn't everything; if Gson was only marginally slower than Jackson I still would have preferred it; I find their streaming parsing paradigm easier to use.

12 comments:

  1. Nice analysis, I appreciate that you use the Dalvik VM to provide a real-world comparison for the Android use case.

    I'm curious about what you find easier to use about the Gson streaming API. With just a quick look at JavaDocs (no hands on experience) the two seem roughly equivalent, modulo minor style differences, so I'd be interested in hearing what you think Gson does better.

    ReplyDelete
    Replies
    1. It's not a major difference, I agree, but Jackson makes you parse tokens before you read them, whereas Gson has a minor amount of look-ahead. This means that with Gson you can check if there is a next token before officially popping it.

      It's subtle, but I've often made the mistake with Jackson where I read one token too many because I should've been using the last parsed token instead of reading another one (or vice versa). With a bit of look-ahead the API is more clear - read until you find the right key, then parse.

      Were I to guess, I would think the look-ahead explains some of the performance differences.

      Delete
  2. Good Work! Kudos!

    It would very helpful in choosing right JSON parsing library. :)

    ReplyDelete
  3. Hi Daniel,

    I need to do some benchmarking myself, but the code for json-benchmark is missing from the Google repository. If you still have a copy, could you please upload it somewhere and send it my way?

    ReplyDelete
  4. Excellent benchmark. I think of moving to GSON from Jackson for small set data binding since my project usually need small set. In another benchmark it shows better performance for that case (without using streaming API)

    ReplyDelete
  5. I applaud the publication of your article on building remaining json parsing performance. It's a good reminder to look on the DevOps training.

    It is recommended to take DevOps training in Chennai quora

    Thank you for sharing with us the post that you have worked so hard to refine.


    DevOps training in chennai with placement | Best DevOps training in chennai | DevOps training in chennai OMR | DevOps training in chennai Velachery |DevOps training in chennai

    ReplyDelete
  6. Nice & Informative Blog !
    you may encounter various issues in QuickBooks that can create an unwanted interruption in your work. To alter such problems, call us at QuickBooks Customer Service Phone Number 1-(855) 738-7873 and get immediate technical services for QuickBooks in less time.

    ReplyDelete
  7. Hey! Mind-blowing blog. Keep writing such beautiful blogs. In case you are struggling with issues on QuickBooks software, dial QuickBooks Customer Service Number The team, on the other end, will assist you with the best technical services.

    ReplyDelete
  8. Hey! Nice Blog, I have been using QuickBooks for a long time. One day, I encountered QuickBooks Customer Service in my software, then I called QuickBooks Customer Service They resolved my error in the least possible time.

    ReplyDelete
  9. Our team at Quickbooks Phone Number 1833-244-5865 is responsible to make sure that particular issue should be discarded in the quickest time. Here you will receive permanent solutions to QuickBooks issues.

    ReplyDelete
  10. Get immediate assistance for QuickBooks errors on Quickbooks Phone Number1 877-948-5867.

    ReplyDelete
  11. If you have QuickBooks errors or problems that are slowing down your business? QuickBooks Customer Service Number is here to help. We have a team of experienced professionals who will be more than happy to solve any issue that you may have in the shortest time possible.
    quickboks customer service(855) 742-7868.

    ReplyDelete