Jackson 2.15: yet faster floating-point reads

One of new features included in Jackson 2.14 was StreamFeature.USE_FAST_DOUBLE_PARSER enabling of which makes parser use an alternative optimized decoding library (FastDoubleParser) to decode floating-point numbers from JSON (float, double, BigDecimal) — and other textual data formats (specifically CSV via jackson-dataformat-csv
).
Upcoming Jackson 2.15 (of which I will write a bit more RSN) upgrades to the latest version of FDP offering possibly even more speedup.

Faster, likely, but how much faster?

FastDoubleParser documentation suggests it can decode many common float and double values 4 time as fast as JDK (*). This is pretty good.
But this only applies to specific decoding part: what has not yet been tested (at least by Jackson project) is how much overall JSON decoding speed might be improved with specific tests data.

So now might be good time to have a look at what speedup we can observe for specific usage. :)

(*) this obviously depends on many things, from input data to JDK versions etc; project README has more information.

What and How to test?

The first thing we need is a test framework to build on. Fortunately there is already jackson-benchmarks project which I use to measure relative performance of different Jackson versions, format back-ends and processing styles (JsonNode vs POJOs, blocking/async etc).
It uses standard JMH micro-benchmark framework and already has a POJO-based setup —main benchmark uses “MediaItem” class and date from jvm-serializers benchmark suite.
So it seems simple enough to add another POJO type.

But we also need data. This is where “Awesome JSON Datasets” repo comes in handy: while most examples have textual documents (at most including some integer numbers), some contain floating-point numbers.
In particular, “Currency” set looks like a good fit: most of the sample document consists of Currency-Id:Exchange-Rate pairs, where latter is a floating-point number with 2-digit fraction:

{"provider":"https://www.exchangerate-api.com",
 "terms":"https://www.exchangerate-api.com/terms",
 "base":"USD",
 "date":"2023-04-01",
 "time_last_updated":1680307201,
 "rates":{
   "USD":1,"AED":3.67,"AFN":86.8,"ALL":104.6
   // ...
 }
}

We can work with that, defining f.ex:

public class Currency {
    public String base, date, provider, terms;
    public int time_last_updated;

    public Map<String, Double> rates;
}

With that I was able to augment existing tests and get some numbers to share.
Sample data files are about 2 kB in size — bit on small side but not unreasonable.

Test setup

Test are run on my home Mac Mini 2018 (3.2 ghz, 6 core Intel), with both JDK 8 (1.8.0_362) and 17 (Temurin 17.0.4.1), using SDKMan to switch between versions.
Typical invocations after building (mvn clean package after git clone ing jackson-benchmarks repo) look something like:

java -Xmx256m -jar target/perf.jar ".*Json.*StdReadVanilla.readCurrencyPojo.*" -wi 3 -w 1 -i 5 -r 1 -f 5

and are included in output sections in case you want to run them on your own.

TL; DNR! +15% throughput

Ok, so, a quick summary for this test suite, comparing non-optimized and optimized performance with Java 8 (Jackson 2.15.0-rc2; tests run with JDK 1.8.0_362):

java -Xmx256m -jar target/perf.jar ".*Json.*StdReadVanilla.readCurrencyPojo.*" -wi 3 -w 1 -i 5 -r 1 -f 5

Benchmark                                    Mode  Cnt      Score     Error  Units
JsonStdReadVanilla.readCurrencyPojoDefault  thrpt   45  61012.389 ± 529.749  ops/s
JsonStdReadVanilla.readCurrencyPojoFast     thrpt   45  70684.408 ± 621.934  ops/s

So it looks like for our test case reading JSON into Currency POJO we get about +15% higher throughput (speed up).

This is a bit less than we might have expected, but if look at couple of reasons this is not entirely outside of expected numbers:

• There is processing overhead with settings up POJO values and other processing beyond simple decoding (possibly 30–40% of total time, half of which is for calling setters via Reflection)
• About half of JSON content in this dataset are property names, decoding of which does not change (no speedup).

So that:

1. Of total time, maybe 60% is spent by low-level decoding of JSON tokens
2. Of decoding time, we could say half is spent on floating-point number decoding (probably bit more in reality); leaving 30% of total time

That is: if FDP makes decoding 2x as fast, we would observe 15% total time reduction. So the results are within roughly reasonable order of magnitude, if on the lower end.

Results: JDK 8 vs JDK 17

Another interesting to look at is if and how optimizations vary across JDK versions. Since JDK 17 is generally much more advanced and optimized than JDK 8 one might expect higher throughput in general — but how about speed up with FDP?

Results (with JDK Temurin-17.0.4.1+), however, are quite unexpected:

java -Xmx256m -jar target/perf.jar ".*Json.*StdReadVanilla.readCurrencyPojo.*" -wi 3 -w 1 -i 5 -r 1 -f 9

Benchmark                                    Mode  Cnt      Score      Error  Units
JsonStdReadVanilla.readCurrencyPojoDefault  thrpt   45  56405.881 ± 1022.490  ops/s
JsonStdReadVanilla.readCurrencyPojoFast     thrpt   45  58975.778 ± 1147.913  ops/s

So basically it looks as if:

• JDK 17 is 10% slower running this benchmark for baseline case
• Speed improvement is in 3–5% range — not very different from test number variation, i.e. barely statistically significant

This is something that I would like to understand better and hope to collaborate more with FDP author and others to investigate.

Results: JSON vs Binary formats

Another interesting dimension is that of relative performance benefits of using a binary format such as Smile or CBOR over JSON. Luckily this test is included as well. Numbers with JDK 8 look something like this (different test run from earlier JSON-only test):

java -Xmx256m -jar target/perf.jar ".*StdReadVan.*readCurrencyPojo.*" -wi 4 -w 1 -i 3 -r 1 -f 3
c.f.j.p.cbor.CBORStdReadVanilla.readCurrencyPojoDefault       thrpt    9  148044.148 ± 4754.251  ops/s
c.f.j.p.json.JsonStdReadVanilla.readCurrencyPojoDefault       thrpt    9   62142.328 ±  948.685  ops/s
c.f.j.p.json.JsonStdReadVanilla.readCurrencyPojoFast          thrpt    9   69232.378 ± 2149.124  ops/s
c.f.j.p.smile.SmileStdReadVanilla.readCurrencyPojoDefault     thrpt    9  139763.953 ± 7425.771  ops/s

And it looks like overall performance of using Smile format is about 2x (JSON -50% throughput) compared to JSON for such floating-point heavy use case, when comparing optimized use case (and bit more, +55% for JDK-based decoder).

Is this a lot? Let’s look at “MediaItem” POJO use case results (JDK8 as well):

java -Xmx256m -jar target/perf.jar ".*StdReadVan.*PojoMedia.*" -wi 4 -w 1 -i 5 -r 1 -f 9 -t 1
c.f.j.p.cbor.CBORStdReadVanilla.readPojoMediaItem         thrpt   45  629841.680 ±  7130.534  ops/sc
c.f.j.p.smile.SmileStdReadVanilla.readPojoMediaItem       thrpt   45  792019.637 ±  7919.285  ops/s
c.f.j.p.json.JsonStdReadVanilla.readPojoMediaItem         thrpt   45  552983.029 ± 11502.097  ops/s

which would give Smile +50% performance benefit when there is no floating-point processing (*)

This suggests that there are significant performance benefits of using Binary formats specifically when passing floating-point data.

(*) note, however, that MediaItem test documents are rather small — 200–500 bytes depending on format in question, 400 for JSON — so the speed up might be different for bigger documents (for small documents per-document fixed overhead has more effect than bigger ones).

Takeaways?

So, although 15% improvement for a seemingly optimal (*) use case does not sound very impressive, one can also think about ease-of-use: you can enable this processing by enabling it for ObjectMapper (or ObjectReader) with:

JsonFactory f = JsonFactory.builder()
   .enable(StreamReadFeature.USE_FAST_DOUBLE_PARSER)
   .build();
ObjectMapper mapper = new JsonMapper(f);

// Or, with existing JsonMapper enable for ObjectReader
ObjectReader r = mapper.reader()
  .with(StreamReadFeature.USE_FAST_DOUBLE_PARSER);

and get up to 15% speedup for such use cases (and possibly 5–10% for use cases where there is some FP data to consider). This seems like a very easy win for some usage, with limited downside.

(*) actually this is NOT the most optimal use case: one could have case where majority of data are floating-point numbers — big arrays of FP numbers. But I did not have easy access to such data.

Possible Future Work

So, results are quite interesting; and definitely more suggestive than definitive. I would really like to understand them more (and/or find issues with test itself). For example:

1. Is Jackson’s usage of FDP sub-optimal? While numbers are roughly in acceptable ballpark it seems like speedup should be slightly bigger (for example, decoding property names is probably faster than number decoding, so FP decoding improvement should have bigger weight)
2. What is the reason for observed unexpectedly poor performance with JDK 17 — both in general, and wrt. less improvement from optimized decoder
3. If we had truly FP-dominating use case — where payload is mostly JSON Array(s) containing floating-point values — how would results differ?

So if you are interested in helping here (or just generally curious), please have a look and share your results!