Jackson 2.9 features

Wow. Now that Jackson 2.9.0 is finally out (*) — it took almost 1 full year to get from 2.8.0 (Features) to 2.9.0, whereas previous versions took 4–6 months — it is time time to go over treasure trove of new features included in 2.9.

As usual, the full 2.9 release notes should give 360 degree view of all changes, but I will try to pick most impactful features for a brief overview here. I will also try to blog some more about one of features (non-blocking parsing), usage scenarios (config input merging, handling), and format improvements (Avro reader/writer schema evolution) at a later point, but let’s start with an overview.

(*) in fact 2.9.1 patch version is already out.

Yet More Bigger “Minor” Release

  1. We started with a concise but ambitious list of about dozen major “should have” features, and while working on other things too, really wanted to make sure most of these features will make it in. This tends to fill release more than focusing on date of delivery (but extend timeline obviously).
  2. It is very likely that 2.9 will be the last minor version for Jackson 2.x — so any API changes, new features NEED to be included, or wait until big version change of 3.0. This had the result of forcing inclusion of some things that previously might have got moved into the “next minor version” bucket.

Compatibility

Modules

New Modules

Significant Module Improvements

  • Avro format module: full support for Schema evolution: ability to specify separate “Reader” and “Writer” schemas. Check AvroSchema method withReaderSchema() for entry point — Writer schema created first, then a new instance with Reader schema created, and this is given to ObjectReader (NOTE: I need to blog more about this)
  • CSV format module: misc added fixes, features, including but not limited to CsvParser.Feature.INSERT_NULL_FOR_MISSING_COLUMNS, CsvParser.Feature.ALLOW_TRAILING_COMMA, CsvParser.Feature.FAIL_ON_MISSING_COLUMNS — also, new Exception type, CsvMappingException for better error handling
  • Properties format module: better interoperability with System properties (“read” System properties); reading from / writing as java.util.Properties — especially convenient to allow, say, command-line overrides!

Many other modules were improved as well, as per Release Notes.

New Core Functionality

“No Trailing Junk”

With 2.9 you can configure ObjectMapper or ObjectReader to briefly check that input stream contains nothing after value has been bound:

ObjectMapper mapper = new ObjectMapper()
.enable(DeserializationFeature.FAIL_ON_TRAILING_TOKENS);
Value v = mapper.reader();
// throws MismatchedInputException if valid tokens found;
// otherwise throws tokenization exception

Note that this effectively calls JsonParser.nextToken() after binding (which would trigger decoding exception, JsonParseException, if non-json content follows); and verifies that method returns null which signals end of content: if this is not true, MismatchedInputException is thrown.

@JsonAlias

Usage is similar to common @JsonProperty annotation (and similar):

public class Bean {
@JsonAlias({ "id", "_name" })
public String name;
}

In this case, messages like:

{ "name" : "Alice" }
{ "id" : "Bob" }
{ "_name" : "Fred" }

would all bind seamlessly into Bean while still serializing using primary property name, name.

Alias information is also exposed through Jackson’s Introspection, used for generating JSON Schema, Avro Schemas and Protobuf protoc definitions, so it can help interoperability: this is especially useful with Avro schema evolution.

Custom `@JsonInclude`

public class Filtered {
@JsonInclude(value=Include.CUSTOM,
valueFilter=MyFilter.class)
public String value;
}
public class MyFilter {
@Override
public boolean equals(Object other) {
// return 'true' to not write (exclude), 'false' to write
return "secret".equals(other);
}
}

Filtering is decided based on calling basic boolean equals() method: if true is returned, value is filtered out (property and value are NOT serialized); if false returned it is included normally.

Note that for Map values (but not arrays or Collections, for now) you can alternatively specify content / contentFilter settings — these apply to contents (values of entries within Map), whereas value / valueFilter apply to the Map value itself.

Null replacement/error/skipping (on deserialization)

Jackson 2.9 adds new properties to existing (but rarely used) @JsonSetter annotation to allow specifying one of common alternatives:

public class Config {
@JsonSetter(nulls=Nulls.SKIP) // don't override if null
public Coordinates coordinates = new Coordinates(1, 2);
@JsonSetter(nulls=Nulls.AS_EMPTY) // null will clear to empty list
public List<String> names = Collections.emptyList();
@JsonSetter(nulls=Nulls.AS_EMPTY,
contentNulls=Nulls.FAIL)
public Map<String, Extra> attributes;
}

In this example, we have set a suitable default POJO for coordinates, and should incoming content have null for that property, we want to just ignore this null.
Property names just needs to exist as empty List; we could have either used SKIP as above, or force it to be set as “empty” object (relevant for structured types and Strings — deserializer is asked for what that value is via JsonDeserializer.getEmptyValue(context) method).
And finally, there is also another property, contentNulls, which is applied to values of Maps: in this case we make mapper throw JsonMappingException if null value is passed in. This could as well have been SKIP, if we wanted to quietly just ignore such values: this a design choice that some developers feel strong about (defensive vs assertive).

But wait! There’s more. In above example, we applied settings to individual properties. But there are 2 other possibilities too:

  1. Specify defaults for a type (possibly overridden by per-property annotations)
  2. Specify global defaults to use (unless overridden by type, property)
ObjectMapper mapper = new ObjectMapper();// Prevent assigned of `null` for any `String` property!
mapper.configOverride(String.class)
.setSetterInfo(JsonSetter.Value.forValueNulls(Nulls.FAIL));
// Skip all null assignments (unless overridden)
mapper.setDefaultSetterInfo(JsonSetter.Value.forValueNulls(Nulls.SKIP));

So. For all you null-haters out there: you have new tools to discriminate against those pesky null values that may try to sneak in that content.
This feature is of particular importance when using next feature, merging, covered next.

Merging

Config config = ... ; // create and initialize
mapper.readerForUpdating(config).readValue(src);

which works fine but only for scalar values: structured values like Maps and POJOs would still be replaced. With 2.9 we will finally support “deep” merge, in which structured values are merged as well.

Support is added using combination of annotations and config overrides, quite similar to how null-handling was specified above. So it a property of types and properties (where should deep merge be applied, where not), and not a Feature to enable. Let’s have a look at augmented example

public class Config {
@JsonMerge // yes please merge
@JsonSetter(nulls=Nulls.SKIP) // don't override if null
public Coordinates coordinates = new Coordinates(1, 2);
@JsonMerge(OptBoolean.FALSE) // don't try merging Lists
@JsonSetter(nulls=Nulls.AS_EMPTY)
public List<String> names = Collections.emptyList();
}

As with null-handling, you can also specify merging defaults by type and globally:

// to allow merging for values of specific type:
mapper.configOverride(MyConfig.class).setMergeable(true);
// or by default, for types for which applicable:
mapper.setDefaultMergeable(true);

There are some nuances to merging, and ability to do so:

  1. Scalar types can not be merged: if annotated (or via defaulting), attempts ignored (unless MapperFeature.IGNORE_MERGE_FOR_UNMERGEABLE is disabled)
  2. Merging of Collections is simple: entries are appended; no checks for existence applied — use Sets when you want “true” merge
  3. Although arrays can not be modified in place, “merge” is performed by constructing a new array instance with contents appended
  4. Support is needed by JsonDeserializer so datatype modules need to be retrofitted to support it for relevant types (collections, Maps)

Avro Module: support reader/writer schema

public class XY {
public int x, y;
}
public class XYZ {
public int x, y, z;
}
AvroMapper mapper = new AvroMapper();
final AvroSchema xySchema = mapper.schemaFor(XY.class);
// write an XY:
byte[] avro = mapper.writer(xySchema)
.writeValueAsbytes(new XY(1, 2));
// but "translate" into XYZ via reader/writer schema
final AvroSchema trSchema = xySchema.withReaderSchema(
mapper.schemaFor(XYZ.class));
XYZ xyz = mapper.readerFor(XYZ.class)
.with(trSchema)
.readValue(avro);

It is worth noting that Jackson’s data-binding typically supports such changes (addition and removal of properties; rename via new @JsonAlias) automatically. But Avro encoder/decoder absolutely require use of Reader/Writer schema concept: attempts to “just read” something encoded using Schema 1 with different (even if compatible!) Schema 2 will result either in failure (best case) or data corruption (worst case) — Avro format does not have enough metadata to distinguish these cases.
Schemas used as Reader/Writer schema can come from external files (read with AvroMapper.schemaFrom()), generated from POJOs (AvroMapper.schemaFor()) and even built using Apache’s standard Avro lib if necessary.

Besides support for reader/writer schemas, there is plenty more included in 2.9 Avro module:

  • Support for many of annotations from Apache Avro lib (@Stringable, @AvroAlias, @AvroEncode.
  • Improved read performance with native decoder (although Apache Avro lib BinaryDecoder is still be available as an alternative as well)
  • Ability to write “File Avro” with AvroGenerator.Feature.AVRO_FILE_OUTPUT — this will add header into output, needed by some processing tools and storage systems (note, however, that Jackson does not currently read such format — room for future improvement!)

Non-Blocking Reading (aka Async Parsing)

The idea is that instead of using a blocking input source like InputStream or Reader input is “fed” (pushed) when it comes available. Minimally intrusive approach (as shown by Aalto which extends Stax with minimal changes) includes just two things:

  • Possibility for JsonParser to return JsonToken.NOT_AVAILABLE when it does not have enough input to decode the next token
  • Ability to feed more input after NOT_AVAILABLE has been returned

I will have to write a bigger example of usage at a later point, but here is the process:

JsonFactory f = new JsonFactory();
// will create async feeder based on byte[]:
JsonParser asyncP = f.createNonBlockingByteArrayParser();
while (true) {
JsonToken t;
while ((t = asyncP.nextToken()) == JsonToken.NOT_AVAILABLE) {
// need to feed more
ByteArrayFeeder feeder =
(ByteArrayFeeder) asyncP.getNonBlockingInputFeeder();
feeder.feedInput(buffer, offset, length);
}
// token available, process:
// ...
}

So basically feeding of input occurs when previous content has been decoded and handled. Future versions may (and should) add support for other feeders such as ByteBuffer.

But wait! There is more! 2.9 also includes Async/Non-blocking decoder for Smile format (“binary JSON”); and we hope to provide one for CBOR relatively soon as well. It should in fact be possible to add support for almost all (if not all?) formats eventually, if there is interest.

And perhaps best of all, upcoming Spring 5 release will include support for non-blocking reading, using Jackson, as well.

Misc Improvements

  • @JsonValue for fields — formerly it was only allowed on methods (getters)
  • @JsonView on classes: if set, annotated View will be used as the default view for properties that are not otherwise annotated — this can reduce need for common View annotations a lot.
  • @JsonStreamContext.pathAsPointer(): convenient way to get JsonPointer from context accessed by JsonParser.getParsingContext() and JsonGenerator.getOutputContext(). Useful for diagnostics, error messages, or perhaps even for keeping references to content as JsonPointer expressions can be used both for traversing JsonNodes and filtering JsonParser content.
  • Strict vs Lenient parsing of java.util.Dates (and Calendars): using @JsonFormat(lenient = OptBoolean.FALSE) you can force strict handling of Dates (default is lenient handling) — this prevents use of Dates like “35th of February”. NOTE: aside from per-property annotations, you can also use “Config Override” system for type defaults.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
@cowtowncoder

Open Source developer, most known for Jackson data processor (nee “JSON library”), author of many, many other OSS libraries for Java, from ClassMate to Woodstox