In my previous blog post, I took a closer look at the new numeric type BigInt in JavaScript
Problem: JSON ¶
I wrote in my previous blog post that JSON does not support BigInt.
If you try to serialize and deserialize an object with BigInt values in it, the methods JSON.stringify() and JSON.parse() throw errors:
JSON.stringify({a:10n})
// Uncaught TypeError: Do not know how to serialize a BigInt
JSON.parse("{\"a\":10n}")
// Uncaught SyntaxError: Unexpected token n in JSON at position 7
Fortunately, there is a way to work around this problem by converting BigInt to string and back. I already showed you a JavaScript solution for this in my previous blog post. I will show you this solution again a bit further below, but first, we look at the problems that a BigInt data type tries to solve.
Problem: Maximum values ¶
In this example, we use a Spring Boot RestController with a GET handler that creates a POJO with three fields and sends it back to the caller as JSON.
@GetMapping("/fetchData1")
public Payload fetchData1() {
long mersenne8 = (long) (Math.pow(2, 31) - 1);
var mersenne9 = new BigInteger("2").pow(61).subtract(BigInteger.ONE);
return new Payload(1, mersenne8, mersenne9);
}
The Payload POJO consists of three fields. All data types can store potentially larger values than the JavaScript number type can hold.
public class Payload {
private long value1;
private Long value2;
private BigInteger prime;
From JavaScript, we call the service with the Fetch API.
const response1 = await fetch('fetchData1');
const obj1 = await response1.json();
console.log(obj1);
We get back this response.
{value1: 1, value2: 2147483647, prime: 2305843009213694000}
No error, and at first glance looks okay, but prime contains the wrong value. It should be 2305843009213693951 (2^61 - 1).
A very subtle error because there is no error or warning message. JavaScript cannot accurately represent this integer with the number data type.
The largest integer, JavaScript, can exactly represent with number, is 2^53. There is also a constant Number.MAX_SAFE_INTEGER available you can access from your code that returns 2^53 - 1, the largest
integer where you can add one and still get an exact result.
Number.MAX_SAFE_INTEGER // 9007199254740991
Number.MAX_SAFE_INTEGER + 1 // 9007199254740992
Number.MAX_SAFE_INTEGER + 2 // 9007199254740992 (wrong!!!)
This is not a bug. It's just how the number data type is implemented in JavaScript. See the specification for more information.
Next, we look at an endpoint that returns a number (2^1279 - 1) that is by far too large for the number type.
@GetMapping("/fetchData2")
public Payload fetchData2() {
var mersenne15 = new BigInteger("2").pow(1279).subtract(BigInteger.ONE);
return new Payload(2, 9007199254740991L, mersenne15);
}
If we call this endpoint from JavaScript, we get back the following response:
{value1: 2, value2: 9007199254740991, prime: Infinity}
No error, but we get back this strange Infinity. Infinity is a number in JavaScript, and the JSON parser assigns this value when he encounters a number in the JSON that is too big for
storing into a number.
There are similar problems when sending objects from a browser to the back end.
@PostMapping("/storeData")
@ResponseStatus(code = HttpStatus.NO_CONTENT)
public void postData(@RequestBody Payload payload) {
System.out.println(payload);
}
From JavaScript, we send an object with a number that is bigger than Number.MAX_SAFE_INTEGER
const storeObj = {
value1: 3,
value2: 2147483647,
prime: 2305843009213693951
};
fetch('storeData', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(storeObj)
})
On the server, the POST handler prints out this information.
// Payload [value1=3, value2=2147483647, prime=2305843009213694000]
Same error as before. 2305843009213693951 can't be represented precisely in the number data type.
JSON.stringify() therefore writes 2305843009213694000 into the JSON and sends that over the wire to the back end.
You can also try and send Infinity.
const storeObj = {
value1: 3,
value2: 2147483647,
prime: Infinity
};
The JSON.stringify() method handles this case by writing null into the JSON.
"{"value1":3,"value2":2147483647,"prime":null}"
Solution: Sending BigInt as a string in JSON ¶
As a solution for the missing BigInt support in JSON, we convert the BigInts to strings and then convert them back to a number type. We do this in JavaScript and in the Spring Boot back end application.
Server: Serialization ¶
To customize the serialization process in Jackson, we have to write subclasses of the com.fasterxml.jackson.databind.JsonSerializer class
and override the serialize method.
public class BigIntegerSerializer extends JsonSerializer<BigInteger> {
@Override
public void serialize(BigInteger value, JsonGenerator gen,
SerializerProvider serializers) throws IOException {
gen.writeString(value.toString() + "n");
}
}
public class LongSerializer extends JsonSerializer<Long> {
@Override
public void serialize(Long value, JsonGenerator gen, SerializerProvider serializers)
throws IOException {
gen.writeString(value.toString() + "n");
}
}
The Long serializer works with the Long object and supports the primitive data type long.
Notice that the value parameter passed to the method is never null; we can omit any null checks.
Next, we need to customize the Jackson JSON parser. In Spring Boot, we can do this in two ways. Either globally or locally on the field level.
To install these serializers globally, we create a class that extends the com.fasterxml.jackson.databind.module.SimpleModule class
and then call this.addSerializer() in the constructor to register our serializers. Notice that we add the LongSerializer twice
for handling the object Long and once for the primitive data type long.
We also annotate this class with @Component to denote this class as a Spring-managed bean. Spring Boot automatically picks up this class and configures the ObjectMapper instance used for JSON serialization.
import com.fasterxml.jackson.databind.module.SimpleModule;
@Component
public class CustomModule extends SimpleModule {
public CustomModule() {
this.addSerializer(BigInteger.class, new BigIntegerSerializer());
this.addSerializer(Long.class, new LongSerializer());
this.addSerializer(long.class, new LongSerializer());
}
}
When we call our two endpoints from the previous section, we get back these JSON responses.
/fetchData1
{"value1":"1n","value2":"2147483647n","prime":"2305843009213693951n"}
/fetchData2
{"value1":"2n","value2":"9007199254740991n","prime":"10407932194664399081925240327364085538615262247266...958028878050869736186900714720710555703168729087n"}
Installing serializers globally looks very convenient but can lead to problems, especially when you add a serializer for a common type like long. Maybe most long fields will never exceed the
range of a JavaScript number in your application, and in these cases, you want to send them like regular numbers over JSON.
Instead of configuring the serializers globally, you can configure them directly on a POJO field.
This POJO applies the custom serializers on the fields value2 and prime.
You may place the @JsonSerialize on the field or the get method.
public class Payload {
private long value1;
private Long value2;
@JsonSerialize(using = BigIntegerSerializer.class)
private BigInteger prime;
@JsonSerialize(using = LongSerializer.class)
public Long getValue2() {
return this.value2;
}
With this setup, we get the following JSON responses.
Notice that value1 is serialized as a regular number because we didn't add a serializer to that field.
/fetchData1
{"value1":1,"value2":"2147483647n","prime":"2305843009213693951n"}
/fetchData2
{"value1":2,"value2":"9007199254740991n","prime":"10407932194664399081925240327364085538615262247266...958028878050869736186900714720710555703168729087n"}
Client: Deserialization ¶
After setting up the server, we now receive strings in the form "2147483647n" that signify a BigInt value.
If we call JSON.parse() on this JSON, we get back a string, but we want an object with a BigInt property.
The JSON.parse() method supports an optional second parameter called reviver. This is a function that is called for each key/value pair. Here we check if the value is a string, contains just numbers, and the letter n at the end.
If that is the case, convert it to a BigInt.
function parseReviver(key, value) {
if (typeof value === 'string' && /^\d+n$/.test(value)) {
return BigInt(value.slice(0, -1));
}
return value;
}
Unfortunately, the json() method of the Fetch response object does not support the reviver argument,
so we can't use this pattern.
const response1 = await fetch('fetchData1');
const obj1 = await response1.json();
Instead, we must access the raw string response with text() and then call JSON.parse().
const response1 = await fetch('fetchData1');
const obj1 = JSON.parse(await response1.text(), parseReviver);
console.log(obj1);
// {value1: 1, value2: 2147483647n, prime: 2305843009213693951n}
const response2 = await fetch('fetchData2');
const obj2 = JSON.parse(await response2.text(), parseReviver);
console.log(obj2);
// {value1: 2, value2: 9007199254740991n, prime: 10407932194664399081925240327364085538615262247266…958028878050869736186900714720710555703168729087n}
Client: Serialization ¶
As we have seen in the first section, serializing an object with JSON.stringify() throws an error if it contains properties with BigInt values.
Fortunately, this method also supports a second optional parameter called replacer. A function that is called for each key/value pair.
In this replacer function, we check the data type, and when it's a BigInt, convert it to a string and append the letter 'n'.
function stringifyReplacer(key, value) {
if (typeof value === 'bigint') {
return value.toString() + 'n';
} else {
return value;
}
}
The following code sends a POST request to our Spring Boot backend and passes the replacer function to the JSON.stringify() method.
const storeObj = {
value1: 3,
value2: 2147483647n,
prime: 2305843009213693951n
};
await fetch('storeData', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(storeObj, stringifyReplacer)
})
This is the JSON the browser sends to the server.
{"value1":3,"value2":"2147483647n","prime":"2305843009213693951n"}
Server: Deserialization ¶
We may customize the deserialization process with the Jackson library by creating subclasses of the com.fasterxml.jackson.databind.JsonDeserializer class.
I wrote a deserializer for BigInteger and Long and long for this example.
public class BigIntegerDeserializer extends JsonDeserializer<BigInteger> {
@Override
public BigInteger deserialize(JsonParser jp, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
String value = jp.getText();
if (value != null) {
value = value.strip();
if (value.length() > 0) {
return new BigInteger(value.substring(0, value.length() - 1));
}
}
return null;
}
}
public class LongDeserializer extends JsonDeserializer<Long> {
@Override
public Long deserialize(JsonParser jp, DeserializationContext ctxt)
throws IOException, JsonProcessingException {
String value = jp.getText();
if (value != null) {
value = value.strip();
if (value.length() > 0) {
return Long.valueOf(value.substring(0, value.length() - 1));
}
}
return null;
}
}
In these two implementations, I check for null and for empty strings. In these two cases, the deserializer returns null in all other cases it removes the last letter ('n') and tries to convert the string to a BigInteger, respectively, a Long.
You can configure the deserializers, like the serializers, globally or locally on a field-by-field basis.
Globally, you register the deserializers with this.addDeserializer() in a module class.
@Component
public class CustomModule extends SimpleModule {
public CustomModule() {
...
this.addDeserializer(BigInteger.class, new BigIntegerDeserializer());
this.addDeserializer(Long.class, new LongDeserializer());
this.addDeserializer(long.class, new LongDeserializer());
}
}
Alternatively, specify the deserializer with the @JsonDeserialize annotation on the field or the set method.
public class Payload {
private long value1;
private Long value2;
@JsonDeserialize(using = BigIntegerDeserializer.class)
private BigInteger prime;
@JsonDeserialize(using = LongDeserializer.class)
public void setValue2(Long value2) {
this.value2 = value2;
}
You get these values when the client sends the POST from the previous section.
Notice the correct value for prime
// Payload [value1=3, value2=2147483647, prime=2305843009213693951]
This concludes the tutorial about sending BigInt over JSON.
You find the complete source code for this example on GitHub:
https://github.com/ralscha/blog2019/tree/master/bigintjson