Home | Send Feedback

JavaScript BigInt and JSON

Published: January 04, 2019  •  javascript, java

Update January 2019: Feature set for ECMAScript 2019 is final and BigInt is not part of it. See this blog post for all the features in ES2019: http://2ality.com/2018/02/ecmascript-2019.html


In my previous blog post I took a closer look at the upcoming new numeric type BigInt in JavaScript

This feature is currently (January 2019) not standardized yet. It is still in stage 3 of the TC39 process. Although not standardized yet we can already experiment with the feature in Chrome since version 67.

Problem: JSON

One issue I described in my blog post is that JSON does not support BigInt. If you try to serialize and deserialize an object with BigInt values in it, the methods JSON.stringify() and JSON.parse() throw errors:

JSON.stringify({a:10n})
// Uncaught TypeError: Do not know how to serialize a BigInt

JSON.parse("{\"a\":10n}")
// Uncaught SyntaxError: Unexpected token n in JSON at position 7

Fortunately there is a way to work around this problem by converting BigInt to string and back. I already showed you a JavaScript solution for this in my previous blog post. I will show you this solution again a bit further below, but first we look at the problems that a BigInt data type tries to solve.

Problem: Maximum values

In this example we use a Spring Boot RestController with a GET handler, that creates a POJO with 3 fields and sends it back to the caller as JSON.

  @GetMapping("/fetchData1")
  public Payload fetchData1() {
    long mersenne8 = (long) (Math.pow(2, 31) - 1);
    var mersenne9 = new BigInteger("2").pow(61).subtract(BigInteger.ONE);
    return new Payload(1, mersenne8, mersenne9);
  }

Application.java

The Payload POJO just consists of three fields. These are all data types that can store values that are potentially larger than the JavaScript number type can store.

public class Payload {
  private long value1;

  private Long value2;

  private BigInteger prime;

From JavaScript we call the service with the Fetch API.

            const response1 = await fetch('fetchData1');
            const obj1 = await response1.json();
            console.log(obj1);

We get back this response

{value1: 1, value2: 2147483647, prime: 2305843009213694000}

No error and at first glance looks okay, but prime contains the wrong value. It should be 2305843009213693951 (2^61 - 1). A very subtle error because there is no error or warning message. JavaScript is not able to exactly represent this integer with the number data type. The largest integer, JavaScript is able to exactly represent with number, is 2^53. There is also a constant Number.MAX_SAFE_INTEGER available you can access from your code that returns 2^53 - 1, the largest integer where you can add 1 and still get an exact result.

Number.MAX_SAFE_INTEGER     // 9007199254740991
Number.MAX_SAFE_INTEGER + 1 // 9007199254740992
Number.MAX_SAFE_INTEGER + 2 // 9007199254740992 (wrong!!!)

This is not a bug, it's just how the number data type is implemented in JavaScript. See the specification for more information.

Next we look at an endpoint that returns a number (2^1279 - 1) that is by far too large for the number type.

  @GetMapping("/fetchData2")
  public Payload fetchData2() {
    var mersenne15 = new BigInteger("2").pow(1279).subtract(BigInteger.ONE);
    return new Payload(2, 9007199254740991L, mersenne15);
  }

Application.java

If we call this endpoint from JavaScript we get back the following response:

{value1: 2, value2: 9007199254740991, prime: Infinity}

No error, but we get back this strange Infinity. Infinty is a number in JavaScript and the JSON parser assigns this value when he encounters a number in the JSON that is too big for storing into a number.


There are similar problems when we try to send objects from a browser to the back end.

  @PostMapping("/storeData")
  @ResponseStatus(code = HttpStatus.NO_CONTENT)
  public void postData(@RequestBody Payload payload) {
    System.out.println(payload);
  }

Application.java

From JavaScript we send an object with a number that is bigger than Number.MAX_SAFE_INTEGER

    const storeObj = {
        value1: 3,
        value2: 2147483647,
        prime: 2305843009213693951
    };

    fetch('storeData', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json'
        },
        body: JSON.stringify(storeObj)
    })

On the server the POST handler prints out this information

// Payload [value1=3, value2=2147483647, prime=2305843009213694000]

Same error as before. 2305843009213693951 can't be represented exactly in the number data type. JSON.stringify() therefore writes 2305843009213694000 into the JSON and sends that over the wire to the back end.

As a side note you can also try and send Infinity.

const storeObj = {
        value1: 3,
        value2: 2147483647,
        prime: Infinity
    };

The JSON.stringify() method handles this case by writing null into the JSON.

"{"value1":3,"value2":2147483647,"prime":null}"

Solution: Sending BigInt as string in JSON

As a solution for the missing BigInt support in JSON, we convert the BigInts to strings and then convert them back to a number type. We do this in JavaScript and in the Spring Boot back end application.


Server: Serialization

To customize the serialization process in Jackson we have to write subclasses of the com.fasterxml.jackson.databind.JsonSerializer class and override the serialize method.

public class BigIntegerSerializer extends JsonSerializer<BigInteger> {

  @Override
  public void serialize(BigInteger value, JsonGenerator gen,
      SerializerProvider serializers) throws IOException {
    gen.writeString(value.toString() + "n");
  }

}

BigIntegerSerializer.java

public class LongSerializer extends JsonSerializer<Long> {

  @Override
  public void serialize(Long value, JsonGenerator gen, SerializerProvider serializers)
      throws IOException {
    gen.writeString(value.toString() + "n");
  }

}

LongSerializer.java

The Long serializer not only works with the Long object it also supports the primitive data type long. Notice that the value parameter passed to the method is never null, we can omit any null checks.

Next we need to customize the Jackson JSON parser. In Spring Boot we can do this in two ways. Either globally or locally on the field level.

To install these serializers globally we create a class that extends the com.fasterxml.jackson.databind.module.SimpleModule class and then call this.addSerializer() in the constructor to register our serializers. Notice that we add the LongSerializer twice, once for handling the object Long and once for handling the primitive data type long.

We also annotate this class with @Component to denote this class as a spring managed bean. Spring Boot automatically picks up this class and configures the ObjectMapper instance used for JSON serialization.

import com.fasterxml.jackson.databind.module.SimpleModule;

@Component
public class CustomModule extends SimpleModule {
  public CustomModule() {
    this.addSerializer(BigInteger.class, new BigIntegerSerializer());
    this.addSerializer(Long.class, new LongSerializer());
    this.addSerializer(long.class, new LongSerializer());
  }
}

When we call our two endpoints from the previous section, we get back these JSON responses

/fetchData1

{"value1":"1n","value2":"2147483647n","prime":"2305843009213693951n"}

/fetchData2

{"value1":"2n","value2":"9007199254740991n","prime":"10407932194664399081925240327364085538615262247266...958028878050869736186900714720710555703168729087n"}

Installing serializers globally looks very convenient, but can lead to problems especially when you add a serializer for a common type like long. Maybe most long fields will never exceed the range of a JavaScript number in your application and in these cases you want to send them as normal numbers over JSON. Instead of configure the serializers globally you can configure them directly on a POJO field.

In this POJO we apply the custom serializers on the fields value2 and prime. You may place the @JsonSerialize on the field or on the get method.

public class Payload {
  private long value1;

  private Long value2;

  @JsonSerialize(using = BigIntegerSerializer.class)
  private BigInteger prime;

  @JsonSerialize(using = LongSerializer.class)
  public Long getValue2() {
    return this.value2;
  }

With this setup we get the following JSON responses. Notice that value1 is serialized as a normal number because we didn't add a serializer to that field.

/fetchData1

{"value1":1,"value2":"2147483647n","prime":"2305843009213693951n"}

/fetchData2

{"value1":2,"value2":"9007199254740991n","prime":"10407932194664399081925240327364085538615262247266...958028878050869736186900714720710555703168729087n"}

Client: Deserialization

After setting up the server we now receive strings in the form "2147483647n" that signify a BigInt value. If we simply call JSON.parse() on this JSON we get back a string, but we want an object with a BigInt property.

The JSON.parse() method supports an optional second parameter called reviver. This is a function that is called for each key/value pair. Here we check if the value is a string, contains just numbers and the letter n at the end. If that is the case convert it to a BigInt.

function parseReviver(key, value) {
    if (typeof value === 'string' && /^\d+n$/.test(value)) {
        return BigInt(value.slice(0, -1));
    }
    return value;
}

index.html

Unfortunately the json() method of the Fetch response object does not support the reviver argument, so we can't use this pattern.

  const response1 = await fetch('fetchData1');
  const obj1 = await response1.json();

Instead, we have to access the raw string response with text() and then call JSON.parse()

  const response1 = await fetch('fetchData1');
  const obj1 = JSON.parse(await response1.text(), parseReviver);
  console.log(obj1);
  // {value1: 1, value2: 2147483647n, prime: 2305843009213693951n}

  const response2 = await fetch('fetchData2');
  const obj2 = JSON.parse(await response2.text(), parseReviver);
  console.log(obj2);
  // {value1: 2, value2: 9007199254740991n, prime: 10407932194664399081925240327364085538615262247266…958028878050869736186900714720710555703168729087n}

Client: Serialization

As we have seen in the first section, serializing an object with JSON.stringify() throws an error if it contains properties with BigInt values. Fortunately this method also supports a second optional parameter called replacer. A function that is called for each key/value pair. In this replacer function we check the data type and when it's a BigInt convert it to a string and append the letter 'n'

function stringifyReplacer(key, value) {
    if (typeof value === 'bigint') {
        return value.toString() + 'n';
    } else {
        return value;
    }
}

index.html

The following code sends a POST request to our Spring Boot backend and passes the replacer function as argument to the JSON.stringify() method.

    const storeObj = {
        value1: 3,
        value2: 2147483647n,
        prime: 2305843009213693951n
    };

    await fetch('storeData', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json'
        },
        body: JSON.stringify(storeObj, stringifyReplacer)
    })

index.html

This is the JSON the browser sends to the server.

{"value1":3,"value2":"2147483647n","prime":"2305843009213693951n"}

Server: Deserialization

With the Jackson library, we may customize the deserialization process by creating subclasses of the com.fasterxml.jackson.databind.JsonDeserializer class. For this example I wrote a deserializer for BigInteger and one for Long and long.

public class BigIntegerDeserializer extends JsonDeserializer<BigInteger> {

  @Override
  public BigInteger deserialize(JsonParser jp, DeserializationContext ctxt)
      throws IOException, JsonProcessingException {
    String value = jp.getText();
    if (value != null) {
      value = value.strip();
      if (value.length() > 0) {
        return new BigInteger(value.substring(0, value.length() - 1));
      }
    }
    return null;
  }

}

BigIntegerDeserializer.java

public class LongDeserializer extends JsonDeserializer<Long> {

  @Override
  public Long deserialize(JsonParser jp, DeserializationContext ctxt)
      throws IOException, JsonProcessingException {
    String value = jp.getText();
    if (value != null) {
      value = value.strip();
      if (value.length() > 0) {
        return Long.valueOf(value.substring(0, value.length() - 1));
      }
    }
    return null;
  }

}

LongDeserializer.java

In these two implementations I check for null and for empty strings. In these two cases the deserializer returns null in all other cases it removes the last letter ('n') and tries to convert the string to a BigInteger respectively a Long.

We can configure the deserializers, like the serializers, globally or locally on a field by field basis.

Globally, you register the deserializers with this.addDeserializer() in a module class.

@Component
public class CustomModule extends SimpleModule {

  public CustomModule() {
    ...
    this.addDeserializer(BigInteger.class, new BigIntegerDeserializer());
    this.addDeserializer(Long.class, new LongDeserializer());
    this.addDeserializer(long.class, new LongDeserializer());
  }
}

Alternatively you specify the deserializer with the @JsonDeserialize annotation on the field or on the set method

public class Payload {
  private long value1;

  private Long value2;

  @JsonDeserialize(using = BigIntegerDeserializer.class)
  private BigInteger prime;

  @JsonDeserialize(using = LongDeserializer.class)
  public void setValue2(Long value2) {
    this.value2 = value2;
  }

When the client sends the POST from the previous section you get these values. Notice the correct value for prime

  // Payload [value1=3, value2=2147483647, prime=2305843009213693951]

This concludes our full stack implementation for sending BigInt over JSON. You find the complete source code for this example on GitHub:
https://github.com/ralscha/blog2019/tree/master/bigintjson