Home | Send Feedback

Exchange Protocol Buffers messages between JavaScript and Java

Published: April 04, 2020  •  javascript, java

In a previous blog post, I demonstrate an example that sends Protocol Buffers messages from a Spring Boot server to a JavaScript (Angular) application. A couple of days ago, a reader asked me how to send Protocol Buffers messages the opposite way, from JavaScript to Spring Boot. This is what I'm going to show you in this post.

proto

For the demo application, I created the following proto file. It defines of two messages, UserRequest, which the JavaScript app sends to Spring Boot, and UserResponse which the Spring Boot application sends back as the response.

syntax = "proto3";

option java_package = "ch.rasc.protobuf";

message UserRequest {
  string firstname = 1;
  string lastname = 2;
  uint32 age = 3;
  enum Gender {
    MALE = 0;
    FEMALE = 1;
  }
  Gender gender = 4;
}

message UserResponse {
  string id = 1;
  enum Status {
    OK = 0;
    NOT_OK = 1;
  }
  Status status = 2;
}

User.proto

Server: Setup

To enable Spring Boot`s Protocol Buffers support, I added the protobuf-java dependency to my project.

    <dependency>
      <groupId>com.google.protobuf</groupId>
      <artifactId>protobuf-java</artifactId>
      <version>3.11.4</version>
    </dependency>        

pom.xml

I also added the protoc-jar-maven-plugin to the pom.xml. This plugin is a wrapper for the Protocol Buffers compiler (protoc). The compiler reads the proto file and generates Java code.

      <plugin>
        <groupId>com.github.os72</groupId>
        <artifactId>protoc-jar-maven-plugin</artifactId>
        <version>3.11.4</version>
        <executions>
          <execution>
            <phase>generate-sources</phase>
            <goals>
              <goal>run</goal>
            </goals>
            <configuration>
              <protocVersion>3.11.4</protocVersion>
              <inputDirectories>
                <include>src/main/protobuf</include>
              </inputDirectories>
            </configuration>
          </execution>
        </executions>
      </plugin>

pom.xml

To generate the Java code, I called the generate-sources Maven phase from the command line:

./mvnw generate-sources

The compiler writes the generated code into the <project>/target/generated-sources folder.

Server: Spring Boot

Next, I added the following bean definition. This instantiates Spring's built-in Protocol Buffers message converter. This converter is responsible for serializing and deserializing Protocol Buffers messages. He supports Protocol Buffers version 2 and 3. Spring Boot does not autoconfigure this message converter. Therefore we have to add it manually.

  @Bean
  public ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufHttpMessageConverter();
  }

Application.java

And lastly, I implemented the rest controller, which receives a UserRequest object and sends back a UserResponse response. You see that there is no difference to a rest controller that handles JSON messages. This is a normal controller with a regular POST mapping. Spring and the message converter handle everything else. Important here is that we use the generated classes from the protoc compiler as input and output objects. The message converter expects these objects to be implementations of the interface com.google.protobuf.Message.

@RestController
@CrossOrigin
public class UserController {

  @PostMapping("/register-user")
  public UserResponse registerUser(@RequestBody UserRequest userRequest) {
    System.out.println(userRequest);

    Status status = Status.OK;
    if (userRequest.getAge() < 18) {
      status = Status.NOT_OK;
    }

    return UserResponse.newBuilder().setId(UUID.randomUUID().toString()).setStatus(status)
        .build();
  }

}

UserController.java

Client: Setup

Like on the server-side, we also need a generator and a library that takes care of message encoding and decoding on the client-side. In this demo project, I added the protobufjs library

npm install protobufjs

Then I added the following script to package.json. This script calls the Protocol Buffers compiler and generates a JavaScript file and a TypeScript definition file.

    "pbts": "pbjs -t static-module ../server/src/main/protobuf/User.proto -o src/app/protos/user.js && pbts --no-comments src/app/protos/user.js -o src/app/protos/user.d.ts"

package.json

To execute the script, call npm run pbts from the shell. Make sure that the output directory exists before you run this command.

Client: Application

The client is a trivial Angular application with a form.

<form #form="ngForm" (ngSubmit)="submit(form.value)" class="options">
  <div>
    <label for="firstName">First Name</label><br>
    <input id="firstName" name="firstname" ngModel type="text">
  </div>
  <div>
    <label for="lastName">Last Name:</label><br>
    <input id="lastName" name="lastname" ngModel type="text">
  </div>
  <div>
    <label for="age">Age:</label><br>
    <input id="age" max="120" min="0" name="age" ngModel type="number">
  </div>
  <div>
    <input id="maleGender" name="gender" ngModel type="radio" value="MALE"><label for="maleGender">Male</label>
    <input id="femaleGender" name="gender" ngModel type="radio" value="FEMALE"><label for="femaleGender">Female</label>
  </div>

  <button type="submit">Submit</button>
</form>

app.component.html


The method submit receives the form inputs, sends them to the server and handles the server response.

First, the method creates and encodes a UserRequest object. The encode({...}).finish() method returns a Uint8Array object.

  submit(formValues: { firstname: string, lastname: string, age: number, gender: string }) {

    const encodedUserRequest = UserRequest.encode({
      firstname: formValues.firstname,
      lastname: formValues.lastname,
      age: formValues.age,
      gender: (formValues.gender === 'MALE') ? UserRequest.Gender.MALE : UserRequest.Gender.FEMALE
    }).finish();

app.component.ts

If we pass the Uint8Array object to Angular's HTTP client, he serializes it into JSON. But we have to send a binary message to the server. Therefore, we have to extract the underlying buffer (ArrayBuffer) of the Uint8Array object. The application does this with the following code.

    const offset = encodedUserRequest.byteOffset;
    const length = encodedUserRequest.byteLength;
    const userRequestArrayBuffer = encodedUserRequest.buffer.slice(offset, offset + length);

app.component.ts

It's important that you not use encodedUserRequest.buffer, because the underlying buffer can be larger than the Uint8Array object. You have to extract the offset and length of the Uint8Array object and then slice the buffer to get the actual data.

When sending Protocol Buffers messages to the server, the request HTTP headers must be set correctly. The demo application sends two headers Accept and Content-Type. Spring on the server reads these headers and selects the matching message converter. The Accept header describes the response, and Content-Type the request. You can omit the Accept header if your endpoint does not send anything in the body back.

It's also essential that the responseType is set to arraybuffer because we get back a binary message.

    const headers = new HttpHeaders({
      Accept: 'application/x-protobuf',
      'Content-Type': 'application/x-protobuf'
    });
    
    this.httpClient.post(`${environment.SERVER_URL}/register-user`, userRequestArrayBuffer, {headers, responseType: 'arraybuffer'})
      .pipe(
        map(response => this.parseProtobuf(response)),
        catchError(this.handleError)
      ).subscribe(this.handleResponse);
  }

app.component.ts

The response is an ArrayBuffer object. To encode it, I first had to wrap it into a Uint8Array object, and then pass it to the static decode method.

  parseProtobuf(response: ArrayBuffer): IUserResponse {
    return UserResponse.decode(new Uint8Array(response));
  }

  handleResponse(userResponse: IUserResponse) {
    console.log(`ID: ${userResponse.id}`);
    console.log(`Status: ${userResponse.status === UserResponse.Status.OK ? 'OK' : 'NOT_OK'}`);
  }

  handleError(error): Observable<any> {
    console.error(error);
    return throwError(error || 'Server error');
  }

app.component.ts


You see, it's quite easy to exchange Protocol Buffers messages between JavaScript and Spring Boot. Spring Boot takes care of the heavy lifting, we only have to register the message converter, and we have to make sure that the client sends the proper HTTP request headers.

You find the complete source code of the demo application on GitHub:
https://github.com/ralscha/blog2020/tree/master/protobuf-js2