In the previous post, we created an example that sends Protocol Buffer messages from one Java application to another.
In this post, we look at an example that sends Protocol Buffer messages from a Java Spring Boot server to a JavaScript application written in Ionic. To compare Protocol Buffer with JSON, the server also provides an endpoint that returns JSON.
For this example, we use real data from The United States Geological Survey agency. They provide lists of current earthquakes in different data formats that can be accessed with simple HTTP get requests. The Spring Boot application fetches this data and provides endpoints for the JavaScript application.
The application is going to use this Protocol Buffer definition file.
syntax = "proto3";
option java_package = "ch.rasc.protobuf";
message Earthquake {
string id = 1;
string time = 2;
double latitude = 3;
double longitude = 4;
float depth = 5;
float mag = 6;
string place = 7;
string magType = 8;
}
message Earthquakes {
repeated Earthquake earthquakes = 1;
}
Every reported earthquake is modeled with an Earthquake message, and Earthquakes is a list of zero to n Earthquake messages and represents the root of the server response.
Server ¶
We start building the Spring Boot application with the Spring Initializr website. We only need Web dependency for this example.
Then we add the Protobuf library and the protoc plugin to the pom.xml. The protobuf-java-util library contains support for converting a Protocol Buffer message to JSON. This is very convenient for our demo application, so we can use the same generated data class for the JSON and Protobuf endpoint.
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>3.24.4</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java-util</artifactId>
<version>3.24.4</version>
</dependency>
<goals>
<goal>run</goal>
</goals>
<configuration>
<protocVersion>3.24.4</protocVersion>
<inputDirectories>
<include>src/main/protobuf</include>
</inputDirectories>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Next, we copy our proto file to src/main/protobuf/ and generate the Java code with
mvn generate-source
Spring 5 supports Protocol Buffers version 2 and 3 out of the box. All we need to do is creating a bean of type ProtobufHttpMessageConverter.
You can put this in any @Configuration class.
@Bean
public ProtobufHttpMessageConverter protobufHttpMessageConverter() {
return new ProtobufHttpMessageConverter();
}
Next, we create a service class that fetches the CSV file from the usgs.gov site, converts it into objects, and stores them in an instance variable. For fetching the data, it uses the OkHttp library. With the help of the univocity-parsers library, the application parses the CSV file and converts it into objects.
Request request = new Request.Builder()
.url("https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv")
.build();
try (Response response = this.httpClient.newCall(request).execute();
ResponseBody responseBody = response.body()) {
if (responseBody != null) {
String rawData = responseBody.string();
CsvParserSettings settings = new CsvParserSettings();
settings.setHeaderExtractionEnabled(true);
settings.setLineSeparatorDetectionEnabled(true);
CsvParser parser = new CsvParser(settings);
List<Record> records = parser.parseAllRecords(new StringReader(rawData));
Earthquakes.Builder earthquakesBuilder = Earthquakes.newBuilder();
for (Record record : records) {
Earthquake.Builder builder = Earthquake.newBuilder()
.setId(record.getString("id")).setTime(record.getString("time"))
.setLatitude(record.getDouble("latitude"))
.setLongitude(record.getDouble("longitude"))
.setDepth(record.getFloat("depth")).setPlace(record.getString("place"));
Float mag = record.getFloat("mag");
if (mag != null) {
builder.setMag(mag);
}
String magType = record.getString("magType");
if (magType != null) {
builder.setMagType(magType);
}
earthquakesBuilder.addEarthquakes(builder);
}
this.earthquakes = earthquakesBuilder.build();
}
}
Then we create the Controller class that provides a JSON and a Protocol Buffer endpoint. The Spring Boot application contains a scheduled task that reads the data from the USGS site every hour. The /refresh endpoint allows us to force an update, so we don't have to wait an hour.
Because both endpoints handle the same URL (/earthquakes), we need to make them distinguishable. For this example, I added the produces attribute to the Protocol Buffer endpoint. Spring takes the Accept HTTP header of incoming requests and compares it with the value of the produces attribute, and if it matches calls that method. Another approach would be to use different pathnames.
With the help of the JsonFormat class from the protobuf-java-util package, we convert the Protobuf object into JSON.
@GetMapping("/earthquakes")
public String getEarthquakesJson() throws InvalidProtocolBufferException {
return JsonFormat.printer().print(this.earthquakeDb.getEarthquakes());
}
@GetMapping(path = "/earthquakes", produces = "application/x-protobuf")
public Earthquakes getEarthquakesProtobuf() {
return this.earthquakeDb.getEarthquakes();
}
@GetMapping(path = "/refresh")
@ResponseStatus(HttpStatus.NO_CONTENT)
public void refresh() throws IOException {
this.earthquakeDb.readEarthquakeData();
}
We also add the Protobuf mime type to the compression mime types configuration. Spring Boot compresses all HTTP responses that match with one of these mime types on the fly with gzip before sending them back to the client.
server.compression.enabled=true
server.compression.mime-types=application/json,application/x-protobuf
Client ¶
The client is based on the tabs Ionic starter application.
For consuming Protobuf messages, we add the ProtoBuf.js library to the project.
npm install protobufjs
Next, we generate JavaScript and TypeScript code from our proto file. ProtoBuf.js provides a command line tool to create these classes.
For that, we add a new task to the package.json file.
},
Now we can call the task with npm run pbts, and it is creating two files in the src/app/protos/ folder (Make sure that you create the folder first).
The earthquake.js is the implementation and contains methods to serialize and deserialize binary protobuf messages into objects. earthquake.d.ts is a TypeScript definition file. For each Protobuf object, it creates an interface and a class (IEarthquake and Earthquake, IEarthquakes, and Earthquakes).
Next, we create a service that contains methods that call the endpoints of our server.
import {Injectable} from '@angular/core';
import {Observable, throwError} from 'rxjs';
import {catchError, map} from 'rxjs/operators';
import {HttpClient, HttpHeaders} from '@angular/common/http';
import {Earthquakes, IEarthquake} from './protos/earthquake';
import {environment} from '../environments/environment';
@Injectable({
providedIn: 'root'
})
export class EarthquakeService {
constructor(private readonly http: HttpClient) {
}
refresh(): Observable<void> {
return this.http.get<void>(`${environment.SERVER_URL}/refresh`);
}
fetchJson(): Observable<IEarthquake[]> {
return this.http.get<{ earthquakes: IEarthquake[] }>(`${environment.SERVER_URL}/earthquakes`)
.pipe(map(res => res.earthquakes), catchError(e => this.handleError(e)));
}
fetchProtobuf(): Observable<IEarthquake[]> {
const headers = new HttpHeaders({Accept: 'application/x-protobuf'});
return this.http.get(`${environment.SERVER_URL}/earthquakes`, {headers, responseType: 'arraybuffer'})
.pipe(map(res => this.parseProtobuf(res)),
catchError(this.handleError)
);
}
parseProtobuf(response: ArrayBuffer): IEarthquake[] {
// eslint-disable-next-line no-console
console.time('decodeprotobuf');
const earthquakes = Earthquakes.decode(new Uint8Array(response));
// eslint-disable-next-line no-console
console.timeEnd('decodeprotobuf');
return earthquakes.earthquakes;
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
handleError(error: any): Observable<any> {
console.error(error);
return throwError(error || 'Server error');
}
}
The refresh() method calls the /refresh endpoint and triggers a refresh of the data stored on the server.
The fetchJson() method requests the data in the JSON format.
The fetchProtobuf() function fetches the data in the binary Protocol Buffers format. We need to specify the Accept header with the value application/x-protobuf that matches the produces attribute in the Spring Boot controller. Because we know that the server returns a binary message, and we need that message in the form of an ArrayBuffer we add a parameter to the GET call {responseType: 'arraybuffer'}. When the response comes back from the server, the method maps it to objects with the decode method from the generated Earthquakes class.
Because in our app, we use two tabs that show the same content, we create a component to reuse some code. This component represents one row in the item list and shows the detail for one earthquake.
<div class="row">
<h1>
{{earthquake.mag | number:'1.1-1'}}
</h1>
<div class="place">
<h2>{{earthquake.place}}</h2>
<ion-note>{{earthquake.time | date:'MMM d HH:mm'}}</ion-note>
</div>
<div class="depth">{{earthquake.depth | number}} km</div>
</div>
import {Component, Input} from '@angular/core';
import {IEarthquake} from '../protos/earthquake';
@Component({
selector: 'app-detail',
templateUrl: './detail.component.html',
styleUrls: ['./detail.component.scss']
})
export class DetailComponent {
@Input()
earthquake!: IEarthquake;
}
Then we create two pages. One shows the earthquakes read from the JSON endpoint, and the other tab shows the same data but read from the Protocol Buffer message endpoint.
<ion-header>
<ion-toolbar>
<ion-title>JSON</ion-title>
</ion-toolbar>
</ion-header>
<ion-content>
<ion-refresher (ionRefresh)="doRefresh($event)" slot="fixed">
<ion-refresher-content></ion-refresher-content>
</ion-refresher>
<ion-list>
<ion-item *ngFor="let earthquake of earthquakes">
<ion-label>
<app-detail [earthquake]="earthquake"></app-detail>
</ion-label>
</ion-item>
</ion-list>
</ion-content>
import {Component, OnInit} from '@angular/core';
import {EarthquakeService} from '../earthquake.service';
import {IEarthquake} from '../protos/earthquake';
@Component({
selector: 'app-json',
templateUrl: './json.page.html',
styleUrls: ['./json.page.scss']
})
export class JsonPage implements OnInit {
earthquakes: IEarthquake[] = [];
constructor(private readonly earthquakeService: EarthquakeService) {
}
doRefresh(event: Event): void {
this.earthquakeService.refresh().subscribe(() => {
this.earthquakeService
.fetchJson()
.subscribe(data => {
this.earthquakes = data;
(event as CustomEvent).detail.complete();
});
});
}
ngOnInit(): void {
this.earthquakeService.fetchJson().subscribe(data => this.earthquakes = data);
}
}
The template and code for the Protobuf page look very similar. Notable difference is the call to this.earthquakeService.fetchProtobuf() instead of fetchJson().
To test the application, you have to start the server first. Either start the class Application class in your IDE or start the server with the maven Spring Boot plugin from the shell or command prompt with
//Linux and macOS
./mvnw spring-boot:run
// Windows
.\mvnw.cmd spring-boot:run
Then start the client with ionic serve. The browser should automatically open with the URL http://localhost:8100. If everything was successful, you should see a list of earthquakes that happened during the last 24 hours.
You find the complete project on GitHub.
The application in this tutorial only sends messages from the server to the client. If in your application you need to send Protocol Buffers messages from a JavaScript client to a Java server check out my blog post about this topic.