Consume Protocol Buffer messages with Ionic 3

Published: January 14, 2017  •  Updated: December 23, 2017  •  ionic3, spring, java, javascript

In the previous post we created an example that sends Protocol Buffer messages from one Java application to another.

In this post we look at an example that sends Protocol Buffer messages from a Java Spring Boot server to a JavaScript application written in Ionic. To compare Protocol Buffer with JSON the server will also provide an endpoint that returns JSON.

For this example we use real data from The United States Geological Survey agency. They provide lists of current earthquakes in different data formats that can be accessed with simple http get requests. The Spring Boot application fetches this data regularly and provides endpoints for the Javascript application.

The applications will use this Protocol Buffer definition file.

syntax = "proto3";
message Earthquake {
  string id = 1;
  string time = 2;
  double latitude = 3;
  double longitude = 4;
  float depth = 5;
  float mag = 6;
  string place = 7;
  string magType = 8;
}

message Earthquakes {
  repeated Earthquake earthquakes = 1;
}

Every reported earthquake is modeled with an Earthquake message and Earthquakes is a list of zero to n Earthquake messages and is the root of the server response


Server

We start building the Spring Boot application with the Spring Initializr website. We only need the Web dependency for this example. Make sure that you choose Spring Boot 2.
Then we add the protobuf library and the protoc plugin to the pom.xml. pom.xml

Next we copy our proto file to src/main/protobuf/ and generate the Java code with

mvn generate-source

Spring 5 supports Protocol Buffers version 2 and 3 out of the box. All we need to do is creating a bean of type ProtobufHttpMessageConverter. You can put this in any @Configuration class.

  @Bean
  public ProtobufHttpMessageConverter protobufHttpMessageConverter() {
    return new ProtobufHttpMessageConverter();
  }

Next we create a service class that consumes the CSV file from the usgs.gov site, converts it into objects and stores them in an instance variable. For fetching the data it uses the OkHttp library. With the help of the univocity-parsers library the application parses the CSV file and converts it into objects.
EarthquakeDb.java

Then we create the Controller class that provides a JSON and a Protocol Buffer endpoint. The Spring Boot application contains a scheduled task that reads the data from the USGS site every hour. The /refresh endpoint allows us to force an update, so we don't have to wait an hour.

Because both endpoints handle the same url (/earthquakes), we need to make them distinguishable. In this example I added the produces attribute to the Protocol Buffer endpoint. Spring takes the http request header Accept of incoming requests and compares it with the value of the produces attribute, and if it matches Spring calls that method. Another approach would be to use different names for the urls.
Another useful tool is the JsonFormat class from the protobuf-java-util package. This class provides methods to convert and parse Protobuf objects into and from JSON.
EarthquakeController.java

If you want that Spring Boot compresses the http responses, you need to add the protobuf mime type to the list of compressible types.

server.compression.enabled=true
server.compression.mime-types=application/json,application/x-protobuf

application.properties


Client

For the Ionic client we start with the tabs starter template and add the ProtoBuf.js library to the project.

ionic start protobuf tabs
npm install protobufjs

Next we generate JavaScript and TypeScript code from our proto file. ProtoBuf.js provides a command line tool to create these classes. For that we add a new task to our package.json file

"pbts": "pbjs -t static-module ../server/src/main/protobuf/Earthquake.proto -o src/protos/earthquake.js && pbts --no-comments src/protos/earthquake.js -o src/protos/earthquake.d.ts"

package.json Now we can call the task with npm run pbts and it will create two files in the src/protos/ folder of our Ionic app (Make sure that you create the folder first). The earthquake.js is the implementation and contains methods to serialize and deserialize binary protobuf messages into objects. earthquake.d.ts is a TypeScript definition file. For each protobuf object it creates an interface and a class (IEarthquake and Earthquake, IEarthquakes and Earthquakes).

Next we create a provider which contains methods that call the endpoints of our server.

ionic g provider earthquake
@Injectable()
export class EarthquakeProvider {
  constructor(private readonly http: HttpClient) {
  }

  refresh(): Observable<void> {
    return this.http.get<void>(`${ENV.SERVER_URL}/refresh`);
  }

  fetchJson(): Observable<IEarthquake[]> {
    return this.http.get<any>(`${ENV.SERVER_URL}/earthquakes`)
      .pipe(map(res => res.earthquakes), catchError(e => this.handleError(e)));
  }

  fetchProtobuf(): Observable<IEarthquake[]> {
    const headers = new HttpHeaders({'Accept': 'application/x-protobuf'});
    return this.http.get(`${ENV.SERVER_URL}/earthquakes`, {headers, responseType: 'arraybuffer'})
      .pipe(map(res => this.parseProtobuf(res)),
        catchError(this.handleError)
      );
  }

  parseProtobuf(response: ArrayBuffer): IEarthquake[] {
    console.time('decodeprotobuf');
    const earthquakes = Earthquakes.decode(new Uint8Array(response))
    console.timeEnd('decodeprotobuf');
    return earthquakes.earthquakes;
  }

  handleError(error): Observable<any> {
    console.error(error);
    return throwError(error || 'Server error');
  }

}

src/providers/earthquake/earthquake.ts

The refresh() method calls the /refresh endpoint and triggers a refresh of the data stored on the server.
The fetchJson() method requests the data in the JSON format.
The fetchProtobuf() function fetches the data in the binary Protocol Buffers format. We need to specify the Accept header with the value application/x-protobuf that matches the produces attribute in the Spring Boot controller. Because we know that the server returns a binary message and we need that message in the form of an ArrayBuffer we add an additional parameter to the get call {responseType: 'arraybuffer'}. When the response comes back from the server the method maps it to objects with the decode method from the generated Earthquakes class.

Because in our app we use two tabs that show the same content, we create a component to reuse some code. This component represents one row in the item list and shows the detail for one earthquake.

ionic g component detail
<div class="row">
 <h1>
   {{earthquake.mag | number:'1.1-1'}}
 </h1>
 <div class="place">
   <h2>{{earthquake.place}}</h2>
   <ion-note>{{earthquake.time | date:'MMM d HH:mm'}}</ion-note>
 </div>
 <div class="depth">{{earthquake.depth | number}} km</div>
</div>

src/components/detail/detail.html

import {Component, Input} from '@angular/core';
import {IEarthquake} from "../../protos/earthquake";

@Component({
  selector: 'detail',
  templateUrl: 'detail.html'
})
export class DetailComponent {

  @Input()
  earthquake: IEarthquake;

}

src/components/detail/detail.ts

Then we create two pages. One shows the earthquakes read from the JSON endpoint and the other tab shows the same data but read from the Protocol Buffer message endpoint.

ionic g page json
ionic g page protobuf
<ion-header>
  <ion-navbar>
    <ion-title>JSON</ion-title>
  </ion-navbar>
</ion-header>

<ion-content padding>
  <ion-refresher (ionRefresh)="doRefresh($event)">
    <ion-refresher-content></ion-refresher-content>
  </ion-refresher>

  <ion-list>
    <ion-item *ngFor="let earthquake of earthquakes">
      <detail [earthquake]="earthquake"></detail>
    </ion-item>
  </ion-list>
</ion-content>

src/pages/json/json.html

import {Component} from '@angular/core';
import {EarthquakeProvider} from "../../providers/earthquake/earthquake";
import {IEarthquake} from "../../protos/earthquake";

@Component({
  selector: 'page-json',
  templateUrl: 'json.html'
})
export class JsonPage {
  earthquakes: IEarthquake[];

  constructor(private readonly earthquakeService: EarthquakeProvider) {
  }

  doRefresh(refresher) {
    this.earthquakeService.refresh().subscribe(() => {
      refresher.complete();
      this.ionViewDidLoad();
    });
  }

  ionViewDidLoad() {
    this.earthquakeService.fetchJson().subscribe(data => this.earthquakes = data);
  }

}

src/pages/json/json.ts

The EarthquakeProvider, we created earlier, is injected into the page class and in the Ionic lifecycle method ionViewDidLoad() the fetchJson function is called to request the data from the server. We use the generated interface IEarthquake as type for the result object.

The code for the protobuf page is almost the same. The only difference is that it calls fetchProtobuf() instead of fetchJson().
src/pages/protobuf/protobuf.html
src/pages/protobuf/protobuf.ts

Finally we have to wire everything together. In the tabs.html and tabs.ts files we add the two pages to the tab.

<ion-tabs>
 <ion-tab [root]="tab1Root" tabTitle="JSON" tabIcon="ios-list-outline"></ion-tab>
 <ion-tab [root]="tab2Root" tabTitle="Protobuf" tabIcon="ios-list-outline"></ion-tab>
</ion-tabs>

src/pages/tabs/tabs.html

@Component({
 templateUrl: 'tabs.html'
})
export class TabsPage {
 tab1Root: any = JsonPage;
 tab2Root: any = ProtobufPage;
}

src/pages/tabs/tabs.ts

And in the app.modules.ts we have to add the component to the declarations and the pages to the declarations and entryComponents section.

@NgModule({
  declarations: [
    MyApp,
    JsonPage,
    ProtobufPage,
    TabsPage,
    DetailComponent
  ],
  imports: [
    BrowserModule,
    HttpClientModule,
    IonicModule.forRoot(MyApp)
  ],
  bootstrap: [IonicApp],
  entryComponents: [
    MyApp,
    JsonPage,
    ProtobufPage,
    TabsPage
  ],
  providers: [
    StatusBar,
    SplashScreen,
    {provide: ErrorHandler, useClass: IonicErrorHandler},
    EarthquakeProvider
  ]
})
export class AppModule {
}

src/app/app.module.ts

To test the application you have to first start the server. Either start the class Application class in your IDE or start the server with the maven spring boot plugin on the commandline with

./mvnw spring-boot run

Then start ionic serve and the browser should automatically open with the url http://localhost:8100. If everything was successful, you should see a list of earthquakes that happened during the last 24 hours.

You find the complete project on GitHub.