Dynamically loading position data with Ionic and Spring Boot

Published: November 05, 2017  •  Updated: January 31, 2018  •  ionic3, java, spring, javascript

In this blog post we are creating an application that loads location data from a MongoDB database and displays them on a map inside an Ionic application. Instead of sending all the location data at once, the client sends a request to the server with the bounds of the current visible map and the server returns the data points that are located inside this box. Each time the users zooms and moves the map, the client sends a new request to the server with the new bounds of the map. To find the requested locations the server application takes advantage of the built in geospatial query support in MongoDB.

This application is based on two blog posts from Joshua Morony:
Dynamically loading markers with MongoDB: Part1 and Part2.
You should definitely check this website when you are learning Ionic or want to improve your Ionic skills: https://www.joshmorony.com
He wrote a beginners book about Ionic and offers a course about advanced Ionic topics and he posted a lot of free and interesting blog posts about Ionic.

Compared to Joshua's example I made two changes to the application. Instead of a Node.js server we will create a Spring Boot server with Spring 5 and play a little bit with the new reactive programming model. And we will use OpenStreetMap and Leaflet to display the data points instead of Google Maps.

As example dataset I chose the earthquakes data collected by the United States Geological Survey (USGS) agency. This dataset is maybe not the best use case for this architecture, because the data points are static. Each time the user zooms and moves the map the client loads the same data points multiple times. For static data it might be more useful to load everything at once. But this architecture might be interesting for an application that has to display a lot of data points where the location of each item changes very often. For example, you need to track vehicles or other assets that move around a lot.


Server

For the server I wanted to use Spring 5 and play with the new reactive programming model. At the time of writing Spring Boot 2, which supports the Spring 5 framework, was not released yet. According to the project site it will be released in December 2017, but we don't have to wait until then and use the latest milestone release. Go to https://start.spring.io, select the latest Spring Boot 2 milestone, enter a group and artifact name, add the dependencies "Reactive Web" and "Reactive MongoDB" and download the zip file. After the download is finished, extract the file and import the project into your favorite IDE.

The data we will download from the USGS site is stored in the CSV (comma-separated values) format. To parse it we will use the Univocity parser library, therefore we add this dependency to our pom.xml

<dependency>
  <groupId>com.univocity</groupId>
  <artifactId>univocity-parsers</artifactId>
  <version>2.7.2</version>
</dependency>

pom.xml

Next we write the Java code for the MongoDB configuration. We have to tell Spring where our MongoDB instance is running and the name of the database. To do that we create a @Configuration subclass of AbstractReactiveMongoConfiguration and implement the two abstract methods getDatabaseName() and reactiveMongoClient().

@SpringBootApplication
public class Application extends AbstractReactiveMongoConfiguration {

  @Override
  protected String getDatabaseName() {
    return "earthquakes";
  }

  @Override
  public MongoClient reactiveMongoClient() {
    return MongoClients.create();
  }

  public static void main(String[] args) {
    SpringApplication.run(Application.class, args);
  }
}

src/main/java/ch/rasc/dynamicpos/Application.java

MongoClients.create() without a parameter establishes a connection to localhost. When your MongoDB instance is not running on the same computer as the Spring Boot application you have to specify the IP address by passing a connection string to create(). The connection string follows this format: mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database.collection][?options]]
See the API documentation for more information

As usual with MongoDB you don't have to create the database beforehand, the database automatically creates it with the first write request.

Next we create an entity class that holds all the properties for an earthquake. Spring Data uses this class to deserialize and serialize the documents from and to MongoDB. We only store a few selected fields from the USGS file. These are all fields the client application will display in a little popup window on the map.

@Document
public class Earthquake {

  @Id
  private String id;
  private String time;

  @GeoSpatialIndexed(type = GeoSpatialIndexType.GEO_2DSPHERE)
  private GeoJsonPoint location;
  private float depth;
  private float mag;
  private String place;

  public Earthquake() {
    // default constructor
  }

  public Earthquake(Record record) {
    this.id = record.getString("id");
    this.time = record.getString("time");
    this.mag = record.getFloat("mag");
    this.depth = record.getFloat("depth");
    this.place = record.getString("place");
    this.location = new GeoJsonPoint(record.getDouble("longitude"),
        record.getDouble("latitude"));
  }

  // get and set methods
}

src/main/java/ch/rasc/dynamicpos/Earthquake.java

During the application startup Spring Data automatically scans for classes that are annotated with @Document and internally creates a mapping from collection to Java class. Because we want to execute geospatial queries in MongoDB we have to create a special index. Spring Data makes this very convenient by providing the GeoJsonPoint class and the @GeoSpatialIndexed annotation. With this setup Spring Data automatically creates the index in the database during the bootstrap process if it does not already exist.

The default constructor is necessary for Spring Data to deserialize the object when it reads the data from the database. The Earthquake(Record record) constructor is used during the initial import where we convert the CSV record into an entity class and then pass it to Spring Data to save it.

The geo support in MongoDB is not limited to the longitude and latitude coordinate system. You can use any coordinate system, but when you use longitude and latitude, always specify longitude first. The 2d spherical index only recognizes this ordering.

Next we create a Spring Data repository interface and extend the ReactiveSortingRepository interface. This gives our application access to CRUD methods like findAll(), save() and delete().
We add one additional method that returns a collection of earthquakes within a bounding box.

public interface ReactiveEarthquakeRepository
    extends ReactiveSortingRepository<Earthquake, String> {
  Flux<Earthquake> findByLocationWithin(Box box);
}

src/main/java/ch/rasc/dynamicpos/ReactiveEarthquakeRepository.java

Because we use the reactive programming model where all the code has to run in a nonblocking manner, the method returns a stream of Earthquake instances as Flux. Fortunately for us the underlying MongoDB Java driver already supports asynchronous interactions with the database and Spring Data uses this driver to support the reactive programming model.

The Box class is part of the Spring Data library that describes a box spanning from two given points. We don't have to implement the query logic. Spring Data automatically creates the necessary code for this query derived from the method name. The findBy keyword specifies a search query. The Within keyword results in a geoWithin query and the word Locations matches the name of the field in the Earthquake entity class.

The query that Spring Data sends to MongoDB when our code calls the method looks like this.

{
  "location": {
     $geoWithin: {
        $box: [
          [ bottom left longitude, bottom left latitude],
          [ upper right longitude, upper right latitude ]
        ]
     }
  }
}

Next we write the code that downloads and imports the earthquake data. We do this once during the startup of the application. As an enhancement you could add a scheduled job that periodically imports the latest earthquake data. USGS updates the public available files every 15 minute.

public class EarthquakeImporter {

  private final ReactiveEarthquakeRepository repository;
  private final WebClient client;
  private final CsvParser parser;

  EarthquakeImporter(ReactiveEarthquakeRepository repository) {
    this.repository = repository;

    this.client = WebClient.create("https://earthquake.usgs.gov");

    CsvParserSettings settings = new CsvParserSettings();
    settings.setHeaderExtractionEnabled(true);
    settings.setLineSeparatorDetectionEnabled(true);
    this.parser = new CsvParser(settings);
  }

  @PostConstruct
  public void read() {
    String importedData = this.client.get()
        .uri("/earthquakes/feed/v1.0/summary/2.5_month.csv").accept(MediaType.TEXT_PLAIN)
        .retrieve().bodyToMono(String.class).block();

    List<Record> records = this.parser.parseAllRecords(new StringReader(importedData));
    List<Earthquake> earthquakes = records.stream().map(Earthquake::new)
        .collect(Collectors.toList());
    insertData(earthquakes);
  }

  private void insertData(List<Earthquake> earthquakes) {
    this.repository.deleteAll().then(this.repository.saveAll(earthquakes).then())
        .subscribe();
  }
}

src/main/java/ch/rasc/dynamicpos/EarthquakeImporter.java

The importer is a spring managed bean and we inject the ReactiveEarthquakeRepository instance into this class to insert the data into MongoDB. In the constructor we create an instance of the Spring 5 WebClient and the Univocity CsvParser. The earthquake CSV file contains a header on the first line with the field names therefore we set setHeaderExtractionEnabled(true) and Univocity automatically creates keys with the names of the columns. You saw the result of this earlier in the Earthquake constructor where we use the name of the column to extract the data (this.id = record.getString("id");)

To run the import we annotate the method with @PostConstruct and Spring automatically calls the method when the application context is set up.

The importerer first fetches the data with a HTTP GET request, then calls the parserAllRecords method from the CSVParser, converts each record into an Earthquake instance, deletes all the old data in the database with deleteAll() and saves the newly created entity instances with saveAll().

deleteAll() and saveAll() are methods we get for free from the super interface ReactiveSortingRepository of our repository interface.

Last piece of the server is the controller that handles the request from the client and sends back the earthquakes that are located in the requested area.

@RestController
@CrossOrigin
public class EarthquakeController {

  private final ReactiveEarthquakeRepository repository;

  public EarthquakeController(ReactiveEarthquakeRepository repository) {
    this.repository = repository;
  }

  @GetMapping("/earthquakes/{lng1}/{lat1}/{lng2}/{lat2}")
  public Flux<Earthquake> findNear(@PathVariable("lng1") double lng1,
      @PathVariable("lat1") double lat1, @PathVariable("lng2") double lng2,
      @PathVariable("lat2") double lat2) {

    Box box = new Box(new Point(lng1, lat1), new Point(lng2, lat2));
    return this.repository.findByLocationWithin(box);

  }
}

src/main/java/ch/rasc/dynamicpos/EarthquakeController.java

The bean is annotated with @RestController which tells Spring to automatically convert the response of all mapping methods to JSON. The server will run during development on localhost:8080 and the Ionic application runs on localhost:8100 so we have to add the @CrossOrigin annotation to allow requests from different origins (CORS).

Like in the importer bean we inject our Spring Data repository. Then we add the method that handles the GET request. The client sends 4 parameters as path variables. Thanks to the @PathVariable annotation, Spring automatically extracts the values from the path and assigns them to the method parameters. The code then creates a Box instance and calls the findByLocationWithin() method. The first parameter of the Box constructor represents the coordinates of the bottom left corner and the second parameter the upper right corner. And always, when you use the longitude/latitude coordinate system with MongoDB, specify the longitude first.

Like the methods in the data repository the methods in this controller have to run in a nonblocking manner and return either a Mono or a Flux.


Client

When it comes to maps you find a lot of examples that use Google Maps. It's a great service and it's free for most use cases. But Google Maps is not the only map solution on the Internet. In this example we use the map from OpenStreetMap. OpenStreetMap is a community driven map project, where everybody can help collecting data and improving the quality of the map. The data collected by OpenStreetMap is open and free to use for any purpose. You can even run your own map server with it.

Unlike Google Maps OpenStreetMap does not provide an official JavaScript library to display the map in a browser. That's not a problem because there are two popular open source JavaScript libraries we can choose from and are able to read and display the OpenStreetMap data: OpenLayers and Leaflet.
Both libraries are provider agnostic, that means they support different map providers as long as you conform to the terms of use and the server sends the data in a format that these libraries understand.

Which of the two libraries you choose depends on the functionality you need in your application. Leaflet is a lightweight solution (38KB), OpenLayers has more features built in and therefore is bigger in size. Checkout the documentation and example pages of the two project to see what features they support and choose the library that supports the feature you need in your application.

Both libraries are easy to integrate with Angular. For OpenLayers you can use the ngx-openlayers library and for Leaflet ngx-leaflet. These Angular libraries provide directives that make the integration into your application straightforward.

I choose leaflet and ngx-leaflet for this project because the application only needs to display a basic map and draw some circles on it. Leaflet handles this in a very easy and lightweight way.

The client we create in this project is an Ionic app and is based on the blank starter template

ionic start dynamicpos start

Next we add the two mentioned leaflet libraries

npm install @asymmetrik/ngx-leaflet
npm install leaflet

Leaflet needs a few CSS rules to properly display the map. We add a style tag to the src/index.html file to import this leaflet css file.

<link rel="stylesheet" href="assets/leaflet.css">

src/index.html

and then create a custom copy configuration that copies the leaflet.css file from the node_modules folder to the build folder

module.exports = {
 copyLeafletCss: {
   src: ['./node_modules/leaflet/dist/leaflet.css'],
   dest: '{{WWW}}/assets'
 }
}

copy.config.js

And finally add a configuration to package.json to tell the Ionic build script to run this custom copy step.

"config": {
 "ionic_copy": "./copy.config.js"
},

package.json

Importing the css file this way works but it looks a bit strange to me and I'm wondering if there is another (better?) way to import external css files into an Angular application. Send me a feedback when you know other solutions.

Next step is to import the Leaflet and Http client module into our application. Open the file src/app/app.module.ts and add two additional imports. After the change the imports section should look like this.

imports: [
 BrowserModule,
 HttpClientModule,
 LeafletModule.forRoot(),
 IonicModule.forRoot(MyApp),
],

src/app/app.module.ts

Next we create the html template. We use the generated home component and overwrite the code the ionic start command created. Open home.html and overwrite it with with this code.

<ion-header>
 <ion-navbar>
   <ion-title>
     M2.5+ Earthquakes (Past 30 days)
   </ion-title>
 </ion-navbar>
</ion-header>

<ion-content>
 <div id="map"
      leaflet
      (leafletMapReady)="onMapReady($event)"
      [leafletOptions]="options">
 </div>
</ion-content>

src/pages/home/home.html

To make the map work we have to add the leaflet directive to an existing DOM element, style the element with a height and provide an initial zoom factor and center with either the leafletOptions or the more specific leafletZoom and leafletCenter input directives.

In our application we set the height and width of the map to 100% to fill the entire screen

page-home {
 #map {
   height: 100%;
   width: 100%;
 }
}

src/pages/home/home.scss

In the TypeScript code we import a few objects from the leaflet library. Objects starting with a lowercase character are factory methods that create an object. For instance layerGroup creates a LayerGroup.

import {Component} from '@angular/core';
import {HttpClient} from "@angular/common/http";
import {LatLngBounds, LayerGroup, layerGroup, Map, tileLayer, latLng, circleMarker} from "leaflet";
 constructor(private readonly httpClient: HttpClient) {
 }

In the constructor we inject the http client. We need the client for sending requests to our Spring Boot application and request the earthquakes for the displayed map area.

The options instance variable specifies the initial configuration parameters for Leaflet.

 options = {
   layers: [
     tileLayer('http://{s}.tile.osm.org/{z}/{x}/{y}.png', {maxZoom: 18, 
         attribution: '&copy; <a href="https://www.openstreetmap.org/copyright">OpenStreetMap</a> contributors'})
   ],
   zoom: 5,
   center: latLng([46.879966, -121.726909])
 };

The most important configuration is the layer. You have to tell leaflet from where it can download the map data. This is very different to a Google Maps solution because there the JavaScript library and the server are tightly coupled. Leaflet on the other hand supports many map providers. For this example we load the data from the OpenStreetMap (osm) server. It's important that you additionally specify a zoom factor and a center, leaflet throws an exception when one of them is missing.

In the template we added a listener to the leafletMapReady event (leafletMapReady)="onMapReady($event)". Leaflet emits this event as soon the map is loaded and ready for use.

onMapReady(map: Map) {
 this.map = map;
 this.loadEarthquakes(map.getBounds());

 map.on('zoomend', evt => this.loadEarthquakes(evt.target.getBounds()));
 map.on('moveend', evt => this.loadEarthquakes(evt.target.getBounds()));
}

In the TypeScript code we store the reference to the map in an instance variable and call the loadEarthquakes() method with the bounds of the map. And we install two event listeners for the zoom and move end event. Each time the user zooms and moves the map, leaflet emits these events and each time the application needs to load the earthquakes that are visible in this new area.

loadEarthquakes() is responsible for loading the data from the Spring Boot application.

private loadEarthquakes(bounds: LatLngBounds) {
 const southWest = bounds.getSouthWest();
 const northEast = bounds.getNorthEast();
 this.httpClient.get<Earthquake[]>(`http://192.168.178.84:8080/earthquakes/${southWest.lng}/${southWest.lat}/${northEast.lng}/${northEast.lat}`)
   .subscribe(data => this.drawCircles(data));
}

The application sends the bottom left (south west) and upper right (north east) position of the visible map to the server and receives back all the earthquakes that are visible inside this box.

When the data comes back from the server the app executes the drawCircles() method

private drawCircles(earthquakes) {
 if (this.markerGroup) {
   this.markerGroup.removeFrom(this.map);
 }
 this.markerGroup = layerGroup(null).addTo(this.map);
 for (let earthquake of earthquakes) {
   const info = `${earthquake.time}<br>${earthquake.place}<br>${earthquake.mag}`;

   circleMarker([earthquake.location.coordinates[1], earthquake.location.coordinates[0]], {
     color: '#000000',
     fillOpacity: 0.4,
     fillColor: this.getFillColor(earthquake.mag),
     weight: 1,
     radius: earthquake.mag * this.map.getZoom()
   })
     .bindPopup(info)
     .addTo(this.markerGroup);
 }
}

src/pages/home/home.ts

We mark each earthquake with a circle on the map. The radius and color change according to the magnitude of the earthquake. Bigger circles and darker colors mean more powerful earthquakes. Instead of adding each circle individually to the map we add the circles to a marker group and add this group to the map. This simplifies the removing of the circles, we only have to remove the group from the map and consequently remove all the contained markers of this group. Worth mentioning here is that the circleMarker factory method expects the location in the order latitude/ longitude which is different to the order MongoDB expects the location.

We also add a little popup window to each circle. You see this information when you click or tap on the circle. The bindPopup method makes this very easy. The method expects a string that contains html code. The info window in this example displays the time, a description of the place and the magnitude of the earthquake.

Now we are ready to test our application. Make sure that the MongoDB instance is up and running. Start the Spring Boot application, either from inside your IDE or from the command line with mvn spring-boot:run. And finally build the Ionic app and open it in a browser with ionic serve.

If there aren't any errors you should see the earthquakes of the last 30 days. When you move and zoom the map you the see circles dynamically popping in and out depending on the visible area of the map. screenshot

You find the complete source code for the Spring Boot and Ionic application on GitHub:
https://github.com/ralscha/blog/tree/master/dynamicpos