OpenMapTiles, a self-hosted map tile server with OpenStreetMap data

Published: July 10, 2018  •  linux, selfhosted

When you want to add interactive maps to your web application you need some JavaScript and a server that hosts the map data. A popular solution is Google Maps, where you get an all in one package, a JavaScript library and the map data from Google.

Google Maps is not the only map provider. You can choose others, like Apple Maps and Bing Maps.

And then there are providers that host maps generated with data from the OpenStreetMap project. The OpenStreetMap data is public available and everyone can use it. On this wiki page you find a list of commercial companies that provide services around OpenStreetMap data:
https://wiki.openstreetmap.org/wiki/Commercial_OSM_Software_and_Services

The server that sends out the map data to a web application is called a tile server. A tile server either serves vector tiles or pre-renderer tiles in PNG format or is able to support both.

OpenStreetMap itself also runs a tile server (tile.openstreetmap.org) that is used for the map you find on the homepage: https://www.openstreetmap.org
You can use this tile server in your application but you have to comply with the usage policy: https://operations.osmfoundation.org/policies/tiles/

Better solution is to use one of the commercial providers mentioned above or host your own tile server. Fortunately it is not too complicated to do that. All you need is a server in your intranet or a server connected to the Internet and a lot of disk space. If you run your own tile server you can either host the whole world (needs about 51 GB for the base map) or just a region or country.

In this tutorial I will install a tile server on a VPS connected to the Internet. The server I use for this tutorial is a VPS running a plain Ubuntu 18.04 with just an SSH server installed. I configured the server according to my tutorial about setting up a VPS.

As software, we will install tileserver-gl, an open source tile server developed and maintained by Klokan Technologies GmbH a Swiss based company. TileServer GL is written in JavaScript and Node.js. The source code is hosted on GitHub: https://github.com/klokantech/tileserver-gl

The server comes in two flavours, TileServer GL serves vector and raster tiles and TileServer GL Light without the rasterization component is only able to serve vector tiles. The server side rasterization depends on native code that might not run on any platform. TileServer GL Light is pure JavaScript and runs on any Node.js supported platform.

The difference between raster tiles and vector tiles is the location where the map is drawn. With vector data the server just sends the data to the client and the client "draws" the map. Yet another benefit is that the data transfer is greatly reduced, because vector data is much smaller than rendered images. It is also easy to provide interactivity with map features, an user can zoom and rotate the map very easily.
Raster tiles are rendered on the server and then transferred to the client as images, usually in the PNG format. This can be beneficially for mobile devices with slow CPU/GPU, where rendering data on the client is slow. The drawback is that the rasterization requires more processing power on the server.


Data

OpenStreetMap data is free to use and you can download it from different locations. The data is quite big, make sure that you have enough space on the disk. My VPS only has a drive with 20 GB capacity so I can't download the map data for the whole earth, instead I will download only the data for Andorra, a small European country located between Spain and France.

The data you download from the OpenStreetMap project is stored in a format that the TileServer GL cannot read. It needs the map data stored in a mbtiles file. MBTiles is a specification for storing arbitrary tiled map data in SQLite databases.

The conversion from the OpenStreetMap raw data into a mbtiles file is a time consuming process, fortunately we don't have to do that ourselves and we can download weekly updated mbtiles files from https://openmaptiles.com/

Create an account (it's free) and go to the download section. There you can search for a region and then download the OpenStreetMap vector tiles.
These files are free for open-source and open-data project website, non-commercial personal project and evaluation and education purpose. In all other cases you have to pay. Visit the pricing page for more information: https://openmaptiles.com/production-package/.

You can either download the file in the browser and then transfer it your server or download the file directly on the server. You find a wget command on the download page that you can issue on the server.

sudo mkdir /opt/maps
cd /opt/maps
sudo wget -c https://openmaptiles.os.zhdk......._europe_andorra.mbtiles

In the last section of this blog post I show you how to create an mbtiles file on your own computer.


DNS

I create a subdomain for this server: maps.ralscha.ch
This is optional, you can connect to the server with the IP address, but if you want to secure the connection with a TLS certificate you need a domain name. And the name is easier to remember than an IP address.

In the web console of my DNS provider I insert an A and AAAA record

maps.ralscha.ch.        86400   IN      A       51.38.124.133
maps.ralscha.ch.        86400   IN      AAAA    2001:41d0:701:1100::e54

If you are planning to install a TLS certificate I recommend to add a CAA record too. Either for the whole domain if you get all your certificates from one CA or just for the subdomain if you use multiple certificate authorities

ralscha.ch.             86400   IN      CAA     0 issue "letsencrypt.org"
// OR
maps.ralscha.ch.        86400   IN      CAA     0 issue "letsencrypt.org"

Installing

In this section we install the tile server. First we make sure that the current installed packages are all up to date

sudo apt update
sudo apt dist-upgrade

Then we install Node.js. Check the documentation to see the latest installation instructions: https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs

Install the tileserver-gl package with npm

sudo npm install --unsafe-perm=true -g tileserver-gl-light

On Ubuntu 18.04 systemd is responsible for starting and stopping services whenever you reboot the server. systemd needs service files that describe the service that it has to manage.

sudo nano /lib/systemd/system/tileserver-gl-light.service

Paste the following code into the editor, save (ctrl+o) and close (ctrl+x) it.

[Unit]
Description=TileServer GL Light Service
After=network.target

[Service]
Type=simple
User=www-data
ExecStart=/usr/bin/tileserver-gl-light -p 10001
WorkingDirectory=/opt/maps
Restart=on-failure

[Install]
WantedBy=multi-user.target

The service will run under the same user (www-data) as Nginx, it will look for the map data in the /opt/maps directory and it will listen on port 10001. The service listens by default on port 8080, if that is better for your environment you can omit the -p 10001 option.

Make systemd aware of the new configuration file and enable the service, so it will automatically start whenever the server reboots.

sudo systemctl daemon-reload
sudo systemctl enable tileserver-gl-light

Start the service and check the status. If everything is okay you should see a green active (running) text

sudo systemctl start tileserver-gl-light
sudo systemctl status tileserver-gl-light

Nginx

You could now open port 10001 in your firewall and then connect to the TileServer GL directly, but in this installation I want to encrypt the traffic with TLS. For that I install the Nginx HTTP server. Nginx will listen on port 443 and internally redirects all traffic from maps.ralscha.ch to the TileServer GL instance.

Install nginx with apt

sudo apt install nginx

Open ports 80 and 443 in the firewall.

sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

Open a browser and enter the URL http://maps.ralscha.ch. You should see the Nginx welcome page.

Now we create a configuration for the subdomain.

sudo nano /etc/nginx/sites-available/maps

Add the following configuration. This configuration forwards all request to the TileServer GL Light instance.

server {
    listen 80;
    listen [::]:80;
    server_name maps.ralscha.ch;

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Server $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://localhost:10001;
    }
}

Enable the configuration, remove the default Nginx configuration and reload Nginx

sudo ln -s /etc/nginx/sites-available/maps /etc/nginx/sites-enabled/maps
sudo rm /etc/nginx/sites-enabled/default
sudo systemctl reload nginx

When you open the URL http://maps.ralscha.ch/ again you should now see the welcome page of the TileServer GL.


TLS

Next we install the Let's Encrypt client certbot. This program is responsible for creating and renewing certificates. Certificates from Let's Encrypt are free but they are only valid 90 days. certbot installs a job that runs periodically and checks for outdated certificates and automatically renews them.

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository ppa:certbot/certbot

Press Enter

sudo apt update
sudo apt install python-certbot-nginx 

Press y to install the package.

Request a TLS certificate with the following command

sudo certbot --nginx

Certbot first asks for an email address. This is used for urgent renewal and security notices.

Then you have to agree to the terms of services. After that the script asks if it can share your email address with the Electronic Frontier Foundation. Enter n if you don't want to share.

Certbot next lists all installed subdomains in Nginx. In our case there should be only one entry. Enter the number 1 to select this domain.

Select "redirect all traffic to https" (enter 2).

Check if everything works by opening the URL https://maps.ralscha.ch/.
A request to http:// should automatically be redirect to https://


Access map in a web application

We can now start developing our web applications and use the map data from our tile server. Because this tile server can only serve vector data we need a JavaScript library that is able to process this data and render the map in the browser.

For this example I use Mapbox GL JS library. Visit https://www.mapbox.com/mapbox-gl-js/api/ for the documentation. The source code of the library is hosted on GitHub.

Here a simple HTML page that displays the map of Andorra. Change the center option accordingly if you have downloaded a different region of the world.

<!DOCTYPE html>
<html>
<head>
    <meta charset='utf-8' />
    <title>Map Example</title>
    <meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />
    <script src='https://api.tiles.mapbox.com/mapbox-gl-js/v0.46.0/mapbox-gl.js'></script>
    <link href='https://api.tiles.mapbox.com/mapbox-gl-js/v0.46.0/mapbox-gl.css' rel='stylesheet' />
    <style>
        body { margin:0; padding:0; }
        #map { position:absolute; top:0; bottom:0; width:100%; }
    </style>
</head>
<body>

<div id='map'></div>
<script>
  var map = new mapboxgl.Map({
    container: 'map', 
    //style: 'https://maps.ralscha.ch/styles/klokantech-basic/style.json',
	style: 'https://maps.ralscha.ch/styles/osm-bright/style.json',
    center: [1.5813, 42.5341], 
    zoom: 15.65 
  });
  map.addControl(new mapboxgl.NavigationControl());
</script>

</body>
</html>

You don't need to host this file on a server, you can create it locally and open it in a browser.


Restrict access

Our tile server is working and we are able to embed the map into our web application, but we have the problem, because this server is public accessible, that everybody can embed the map into their application. The reason for that is TileServer GL sends by default a CORS response header access-control-allow-origin: * This allows everybody to request data from this server.

To fix that we first disable CORS handling in the TileServer GL and then handle CORS in Nginx where we have more control about the headers.

Open the systemd service file and disable CORS

sudo nano /lib/systemd/system/tileserver-gl-light.service

Add the argument --no-cors to the start command

ExecStart=/usr/bin/tileserver-gl-light -p 10001 --no-cors

Reload the change and restart the service

sudo systemctl daemon-reload
sudo systemctl restart tileserver-gl-light

If you now open your example file from the previous section you no longer see the map and there are error messages in the console

No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin ... is therefore not allowed access.

To give certain web sites access to our tile server open the Nginx configuration

sudo nano /etc/nginx/sites-available/maps

If you only want to give access to exactly one domain you can add an if statement like this

    location / {
        if ($http_origin = https://myapp.ralscha.ch) {
          add_header "Access-Control-Allow-Origin" "$http_origin";
        }
        
        ...

        proxy_pass http://localhost:10001;
    }

You can also compare the origin with a regular expression. With this configuration every application hosted on *.ralscha.ch has access to the tile server

if ($http_origin ~* (^https?://([^/]+\.)*ralscha\.ch$)) {
    add_header "Access-Control-Allow-Origin" "$http_origin";
}

If you need access to the tiles from your development environment you should add a rule for localhost and/or 127.0.0.1

        if ($http_origin ~* (^https?://([^/]+\.)*localhost(\:\d+)*$)) {
          add_header "Access-Control-Allow-Origin" "$http_origin";
        }
        if ($http_origin ~* (^https?://([^/]+\.)*127.0.0.1(\:\d+)*$)) {
          add_header "Access-Control-Allow-Origin" "$http_origin";
        }

Don't forget to reload the Nginx configuration after you changed the configuration

sudo systemctl reload nginx

Visit this website for more information about CORS configuration in Nginx:
https://qa.lsproc.com/post/access-control-allow-origin-multiple-origin-domains


TileServer GL

So far we used the light version of TileServer GL which is only capable of serving vector data. If you need raster tiles you have to install the tileserver-gl package. Unfortunately the installation fails in my environment with an error when npm tries to download the mapbox-gl native package. But fortunately the developers of the TileServer GL provide a docker image where everything is installed.

To run a docker image on our Ubuntu 18.04 VPS we need to install docker. Create a new script sudo nano dockerinstall and past the following code

#!/usr/bin/env bash

sudo apt update

sudo apt --yes install \
    software-properties-common \
    apt-transport-https \
    ca-certificates \
    curl

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

sudo apt update

sudo apt --yes install docker-ce

sudo usermod --append --groups docker $USER

Make it executable and start the script

chmod 700 dockerinstall
./dockerinstall

Now we can start the TileServer GL docker image. The following command maps the local directory /opt/maps with the mbtiles file to the /data directory inside the container. TileServer GL listens on port 80 inside the container and we map it here to the local port 10001.

sudo docker run -d --restart unless-stopped -v /opt/maps:/data -p 10001:80 klokantech/tileserver-gl

Docker is automatically started with systemd when the server reboots and the --restart option automatically starts the image.

When you followed this tutorial so far, TileServer GL Light is still running. Stop it and disable it before running the docker image. Both services listen on the same port 10001.

sudo systemctl stop tileserver-gl-light
sudo systemctl disable tileserver-gl-light

When you open the homepage https://maps.ralscha.ch/ you should now see vector and raster tiles links. You can access the vector data as with the light version with Mapbox GL JS, but for the raster data you need another library.

A popular library that can handles raster tiles is Leaflet. Here a simple example

<!DOCTYPE html>
<html>
<head>
    <meta charset='utf-8' />
    <title>Map Example</title>
    <meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no' />
    <link rel="stylesheet" href="https://unpkg.com/leaflet@1.3.1/dist/leaflet.css"
          integrity="sha512-Rksm5RenBEKSKFjgI3a41vrjkw4EVPlJ3+OiI65vTjIdo9brlAacEuKOiQ5OFh7cOI1bkDwLqdLw3Zg0cRJAAQ=="
          crossorigin=""/>
    <script src="https://unpkg.com/leaflet@1.3.1/dist/leaflet.js"
          integrity="sha512-/Nsx9X4HebavoBvEBuyp3I7od5tA0UzAxs+j83KgC8PU0kgB4XiK4Lfe4y4cgBtaRJQEIFCW+oC506aPT2L1zw=="
          crossorigin=""></script>   
    <style>
        body { margin:0; padding:0; }
        #map { position:absolute; top:0; bottom:0; width:100%; }
    </style>
</head>
<body>

<div id="map"></div>
<script>
  var myMap = L.map('map').setView([42.5341,1.5813], 15);
  L.tileLayer('https://maps.ralscha.ch/styles/klokantech-basic/{z}/{x}/{y}.png', {
    maxZoom: 15
  }).addTo(myMap);
</script>

</body>
</html>

Convert OpenStreetMap data

The data you can download from OpenStreetMap is stored in a raw format that the TileServer GL cannot read directly, we have to convert if first. In the previous steps we downloaded the converted data from https://openmaptiles.com/, but you can convert the data on your own computer if you want.

The developer behind the TileServer GL released all the tools necessary to convert the data on GitHub.

Here an example how you can convert OpenStreetMap data of Andorrau.

sudo git clone https://github.com/openmaptiles/openmaptiles.git
cd openmaptiles
sudo docker run -v $(pwd):/tileset openmaptiles/openmaptiles-tools make
sudo ./quickstart.sh andorra

The script runs for a few minutes, depending on the size of the map and the performance of the computer this can take a long time. After the script ended successfully you find the map data here: data/tiles.mbtiles.

By default the script creates a mbtiles file with data up to zoom level 7. If you need higher zoom levels you have to modify the .env file.

QUICKSTART_MIN_ZOOM=0
QUICKSTART_MAX_ZOOM=7

and then run ./quickstart.sh again. There is a known problem. If you convert the same area the script will ignore the .env settings. You need to change the zoom level in the file ./data/docker-compose-config.yml directly. This file will be regenerated when you convert a different region.

Note that higher zoom levels take more time to convert. Max zoom level is 14.


Links

TileServer GL documentation:
http://tileserver.readthedocs.io/en/latest/index.html

OpenMapTiles Homepage:
https://openmaptiles.org/

Klokan Technologies, the company behind OpenMapTiles and TileServer GL:
https://www.klokantech.com/