Using Dexie.js in a TypeScript application

Published: January 12, 2018  •  database, javascript

In the previous blog post I wrote about Dexie. An IndexedDB wrapper library, that simplifies the IndexedDB programming.

In this blog post I show you a complete application and how to integrate Dexie into a TypeScript (Ionic) application.

The application is a clone of the application I wrote for this blog post about IndexedDB. It downloads a list of earthquakes, that happened in the last month, from the USGS Earthquake Hazard Program website, puts the records into an IndexedDB database and displays the data in a list. The user can filter the data on various criterias like magnitude and time.


Dexie with TypeScript

When you are trying to add Dexie, the way I showed you in the previous blog post, with code like this you will encounter an error.

const db = new Dexie("MyDb");
db.version(1).stores({
  earthquakes: 'id'
});
const allObjects = await db.earthquakes.toArray();

In vanilla JavaScript you get implicit properties for each object store you define in stores(). But since they are defined at runtime TypeScript does not recognizes them and this code does not compile.

A workaround is the table() method.

const allObjects = await db.table('earthquakes').toArray();

But you don't get any code completion and type safety a reason why you use TypeScript in the first place.


Therefore the recommended way is to create a TypeScript class. Before we do that we define an interface for our entity we want to store in the database.

export interface Earthquake {
 id: string;
 time: number;
 place: string;
 mag: number;
 depth: number;
 distance?: number;
 latLng: [number, number];
}

src/providers/earthquakeDb.ts

Now we create the special Dexie database class. It has to be a subclass of Dexie.
In the constructor you have to call the constructor of the super class with the name of the database as parameter.

import Dexie from "dexie";
export class EarthquakeDb extends Dexie {
 earthquakes: Dexie.Table<Earthquake, string>;
 constructor() {
   super("Earthquake");
   this.version(1).stores({
     earthquakes: 'id,mag,time'
   });
 }
}

src/providers/earthquakeDb.ts

The class contains an instance variable for each object store. We only have one object store with name earthquakes. The instance variable is of type Dexie.Table and the generic data types specify the type of the object (Earthquake) and the type of the primary key (string).
Make sure that the name of the instance variable matches the name of the object store.

In the constructor after the super call we add the code for the schema definition. Each time you need to change the schema you add a new this.version(..) statement to the constructor.

Instead of using an interface like we do in this example you can also use classes as entity object. See the documentation for an example: http://dexie.org/docs/Typescript


With the Dexie class in place we can now go ahead and instantiate it in the constructor of our provider.

export class EarthquakeProvider {
 private db: EarthquakeDb;
 constructor() {
   this.db = new EarthquakeDb();
 }

src/providers/earthquakeProvider.ts

This is already everything you need to do for setting up Dexie in a TypeScript application. From here on accessing the database works the same way as all the other examples you see in the documentation or in my previous blog post.


Dataload

Every time the application starts or the user uses the pull to refresh function the application calls the initProvider() method of the provider. Here we first fetch the timestamp of the last update

const lastUpdate = localStorage.getItem('lastUpdate');

After each successful download and insert a timestamp is set into the localStorage.
With that timestamp we can determine how old the data in our database is and what datafile we have to download to keep it up to date. Fortunately for us the USGS website hosts the earthquake data with different time periods (hour, day, week, month). So we don't have to download the complete month if our database is just a few hours old.

async initProvider(): Promise<void> {
 if (!navigator.onLine) {
   return;
 }
 const lastUpdate = localStorage.getItem('lastUpdate');
 if (lastUpdate) {
   const lastUpdateTs = parseInt(lastUpdate);
   const now = Date.now();
   if (lastUpdateTs + EarthquakeProvider.SEVEN_DAYS < now) {
     // database older than 7 days. load the 30 days file
     await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv');
   }
   else if (lastUpdateTs + EarthquakeProvider.ONE_DAY < now) {
     // database older than 1 day. load the 7 days file
     await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_week.csv');
   }
   else if (lastUpdateTs + EarthquakeProvider.ONE_HOUR < now) {
     // database older than 1 hour. load the 1 day file
     await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv');
   }
   else if (lastUpdateTs + EarthquakeProvider.FOURTYFIVE_MINUTES < now) {
     // database older than 45 minutes. load the 1 hour file
     await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_hour.csv');
   }
 }
 else {
   //no last update. load the 30 days file
   await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv');
 }
 this.deleteOldRecords();
}

src/providers/earthquakeProvider.ts

The loadData() methods downloads the file from the USGS website with the Fetch API, parses it with the PapaParse library, creates an Earthquake object for each entry and inserts it with the bulkPut() method into the database.

We don't have to worry about importing duplicates because a primary key is assigned to each earthquake and bulkPut() only inserts a record when the key not already exists. If it exists it overwrites the existing entry.

private async loadData(dataUrl) {
 const response = await fetch(dataUrl);
 const text = await response.text();
 const data = Papa.parse(text, {header: true});
 const earthquakes: Earthquake[] = [];
 for (let row of data.data) {
   if (row.id) {
     earthquakes.push({
       time: new Date(row.time).getTime(),
       place: row.place,
       mag: Number(row.mag),
       depth: Number(row.depth),
       latLng: [Number(row.latitude), Number(row.longitude)],
       id: row.id
     });
   }
 }
 this.db.transaction('rw', this.db.earthquakes, () => {
   this.db.earthquakes.bulkPut(earthquakes);
   localStorage.setItem('lastUpdate', Date.now().toString());
 });
}

src/providers/earthquakeProvider.ts

Note that bulkPut() should be (in most cases) surrounded with a manual transaction. If you don't do that and the operation fails halfway the successfully imported records will not be rolled back. You end up with a partially imported data set.


Delete old data

We don't want to fill the database indefinitely and delete therefore, after each successful import, the earthquake objects that are older than 30 days.

private deleteOldRecords() {
 const thirtyDaysAgo = Date.now() - EarthquakeProvider.THIRTY_DAYS;
 this.db.earthquakes.where("time").below(thirtyDaysAgo).delete();
}

src/providers/earthquakeProvider.ts


Query

The filter() method is responsible for applying the filter criterias and returns a promise with the matching Earthquake objects.

async filter(filter: Filter): Promise<Earthquake[]> {
 const hasMagFilter = !(filter.mag.lower === -1 && filter.mag.upper === 10);
 const hasDistanceFilter = !(filter.distance.lower === 0 && filter.distance.upper === 20000);
 const hasTimeFilter = filter.time !== -1;
 const now = new Date();
 let result: Earthquake[];

We check what filter is enabled and build the query accordingly. For the magnitude filter we use between() to filter earthquakes with a minimal and maximal magnitude. Both bounds are included (3rd and 4th parameter of the between() call)

 if (hasMagFilter && !hasTimeFilter) {
   result = await this.db.earthquakes
                         .where('mag')
                         .between(filter.mag.lower, filter.mag.upper, true, true)
                         .toArray();
 }

When we filter with the time we use aboveOrEqual()

 else if (!hasMagFilter && hasTimeFilter) {
   now.setHours(now.getHours() - filter.time);
   result = await this.db.earthquakes.where('time').aboveOrEqual(now.getTime()).toArray();
 }

When the user enabled both the magnitude and time filter we have to combine the two conditions. IndexedDB does not support AND queries natively.
Dexie provides the and() method for this purpose. The first part of the query runs in IndexedDB and the and() function runs in JavaScript. You see the query inside the comment.

I struggled a bit with this query because it is slow. On my computer with Chrome and about 9400 earthquake objects it takes about 1 second.

After fiddling with different queries, I ended up with the following solution that is much faster (30 - 40 milliseconds). It first executes the time query and then filters the objects with the magnitude in JavaScript.

The reason why this is much faster than the and() query is that toArray() calls the native getAll() method from IndexedDB and this is natively implemented in Chrome, whereas and() uses an IndexedDB cursor which is slower.

This might not be the best solution for every use case. The problem is that toArray() loads all objects at once into memory. Fortunately the individual object in this example are not very big and the time filter returns only about 1800 objects.

 else if (hasMagFilter && hasTimeFilter) {
   now.setHours(now.getHours() - filter.time);
   /*
   result = await this.db.earthquakes.where('time').aboveOrEqual(now.getTime())
     .and(e => e.mag >= filter.mag.lower && e.mag <= filter.mag.upper).toArray();
   */
   result = await this.db.earthquakes.where('time').aboveOrEqual(now.getTime()).toArray();
   result = result.filter(e => e.mag >= filter.mag.lower && e.mag <= filter.mag.upper);
 }
 else {
   result = await this.db.earthquakes.toArray();
 }

The distance filtering is done completely in JavaScript. Because the app can run on a mobile device, we have to recalculate the distance to the earthquakes each time we filter the data. I use the geolib library for the calculation.

 let filtered: Earthquake[] = [];
 if (hasDistanceFilter || filter.sort === 'distance') {
   result.forEach(r => {
     const distanceInKilometers = geolib.getDistance(
       {latitude: r.latLng[0], longitude: r.latLng[1]},
       {latitude: filter.myLocation.latitude, longitude: filter.myLocation.longitude}) / 1000;
     if (hasDistanceFilter) {
       if (filter.distance.lower <= distanceInKilometers 
                 && distanceInKilometers <= filter.distance.upper) {
         r.distance = distanceInKilometers;
         filtered.push(r);
       }
     }
     else {
       r.distance = distanceInKilometers;
       filtered.push(r);
     }
   });
 }
 else {
   filtered = result;
 }

At the end of the filter() method the array is sorted in JavaScript according to the selected sort order.

 if (filter.sort === 'mag') {
   return filtered.sort((a, b) => b.mag - a.mag);
 }
 if (filter.sort === 'distance') {
   return filtered.sort((a, b) => a.distance - b.distance);
 }
 return filtered.sort((a, b) => b.time - a.time);
}

src/providers/earthquakeProvider.ts


You find the complete source code for this version of the app with Dexie and for the native IndexedDB version on GitHub.

When you compare the two versions you see that the database code with Dexie is more concise and easier to read.