Home | Send Feedback

Using Dexie.js in a TypeScript application

Published: January 12, 2018  •  Updated: December 03, 2018  •  database, javascript, ionic4

In the previous blog post I wrote about Dexie. An IndexedDB wrapper library, that simplifies the IndexedDB programming.

In this blog post I show you a complete application and how to integrate Dexie into a TypeScript (Ionic) application.

The application is a clone of the application I wrote for this blog post about IndexedDB. It downloads a list of earthquakes, that happened in the last month, from the USGS Earthquake Hazard Program website, puts the records into an IndexedDB database and displays the data in a list. The user can filter the data on various criteria like magnitude and time.

Dexie with TypeScript

When you are trying to add Dexie, the way I showed you in the previous blog post, with code like this you will encounter an error.

const db = new Dexie("MyDb");
db.version(1).stores({
  earthquakes: 'id'
});
const allObjects = await db.earthquakes.toArray();

In vanilla JavaScript you get implicit properties for each object store you define in stores(). But since they are defined at runtime TypeScript does not recognize them and this code does not compile.

A workaround is to use the table() method.

const allObjects = await db.table('earthquakes').toArray();

But you don't get any code completion and type safety a reason why you use TypeScript in the first place.


Therefore, the recommended way is to create a TypeScript class. Before we do that we define an interface for our entity we want to store in the database.

export interface Earthquake {
  id: string;
  time: number;
  place: string;
  mag: number;
  depth: number;
  distance?: number;
  latLng: [number, number];
}

earthquake-db.ts

Now we create the special Dexie database class. It has to be a subclass of Dexie.
In the constructor you have to call the constructor of the super class with the name of the database as parameter.

export class EarthquakeDb extends Dexie {
  earthquakes: Dexie.Table<Earthquake, string>;

  constructor() {
    super('Earthquake');
    this.version(1).stores({
      earthquakes: 'id,mag,time'
    });
  }
}

earthquake-db.ts

The class contains an instance variable for each object store. In this example we only have one object store with name earthquakes. The instance variable is of type Dexie.Table and the generic data types specify the type of the object (Earthquake) and the type of the primary key (string).
Make sure that the name of the instance variable matches the name of the object store.

In the constructor, after the super call, we add the code for the schema definition. Each time you need to change the schema you add a new this.version(...) statement to the constructor.

Instead of using an interface like we do in this example you can also use classes as entity object. See the documentation for an example: https://dexie.org/docs/Typescript


With the Dexie class in place we can now go ahead and instantiate it in the constructor of our service.

  constructor() {
    this.db = new EarthquakeDb();
  }

earthquake.service.ts

This is already everything you need to do for setting up Dexie in a TypeScript application. From here on accessing the database works the same way as all the other examples you see in the documentation or in my previous blog post.

Data load

Every time the application starts or the user uses the pull to refresh function, the application calls the initProvider() method of the service. Here we first fetch the timestamp of the last update

    const lastUpdate = localStorage.getItem('lastUpdate');

earthquake.service.ts

After each successful download and insert, a timestamp is stored in the localStorage.
With that timestamp we can determine how old the data in our database is and what datafile we have to download to keep it up to date. Fortunately for us the USGS website hosts the earthquake data with different time periods (hour, day, week, month). So we don't have to download the complete month if our database is just a few hours old.

    if (lastUpdate) {
      const lastUpdateTs = parseInt(lastUpdate, 10);
      const now = Date.now();
      if (lastUpdateTs + EarthquakeService.SEVEN_DAYS < now) {
        // database older than 7 days. load the 30 days file
        await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv');
      } else if (lastUpdateTs + EarthquakeService.ONE_DAY < now) {
        // database older than 1 day. load the 7 days file
        await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_week.csv');
      } else if (lastUpdateTs + EarthquakeService.ONE_HOUR < now) {
        // database older than 1 hour. load the 1 day file
        await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv');
      } else if (lastUpdateTs + EarthquakeService.FOURTYFIVE_MINUTES < now) {
        // database older than 45 minutes. load the 1 hour file
        await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_hour.csv');
      }
    } else {
      // no last update. load the 30 days file
      await this.loadData('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_month.csv');
    }

    return this.deleteOldRecords();

earthquake.service.ts

The loadData() methods downloads the file from the USGS website with the Fetch API, parses it with the PapaParse library, creates an Earthquake object for each entry and inserts it with the bulkPut() method into the database.

We don't have to worry about importing duplicates because a primary key is assigned to each earthquake and bulkPut() only inserts a record when the key not already exists. If it exists, it overwrites the existing entry.

  private async loadData(dataUrl) {
    const response = await fetch(dataUrl);
    const text = await response.text();
    const data = Papa.parse(text, {header: true});

    const earthquakes: Earthquake[] = [];

    for (const row of data.data) {
      if (row.id) {
        earthquakes.push({
          time: new Date(row.time).getTime(),
          place: row.place,
          mag: Number(row.mag),
          depth: Number(row.depth),
          latLng: [Number(row.latitude), Number(row.longitude)],
          id: row.id
        });
      }
    }

    this.db.transaction('rw', this.db.earthquakes, async () => {
      await this.db.earthquakes.bulkPut(earthquakes);
      localStorage.setItem('lastUpdate', Date.now().toString());
    });

  }

earthquake.service.ts

Note that bulkPut() should be (in most cases) surrounded with a manual transaction. If you don't do that and the operation fails halfway, the successfully imported records will not be rolled back. You end up with a partially imported data set.

Delete old data

We don't want to fill the database indefinitely and delete therefore, after each successful import, the earthquake objects that are older than 30 days.

  private deleteOldRecords() {
    const thirtyDaysAgo = Date.now() - EarthquakeService.THIRTY_DAYS;
    return this.db.earthquakes.where('time').below(thirtyDaysAgo).delete();
  }

earthquake.service.ts

Query

The filter() method is responsible for applying the filter criteria and returns a promise with the matching Earthquake objects.

  async filter(filter: Filter): Promise<Earthquake[]> {
    const hasMagFilter = !(filter.mag.lower === -1 && filter.mag.upper === 10);
    const hasDistanceFilter = !(filter.distance.lower === 0 && filter.distance.upper === 20000);
    const hasTimeFilter = filter.time !== '-1';
    const now = new Date();

    let result: Earthquake[];

earthquake.service.ts

We check what filter is enabled and build the query accordingly. For the magnitude filter we use between() to filter earthquakes with a minimal and maximal magnitude. Both bounds are included (3rd and 4th parameter of the between() call)

    if (hasMagFilter && !hasTimeFilter) {
      result = await this.db.earthquakes.where('mag').between(filter.mag.lower, filter.mag.upper, true, true).toArray();

earthquake.service.ts

When we filter on the time we use aboveOrEqual()

    } else if (!hasMagFilter && hasTimeFilter) {
      now.setHours(now.getHours() - parseInt(filter.time, 10));
      result = await this.db.earthquakes.where('time').aboveOrEqual(now.getTime()).toArray();

earthquake.service.ts

When the user enabled both the magnitude and time filter we have to combine the two conditions. IndexedDB does not support AND queries natively.
Dexie provides the and() method for this purpose. The first part of the query runs in IndexedDB and the and() function runs in JavaScript.

   result = await this.db.earthquakes.where('time').aboveOrEqual(now.getTime())
     .and(e => e.mag >= filter.mag.lower && e.mag <= filter.mag.upper).toArray();

I struggled a bit with this query because it is slow. On my computer with Chrome and about 9400 earthquake objects it took about 1 second.

After fiddling with different queries, I ended up with the following solution that is much faster (30 - 40 milliseconds). It first executes the time query and then filters the objects with the magnitude in JavaScript.

The reason why this is much faster than the and() query is that toArray() calls the native getAll() method from IndexedDB and this is natively implemented in Chrome, whereas and() uses an IndexedDB cursor which is slower.

This might not be the best solution for every use case. The problem is that toArray() loads all objects at once into memory. Fortunately the individual object in this example are not that big.

    } else if (hasMagFilter && hasTimeFilter) {
      now.setHours(now.getHours() - parseInt(filter.time, 10));
      result = await this.db.earthquakes.where('time').aboveOrEqual(now.getTime()).toArray();
      result = result.filter(e => e.mag >= filter.mag.lower && e.mag <= filter.mag.upper);

earthquake.service.ts

The distance filtering is done completely in JavaScript. Because the app can run on a mobile device, we have to recalculate the distance to the earthquakes each time we filter the data. I use the geolib library for this calculation.

    let filtered: Earthquake[] = [];

    if (hasDistanceFilter || filter.sort === 'distance') {
      result.forEach(r => {
        const distanceInKilometers = geolib.getDistance(
          {latitude: r.latLng[0], longitude: r.latLng[1]},
          {latitude: filter.myLocation.latitude, longitude: filter.myLocation.longitude}) / 1000;

        if (hasDistanceFilter) {
          if (filter.distance.lower <= distanceInKilometers && distanceInKilometers <= filter.distance.upper) {
            r.distance = distanceInKilometers;
            filtered.push(r);
          }
        } else {
          r.distance = distanceInKilometers;
          filtered.push(r);
        }
      });
    } else {
      filtered = result;
    }

earthquake.service.ts

At the end of the filter() method, the code is sorting the result in JavaScript according to the selected sort order.

    if (filter.sort === 'mag') {
      return filtered.sort((a, b) => b.mag - a.mag);
    }

    if (filter.sort === 'distance') {
      return filtered.sort((a, b) => a.distance - b.distance);
    }

    return filtered.sort((a, b) => b.time - a.time);

earthquake.service.ts


You find the complete source code for this version of the app with Dexie and for the native IndexedDB version on GitHub.

When you compare the two versions you see that the database code with Dexie is more concise and easier to read.