A closer look at the Web Cryptography API

Published: September 25, 2017  •  Updated: November 16, 2017  •  javascript

With the release of Safari 11 we finally get a complete Web Cryptography implementation in Apple's browser. All major browsers now support Web Cryptography. Although Microsoft Edge does not have support for PBKDF2, therefore the application I wrote for this article does not work in the Microsoft browser. Despite this annoyance with Edge it's a good time to have a closer look at the Web Cryptography API because with Safari and Chrome we have now a complete Web Cryptography implementation on both dominant mobile web browser platforms.

Web Cryptography API provides several methods in the window.crypto namespace that allows an application to hash, sign and verify, encrypt and decrypt data, import and export keys, generate keys and key pairs, wrapping and unwrapping keys and it provides access to a cryptographically sound random number generator.


Example application

The application I demonstrate in this blog post is a trivial password manager implemented with the Ionic framework on the client and Spring Boot on the server.

The user logs in with a username and a password. The application then fetches the password data stored on the server and displays them in a list. The user can now add new entries and modify existing ones. Every time he changes something the data is sent to the server. This application does not store anything on the client.

On the server side I use a Spring Boot application that stores the data from the users. To keep things simple the data is stored in memory in a Map.

I tried to follow the implementation of LastPass, a popular password manager, described in this whitepaper: https://enterprise.lastpass.com/wp-content/uploads/LastPass-Technical-Whitepaper-3.pdf
Page 6 describes the workflow from password to key:
"When the user logs into their account, the application takes the password and runs it through PBKDF2-SHA256 with 5000 iteration. The result is used as the encryption key. On this key LastPass performs another single round of hashing to create the authentication hash. This hash is sent to the server to perform an authentication check. The server does another 100,000 round PBKDF2 hashing and an additional scrypt run and then uses the result for comparing with the value stored in the database. This is the hash they store in the database to check against when the user next logs in."

My trivial example follows this workflow but I skipped the hashing on the server part, because I wanted to limit this blog post to client side cryptography. But in a real application it makes sense to follow a similar approach. I also increased the iterations on the client side to 100,000. I tested this on a Samsung Galaxy S7 device and even with this high number there was not a noticeable delay during login. It is advisable to choose a very high number for the PBKDF2 iteration, as high as your targeted devices can master. You should test this on the slowest device you want to target and choose a reasonable number and you should revisit this setting from time to time because computers get faster every year. I've found a few articles that define a minimum of 10,000 iterations.


Overview

Here an overview how the application works.

  1. User enters username and password
  2. App runs the password through 100,000 iterations of PBKDF2 SHA-256 with the username as the salt
  3. App runs the result from step 2 through another 10,000 of PBKDF2 SHA-256 iterations. This is the master key used for encryption and decryption
  4. App runs the result from step 2 through another 100,000 iterations of PBKDF2 SHA-256 and uses the result as the authentication key
  5. App fetches the stored data from the server with the authentication key (step 4)
  6. App displays the passwords.
  7. User adds new records and modifies existing entries
  8. Each time the user changes something the application compresses and encrypts the password data with the master key (step 3) and sends the blob together with the authentication key to the server.
  9. The server stores the data into a Map with the authentication key as the key and the encrypted data block as the value.

Client

I started the client application with the Ionic blank starter template and added a few additional libraries

npm install text-encoding-shim
The Web Cryptography API works with ArrayBuffer and Uint8Array objects. The encoding API converts UInt8Array objects to and from string objects.

The text-encoding-shim is not really necessary because almost all major browsers (except Edge) support the Encoding API natively. Unfortunately TypeScript did not recognize the new TextEncoder() construct, therefore I added the shim. (Is there a trick to convince TypeScript that this is a native API?)

npm install lzutf8
This is a string compression library. Perfect for our use case because the application converts the data to JSON and compress it before encrypting it.

npm install uuid
The application uses this library for assigning a unique identifier to each new record.

Almost all methods in the Web Cryptography API are asynchronous and return a Promise. Therefore I took advantage of the async and await keywords (ES2017).

All the code that handles the cryptography is bundled in the PasswordProvider class (src/providers/password/password.ts).

import {Injectable} from '@angular/core';
import {Password} from '../../../password';
import {TextEncoder} from 'text-encoding-shim';
import * as LZUTF8 from 'lzutf8';

@Injectable()
export class PasswordProvider {
 private serverUrl = "https://localhost:8080";
 private ivLen = 12;

 private passwords: Password[] = [];
 private masterKey: CryptoKey;
 private authenticationKey: Uint8Array;
 private textEncoder = new TextEncoder("utf-8");

The serverUrl points to the Spring Boot application. The ivLen variable specifies the length of the initialization vector that is needed for the AES-GCM algorithm. The passwords array contains the records with the stored password data. The authenticationKey is the key that the application sends to the server and the masterKey is used for encryption and decryption and never leaves the client. The textEncoder is used for string to Uint8Array conversions.

  async fetchPasswords(username: string, password: string) {
    await this.initKeys(username, password);

    const headers = new Headers();
    headers.append('Content-Type', 'application/octet-stream');

    const response = await fetch(`${this.serverUrl}/fetch`, {
      headers,
      method: 'POST',
      body: this.authenticationKey
    });

    const blob = await response.blob();

    if (blob.size > 0) {
      const buffer = await this.blobToArrayBuffer(blob);
      this.decrypt(buffer);
    }
  }

The fetchPasswords() method is called immediately after the user entered the username and password. It calls the initKeys() method to initialize the authentication and master key and then sends a POST request to the server to fetch the stored data. The body of the POST request contains the authentication key. When the server sends something back the method calls decrypt() to decypt the data.

 private blobToArrayBuffer(blob): Promise<ArrayBuffer> {
   var fileReader = new FileReader();

   return new Promise((resolve, reject) => {
     fileReader.onload = evt => resolve(fileReader.result);
     fileReader.onerror = reject;

     fileReader.readAsArrayBuffer(blob);
   });
 };

In the fetchPasswords() method we see that the Fetch API returns a Blob, but the Web Cryptography works with ArrayBuffers. Therefore we need to convert the Blob before we can use it in the next method. The blobToArrayBuffer() helper method utilizes the FileReader from the File API to read the Blob into an ArrayBuffer.


 private async initKeys(username: string, password: string): Promise<void> {
   const salt = this.textEncoder.encode(username);

The initKeys() method is responsible for creating the master and authentication key. It takes username and password as arguments. First ist converts the username to an Uint8Array object because it uses the username as salt for the PBKDF2 algorithm.

   const importedPassword = await crypto.subtle.importKey('raw', this.textEncoder.encode(password),
     {name: 'PBKDF2'}, false, ['deriveKey']);

Then it calls the Web Cryptography method importKey(). The following deriveKey() method expects a CryptoKey object as input and this is what importKey() returns. The importKey() method expects an Uint8Array object as input therefore we run the password through the textEncoder. The 4th parameter (false) indicates if the key that we generate here can be extracted from the CryptoKey object. Extracting means to gain access to the raw bytes of the key. The last parameter indicates what the generated key can be used for.

   const tempKey = await crypto.subtle.deriveKey(
     {
       name: 'PBKDF2', salt: salt,
       iterations: 100000, hash: 'SHA-256'
     },
     importedPassword,
     {name: 'AES-GCM', length: 256},
     true,
     ['encrypt']
   );

The deriveKey() method runs the password through the PBKDF2 algorithm. The first parameter specifies and configures the derivation algorithm. PBKDF2 needs to know the number of iterations, the salt and the hash algorithm to use.
The second parameter is the password that we wrapped in a CryptoKey object in the previous step. The third parameter is an object defining the algorithm the derived key will be used for. This is important because the length of the key depends on the encryption algorithm. AES-256 needs a key that is 256 bits long.

My initial intention was to use this key as the master key. But Web Cryptography throws an exception when you specify the usage deriveKey in a derive key operation (['encrypt', 'decrypt', 'deriveKey']).

Therefore I defined the generated key to be extractable (4th parameter) and export and import the key in the next step. The usage parameter (5th parameter) does not matter but you cannot provide an empty object ([]) otherwise Web Cryptography throws an exception.

   const exportedTempKey = await crypto.subtle.exportKey('raw', tempKey);
   const importedTempKey = await crypto.subtle.importKey('raw', exportedTempKey,
     {name: 'PBKDF2'}, false, ['deriveKey']);

This is the mentioned workaround that exports and imports the key. exportKey() takes a CryptoKey object and returns the raw bytes of the key as ArrayBuffer. exportKey() only works then the CryptoKey was created with the extractable option set to true.

   this.masterKey = await crypto.subtle.deriveKey(
     {
       name: 'PBKDF2', salt: salt,
       iterations: 10000, hash: 'SHA-256'
     },
     importedTempKey,
     {name: 'AES-GCM', length: 256},
     false,
     ['encrypt', 'decrypt']
   );

The code runs another 10,000 iteration of PBKDF2 on the temporary key and stores the result in the instance variable masterKey. This is the key the application uses for encryption and decryption (5th parameter). The key is not extractable (4th parameter). You should make the keys non-extractable when you store them in an instance variable. This prevents an attacker to open the browser console and extract the raw bytes from the key.

   const authKey = await crypto.subtle.deriveKey(
     {
       name: 'PBKDF2', salt: salt,
       iterations: 100000, hash: 'SHA-256'
     },
     importedTempKey,
     {name: 'AES-GCM', length: 256},
     true,
     ['encrypt']
   );

   this.authenticationKey = new Uint8Array(await crypto.subtle.exportKey('raw', authKey));
 }

For creating the authentication key the application runs the temporary key through another 100,000 PBKDF2 iterations, exports the bytes from the key and stores them in the instance field authenticationKey.


 private async encryptAndStore(): Promise<void> {
    const encryptedData = await this.encrypt();
    const authKeyAndData = this.concatUint8Array(this.authenticationKey, encryptedData);

    const headers = new Headers();
    headers.append('Content-Type', 'application/octet-stream');

    const requestParams = {
      headers,
      method: 'POST',
      body: authKeyAndData
    };
    fetch(`${this.serverUrl}/store`, requestParams);
 }

The encryptAndStore() method is called each time the user adds, modifies or deletes an entry. The method first calls encrypt() which returns an Uint8Array that holds the encrypted data. Then it joins the authentication key and the encrypted data block together and sends it with a POST request to the server.

 private async encrypt(): Promise<Uint8Array> {
   const compressed = LZUTF8.compress(JSON.stringify(this.passwords));
   const initializationVector = new Uint8Array(this.ivLen);
   crypto.getRandomValues(initializationVector);

   const encrypted = await crypto.subtle.encrypt({
       name: 'AES-GCM',
       iv: initializationVector
     },
     this.masterKey,
     compressed
   );

   return this.concatUint8Array(initializationVector, new Uint8Array(encrypted));
 }

The encrypt() method converts the passwords array to a JSON string, compresses it with the LZUTF8 library and then encrypts it with the AES-GCM algorithm. AES-GCM needs an initialization vector (iv) for his work. This is an array, with the recommended size of 12 bytes, that is filled with random data. It's important that your code never uses the same iv with the same key to encrypt a message. The application utilizes the Web Cryptography random generator to fill in the data into the iv array. crypto.getRandomValues() is the only method in the Web Cryptography API that is synchronous.

The crypto.subtle.encrypt() method expects three parameters: An object that describes and configures the algorithm, a CryptoKey object containing the key and the data block in plaintext to be encrypted.

The initialization vector does not have to be secret and the decryption process needs to use the exact same iv to successfully decrypt the message. Therefore the application joins the iv together with the decrypted data block.

 private async decrypt(buffer: ArrayBuffer): Promise<void> {
   const iv = buffer.slice(0, this.ivLen);
   const data = buffer.slice(this.ivLen);

   const decrypted = await window.crypto.subtle.decrypt({
       name: "AES-GCM",
       iv: iv
     },
     this.masterKey,
     data
   );

   const uncompressed = LZUTF8.decompress(new Uint8Array(decrypted));
   this.passwords = JSON.parse(uncompressed);
 }

The decrypt() method does exactly the same as the encrypt() method but in reverse order. First it extracts the iv from the data block. Then it decrypts the data blob, decompresses it and parses the string with JSON.parse.

crypto.subtle.decrypt() expects three parameters. A data object that configures the encryption algorithm, the key and the encrypted data block. The first and second parameter must be the same objects as you use in the encrypt() call.

 private concatUint8Array(...arrays: Uint8Array[]): Uint8Array {
   let totalLength = 0;
   for (let arr of arrays) {
     totalLength += arr.length;
   }
   const result = new Uint8Array(totalLength);
   let offset = 0;
   for (let arr of arrays) {
     result.set(arr, offset);
     offset += arr.length;
   }
   return result;
 }
}

concatUint8Array() is a helper method that takes an arbitrary number of Uint8Array objects and concatenates them together. The application uses this method to join the authentication key, iv and encrypted data block together.


This concludes the part of the client application that handles the cryptography. You find the source code for the complete application on GitHub:

https://github.com/ralscha/blog/tree/master/pwmanager


Server

The server is a Spring Boot application where I implemented one Controller class with two HTTP endpoints (/fetch and /store). The db instance field is our database and maps the encrypted data blobs the authentication key.

@RestController
@CrossOrigin
public class PwManagerController {

  private Map<ByteBuffer, byte[]> db = new ConcurrentHashMap<>();

  @PostMapping("/fetch")
  public byte[] fetch(@RequestBody byte[] authenticationKey) {
    ByteBuffer key = ByteBuffer.wrap(authenticationKey);
    return this.db.get(key);
  }

src/main/java/ch/rasc/pwmanager/PwManagerController.java

The fetch method takes the authentication key and returns the encrypted data blob it finds in the map. I had to wrap the authentication key byte array in a ByteBuffer instance because it's not possible to use byte[] as the key of a Map.

  @PostMapping("/store")
  public void store(@RequestBody byte[] payload) {
    byte[] keyBytes = Arrays.copyOfRange(payload, 0, 32);
    byte[] value = Arrays.copyOfRange(payload, 32, payload.length);
    ByteBuffer key = ByteBuffer.wrap(keyBytes);
    this.db.put(key, value);
  }
}

src/main/java/ch/rasc/pwmanager/PwManagerController.java

The store() method takes the encrypted blob and stores it into the map. The client prepends the encrypted data blob with the authentication key. This key has a size of 32 bytes (256 bits). The code takes the first 32 bytes and uses them as the map key and stores the rest of the data block as value in the Map.


Testing

There is one problem when you want to test the application. It works fine when you test the application on the same server where ionic serve and the Spring Boot application is running. You can open the url http://localhost:8100 in the browser and the application works fine.

But when you try to test the application on another device you encounter the error that the window.crypto namespace is not available. The Web Cryptography specification contains a restriction that window.crypto is only accessible from secure origin pages. Secure origins follow these patterns (scheme, host, port)

Even websites served over http from internal ip addresses like 192.168.. are considered to be unsafe and don't have access to the Web Cryptography API

In a production environment you have a web server with TLS certificates and proxy settings that forward the requests to Spring Boot. For testing, you could setup a similar environment, but a very trivial approach is using ngrok instead. ngrok is a service that creates public available tunnels to services behind firewalls. ngrok gives you a non-secure and a secure (TLS) url to access your service.

In our setup we have two services. The Spring Boot application listens on port 8080 and the ionic serve command on port 8100, so we need to create two ngrok tunnels

ngrok http 8100
ngrok http 8080

Next you have to open the file password.ts and change the serverUrl instance field to the https url you get from the 8080 port forward.

private serverUrl = "https://bd59a237.ngrok.io";

src/providers/password/password.ts

Now you can open the application with the https url you get from the 8100 port forward and everything should work fine.


Disclaimer

I'm not a security expert, be cautious when you copy code from this example, double check every line I wrote and read the Web Cryptography API documentation thoroughly.

Send me a feedback when I explained something wrong in this blog post.