Upload files from Ionic to Spring Boot with Flow.js

Published: September 26, 2017  •  java, ionic, javascript, spring

When you have an application that uploads files with input file fields elements, then this is by default an all or nothing operation. When the user's internet connections breaks midway he has to start the upload process from the beginning.

For downloading resources via HTTP there is a built in way to resume the process described the HTTP/1.1 specification. When the download breaks the client can send a request with a range query and tell the server how many bytes he already downloaded. When the server supports the range query he can send only the missing bytes to the client: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range

Unfortunately for uploading files there is no such built in mechanism in the HTTP specification. Fortunately for us we got the File API which is widely supported in all the mainstream browsers. With this interface we have access to the raw bytes of files from JavaScript and we could build our own resume feature. Fortunately for us we don't have to do that ourselves because there are already JavaScript libraries out there that do that for us.

In this blog post I show you how to use Flow.js. Flow.js is a JavaScript library providing multiple simultaneous, stable, fault-tolerant and resumable/restartable file uploads via the File API.

I integrated the library into an Ionic app (npm install @flowjs/flow.js) and upload files to a Spring Boot application.

Flow.js is a fork of the resumable.js library. According to the project page Flow.js has been supplemented with tests and features, such as drag and drop for folders, upload speed, time remaining estimation, separate files pause, resume and more.
But resumable.js is still actively maintained and worth having a look at when Flow.js does not work in your setup.

Flow.js splits the files into multiple pieces (chunks) and sends them one by one to the server. Before it uploads a chunk it sends a GET request to ask the server if this specific piece is already uploaded. The server answers with either a 200 (already uploaded) or a 404 (not uploaded) HTTP status code. When it receives the 404 status code the library uploads the chunk, otherwise it skips it.

It's not as perfect as the range solution because in the worst case the client uploads the same chunk multiple times, which is not that bad as uploading the whole file multiple times.


Client

The example I wrote for this post is an Ionic application that captures the video stream from the camera and displays it on the screen. The user can take a snapshot of the current frame and upload it to the server as jpeg file. And he can upload the whole video recording. Both times Flow.js takes care of the upload process.

Here a screenshot how the application looks like.

When the user clicks on "Start" the application captures the video stream with the Media Capture and Streams API and streams it into a video element on the page. Additionally it starts recording the stream with the RecordRTC (npm install recordrtc) library.

  start() {
    this.recording = true;
    navigator.mediaDevices.getUserMedia({video: true, audio: false})
      .then(stream => {
        this.videoElement.nativeElement.srcObject = stream;
        this.videoElement.nativeElement.play();

        var options = {
          mimeType: 'video/webm\;codecs=vp9',
          recorderType: RecordRTC.MediaStreamRecorder
        };
        this.recordRTC = RecordRTC(stream, options);
        this.recordRTC.startRecording();

      })
      .catch(function (err) {
        console.log("An error occured! " + err);
      });
  }

src/pages/home/home.ts

Every time the user taps on the "Take a Snapshot" button the current frame of the video is internally drawn onto a canvas element and then converted to a jpeg file.

takeSnapshot() {
 const canvas = document.createElement('canvas');
 canvas.width = 1280;
 canvas.height = 960;

 const ctx = canvas.getContext('2d');
 ctx.drawImage(this.videoElement.nativeElement, 0, 0, canvas.width, canvas.height);

 canvas.toBlob(this.uploadSnapshot.bind(this), 'image/jpeg', 1)
}

src/pages/home/home.ts

The canvas.toBlob() method converts the content of the canvas element to a jpeg image. The first parameter is a callback function that is called when the conversion is finished. In this callback function the blob, that contains the jpeg image data, is wrapped in a File object, because Flow.js needs a File object as input.

private uploadSnapshot(blob: Blob) {
 const f = new File([blob], `snapshot_${Date.now()}.jpg`, {
   type: 'image/jpeg'
 });

 this.uploadFile(f);
}

src/pages/home/home.ts

When the user taps on the "Stop" button the recording of the video is uploaded to the server. The RecordRTC library returns a Blob with the recording, which the application then wraps in a File object and calls the uploadFile() method.

  stop() {
    this.recording = false;
    if (this.recordRTC) {
      this.recordRTC.stopRecording((audioVideoWebMURL) => {
        var recordedBlob = this.recordRTC.getBlob();
        this.uploadVideo(recordedBlob);
      });
    }

    this.videoElement.nativeElement.pause();
    this.videoElement.nativeElement.srcObject = null;
  }

  private uploadVideo(blob: Blob) {
    const f = new File([blob], `video_${Date.now()}.webm`, {
      type: 'video/webm'
    });

    this.uploadFile(f);
  }

src/pages/home/home.ts

The uploadFile() method is responsible for uploading the file to the server and here we see Flow.js in action.

private uploadFile(file: File) {
 const flow = new Flow({
   target: 'http://localhost:8080/upload',
   chunkSize: 50000,
   forceChunkSize: true,
   simultaneousUploads: 1,
   permanentErrors: [415, 500, 501]
 });

 flow.addFile(file);
 flow.upload();

src/pages/home/home.ts

The method creates a new Flow object and sets a few configuration parameters.

target is the URL of the upload service. The handler behind this URL needs to respond to GET and POST requests. GET requests are called for testing if the chunk is already uploaded and POST requests are used for uploading the binary data. If you don't want to use the same URL for both services, you can specify a separate URL for the test method (testMethod).

chunkSize specifies the size of the chunks in bytes. By default, Flow.js uses a chunk size of 1MB, but the files in this example are quite small so I set this to 50'000 Bytes.

The forceChunkSize option specifies if all chunks have to be less or equal than the chunkSize (true) or if the size of the last chunk can be greater or equal than the chunkSize (false).

I had to specify the permanentErrors option because the test services returns a 404 HTTP status code and by default 404 is part of the permanentErrors list. If Flow.js encounters one of the listed HTTP status codes the entire upload process is cancelled.

Flow.js supports more configuration options. You find the complete list of options on the project site:
https://github.com/flowjs/flow.js#configuration

After the setup of the Flow object the method adds the file to the upload queue and starts the upload process with flow.upload().

Flow.js emits several events. In this example I registered handlers for three of these events:

    flow.on('fileSuccess', (file, message) => {
      const toast = this.toastCtrl.create({
        message: 'Upload successful',
        duration: 3000,
        position: 'top'
      });
      toast.present();
    });

    flow.on('fileError', (file, message) => {
      const toast = this.toastCtrl.create({
        message: 'Upload failed',
        duration: 3000,
        position: 'top'
      });
      toast.present();
    });

    flow.on('fileProgress', file => {
      if (flow.progress()) {
        this.loadProgress = Math.floor(flow.progress() * 100);
      }
    });
  }

src/pages/home/home.ts

The progress of the file upload is visualized in a progress bar. I copied the code for this component from Josh Morony's blog post:
https://www.joshmorony.com/build-a-simple-progress-bar-component-in-ionic-2/


Server

For the server we don't have a library to our disposal that joins the chunks back together, fortunately the implementation is not that complicated and you find examples in the Flow.js GitHub repository: https://github.com/flowjs/flow.js/tree/master/samples

The server for this example is a Spring Boot application with a controller and two HTTP endpoints (GET /upload and POST /upload).

The upload path, where the application stores the uploaded files, is configurable and stored in the application.properties file.

uploadDirectory: e:/upload

spring.servlet.multipart.max-file-size=50KB
spring.servlet.multipart.max-request-size=50KB

src/main/resources/application.properties

The two additional spring configurations are important for applications that work with HTTP uploads. The max request size is by default 10MB and the max file size is 1 MB. The reason why there are two configuration options is that one request can transfer multiple files. Our example only uploads one file per request and we set them to the same value. The size should be as big as the chunk size we set in the Flow.js configuration.

@Controller
@CrossOrigin
public class UploadController {
  private final Map<String, FileInfo> fileInfos = new ConcurrentHashMap<>();
  private final String uploadDirectory;

  public UploadController(@Value("#{environment.uploadDirectory}") String uploadDirectory) {
    this.uploadDirectory = uploadDirectory;
    Path dataDir = Paths.get(uploadDirectory);

    try {
      Files.createDirectories(dataDir);
    }
    catch (IOException e) {
      Application.logger.error("constructor", e);
    }
  }

src/main/java/ch/rasc/upload/UploadController.java

The controller needs to know the value of the uploadDirectory property, so I inject it into the constructor with a parameter annotated with @Value. The constructor creates the directory when it does not already exist.

The controller keeps track of each upload with the fileInfos map. Key is the Flow.js identifier which is a unique string and the value is an instance of the FileInfo class. FileInfo keeps track of the uploaded chunk numbers for each file.

  @GetMapping("/upload")
  public void chunkExists(HttpServletResponse response,
        @RequestParam("flowChunkNumber") int flowChunkNumber,
        @RequestParam("flowIdentifier") String flowIdentifier) {

    FileInfo fi = this.fileInfos.get(flowIdentifier);
    if (fi != null && fi.containsChunk(flowChunkNumber)) {
      response.setStatus(HttpStatus.OK.value());
      return;
    }

    response.setStatus(HttpStatus.NOT_FOUND.value());
  }

src/main/java/ch/rasc/upload/UploadController.java

The GET /upload request handler checks if a chunk with the requested number was already uploaded. The method receives the identifier and the chunk number as request parameters and checks the fileInfo map if a mapping with the identifier exists and if a chunk with the requested number exists. If the chunk was already uploaded the method returns the HTTP status code 200 otherwise 404.

The second method in our Controller is the POST /upload handler that receives the binary data of the chunk and stores them in a file.

  @PostMapping("/upload")
  public void processUpload(HttpServletResponse response,
    @RequestParam("flowChunkNumber") int flowChunkNumber,
    @RequestParam("flowTotalChunks") int flowTotalChunks,
    @RequestParam("flowChunkSize") long flowChunkSize,
    @RequestParam("flowTotalSize") long flowTotalSize,
    @RequestParam("flowIdentifier") String flowIdentifier,
    @RequestParam("flowFilename") String flowFilename,
    @RequestParam("file") MultipartFile file) throws IOException {

Flow.js sends a series of request parameters together with the binary data.

file is a MultipartFile instance that holds the binary data of the chunk.

    FileInfo fileInfo = this.fileInfos.get(flowIdentifier);
    if (fileInfo == null) {
      fileInfo = new FileInfo();
      this.fileInfos.put(flowIdentifier, fileInfo);
    }

The method first checks the map if there is already an entry for this identifier. If not it creates a new FileInfo instance and stores it into the map.

    Path identifierFile = Paths.get(this.uploadDirectory, flowIdentifier);

    try (RandomAccessFile raf = new RandomAccessFile(identifierFile.toString(), "rw");
         InputStream is = file.getInputStream()) {
      raf.seek((flowChunkNumber - 1) * flowChunkSize);

      long readed = 0;
      long content_length = file.getSize();
      byte[] bytes = new byte[1024 * 100];
      while (readed < content_length) {
        int r = is.read(bytes);
        if (r < 0) {
          break;
        }
        raf.write(bytes, 0, r);
        readed += r;
      }
    }

Next it creates a RandomAccessFile in the upload directory with the identifier as filename and writes the bytes from the chunk into the file at the correct position.

    fileInfo.addUploadedChunk(flowChunkNumber);

    if (fileInfo.isUploadFinished(flowTotalChunks)) {
      Path uploadedFile = Paths.get(this.uploadDirectory, flowFilename);
      Files.move(identifierFile, uploadedFile, StandardCopyOption.ATOMIC_MOVE);

      this.fileInfos.remove(flowIdentifier);
    }

    response.setStatus(HttpStatus.OK.value());
  }

src/main/java/ch/rasc/upload/UploadController.java

After the write operation the method marks the chunk number as completed by storing it into the fileInfo instance. Then It has to check if all parts of the file are uploaded. The fileInfo.isUploadFinished() method compares the value of flowTotalChunks with the number of uploaded chunks and returns true when all parts of the file are uploaded. When the file is completely uploaded the application renames the file to the real file name.


You find the complete source code for the client and server application on GitHub:
https://github.com/ralscha/blog/tree/master/uploadflowjs