I'm currently using ExcelJS to read large Excel files (10,000+ lines) into a NodeJS/Angular/Electron app. It reads the smaller files just fine, but the larger files take anywhere from 3.9 to 5 seconds to read, and during that time, the CSS doesn't update at all.

I'm currently using async/await to load the files, as I thought that would allow other operations to happen while waiting on the file load, as I thought I/O operations in Node had their own thread. I also read that CPU intensive tasks block all other Node processes.

This is the code that currently loads the workbook:

async openWorkbook(filename: string) {
    this.loading = true;
    const workbook = new Workbook();
    const promise = workbook.xlsx.readFile(this.path + '/sheets/' + filename)
      .then(() => {
        // use workbook
        workbook.getWorksheet(1).name = workbook.getWorksheet(1).name.slice(0, 31);
        const excelFile: ExcelFile = { workbook: workbook, filename: filename };
        this.wbSource.next(excelFile);
      });

    const read = await promise;
  }

This code works the same as the callback implementation, they both block the app's CSS. How would I go about reading a file (or do any CPU intensive task) without blocking the UI?

I'm currently using ExcelJS to read large Excel files (10,000+ lines) into a NodeJS/Angular/Electron app. It reads the smaller files just fine, but the larger files take anywhere from 3.9 to 5 seconds to read, and during that time, the CSS doesn't update at all.

I'm currently using async/await to load the files, as I thought that would allow other operations to happen while waiting on the file load, as I thought I/O operations in Node had their own thread. I also read that CPU intensive tasks block all other Node processes.

This is the code that currently loads the workbook:

async openWorkbook(filename: string) {
    this.loading = true;
    const workbook = new Workbook();
    const promise = workbook.xlsx.readFile(this.path + '/sheets/' + filename)
      .then(() => {
        // use workbook
        workbook.getWorksheet(1).name = workbook.getWorksheet(1).name.slice(0, 31);
        const excelFile: ExcelFile = { workbook: workbook, filename: filename };
        this.wbSource.next(excelFile);
      });

    const read = await promise;
  }

This code works the same as the callback implementation, they both block the app's CSS. How would I go about reading a file (or do any CPU intensive task) without blocking the UI?

• javascript
• node.js
• typescript
• electron
• exceljs

Share Improve this question Follow asked Aug 16, 2019 at 15:31 MusicDevMusicDev 9411 gold badge22 silver badges1313 bronze badges 17

• 1 Isn't this an ideal candidate for serviceworkers? or web workers? developer.mozilla/en-US/docs/Web/API/Web_Workers_API/… – Deckerz Commented Aug 16, 2019 at 15:33
• Create a timer using setInterval to run at regular intervals, read N rows at a time, do something with them then exit the timer and allow it to run again, when all rows have been read clean up by stopping the timer and closing the open workbook. – SPlatten Commented Aug 16, 2019 at 15:34
• @SPlatten - ouch! that is doing it the hard way. – I wrestled a bear once. Commented Aug 16, 2019 at 15:35
• @Deckerz I thought I read somewhere that you should avoid using web/service workers for I/O tasks? – MusicDev Commented Aug 16, 2019 at 15:37
• @Iwrestledabearonce, whats hard about it? – SPlatten Commented Aug 16, 2019 at 15:37

 |  Show 12 more ments

3 Answers 3

Sorted by: Reset to default 3

The first thing i would do in this case is preventing memory overhead by not "opening" large files at all.

So something like this should work:

const stream = fs.createReadStream(filePath);
const workbook = new Excel.Workbook();
stream.pipe(workbook.xlsx.createInputStream());

and as those chunks block the cycle for a minor amount of time you can use this even in the javascript event loop ;)

JavaScript is a single threaded language. If one of your task is taking long to plete, other tasks like rendering will be blocked. I can't write plete code for you but Web Worker seems a perfect solution for what you're trying to do. (Which people have already suggested in ments)

You can refer:

https://developer.mozilla/en-US/docs/Web/API/Web_Workers_API/Using_web_workers

Additionally, also check out https://github./GoogleChromeLabs/link which is a wrapper around the Web Worker APIs. It'll make it easier for you to migrate existing code to worker with less hassle.

I'm just going to leave my solution here using my Thread function from the ments.

async openWorkbook(filename: string) {
  this.loading = true;

  var path = this.path + '/sheets/' + filename;
  const excelFile = await Thread(path, function(path, exit) {

    // You'll need to import excel again since it's a new thread
    import Excel from 'exceljs';

    const workbook = new Excel.Workbook();
    workbook.xlsx.readFile(path).then(() => {
      workbook.getWorksheet(1).name = workbook.getWorksheet(1).name.slice(0, 31);
      const excelFile: ExcelFile = {
        workbook: workbook,
        filename: filename
      };
      exit(excelFile);
    });
  });

  this.wbSource.next(excelFile);

}