App loads user's text files and each of them can be changed by user. On app's start I want to check if any file was changed since last time. I think the most efficient way is to calculate checksum of each file and save to one json file. On app's start I will check each file checksum and pare it to data from json file Is there any more optimal/efficient way of doing this ? Or how exactly calculate file checksum ?

App loads user's text files and each of them can be changed by user. On app's start I want to check if any file was changed since last time. I think the most efficient way is to calculate checksum of each file and save to one json file. On app's start I will check each file checksum and pare it to data from json file Is there any more optimal/efficient way of doing this ? Or how exactly calculate file checksum ?

• javascript
• json
• node.js
• file
• checksum

Share Follow asked Dec 2, 2016 at 23:22 Piotr WuPiotr Wu 1,36233 gold badges1414 silver badges3131 bronze badges 1

• 1 Wouldn't be simpler to check the last modified time stamp of the file? – Anthony C Commented Dec 3, 2016 at 0:13

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 9

I believe Using fs.stat and checking 'last modified' is much faster than reading the whole file and paring checksum as it is only metadata (and you don't actually read the whole file).

Also, If your files are located within different directories you can test if a folder was changed. this can reduce I/O calls (in case the last modified date didn't change you can skip checking the files on that folder).

You will have to store the 'last modified' date, I would use Redis for this. you will need to update it on every modification change and on the first run of course.

here is a function (and a function call) to test if a file or folder was changed:

let fs = require('fs');
let moment = require('moment');

let path = 'views'; //your folder path (views is an example folder)
wasFileChanged(path, (err,wasChanged) => {
  if (wasChanged){
    console.log('folder was changed, need to pare files');
    //need to update redis here
    //...apre files to find what was changed
  }
  else{
    console.log('folder was not changed');
  }
});

/**
 * Checks if a file/folder was changed 
 */
function wasFileChanged(path, callback) {
  fs.open(path, 'r', (err, fd) => {
    if (err) {
      return callback (err);
    } else {

      //obtain previous modified date of the folder (I would use redis to store/retrieve this data)
      let lastModifed = '2016-12-03T00:41:12Z'; //put the string value here, this is just example

      fs.stat(path, (err, data) => {
        console.log('check if file/folder last modified date, was it after my last check ');

        //I use moment module to pare dates
        let previousLMM = moment(lastModifed);
        let folderLMM = moment(data.mtime.toISOString());
        let res = !(folderLMM.isSame(previousLMM, 'second')); //seconds granularity
        return callback (null, res);
      });
    }
  });
}

Seems like this blog is a good read for you: http://blog.tompawlak/calculate-checksum-hash-nodejs-javascript

code example (from the blog):

var crypto = require('crypto');

function checksum (str, algorithm, encoding) {
    return crypto
        .createHash(algorithm || 'md5')
        .update(str, 'utf8')
        .digest(encoding || 'hex')
}

To read from a file and present its hash:

fs.readFile('test.dat', function (err, data) {
    checksum(data);         // e53815e8c095e270c6560be1bb76a65d
    checksum(data, 'sha1'); // cd5855be428295a3cc1793d6e80ce47562d23def
});

Comparing checksum to find if a file was changed is valid and you can also pare when file was last modified by using fs.stat