I am trying to select a number of random files from a given directory. Below is my current implementation; however, there are too many files inside the folder iterating them all and then pick few random ones seems overkill.

Is there a better solution? Because what I am thinking is knowing all the files inside the folder is the precondition for random selection?

    const dirs = fs.readdirSync(IMAGE_BANK_SRC)
          .map(file => {
              return path.join(IMAGE_BANK_SRC, file);
          });

    const srcs_dup = [];

    dirs.forEach(path => {
        fs.readdirSync(path).forEach(file => {
            srcs_dup.push(file);
        });
    });

    // Pick few random ones from `srcs_dup`

Requirements:

1. The selected random files should be unique
2. The code is still working if the folder contains less files than expected
3. As fast as possible

I am trying to select a number of random files from a given directory. Below is my current implementation; however, there are too many files inside the folder iterating them all and then pick few random ones seems overkill.

Is there a better solution? Because what I am thinking is knowing all the files inside the folder is the precondition for random selection?

    const dirs = fs.readdirSync(IMAGE_BANK_SRC)
          .map(file => {
              return path.join(IMAGE_BANK_SRC, file);
          });

    const srcs_dup = [];

    dirs.forEach(path => {
        fs.readdirSync(path).forEach(file => {
            srcs_dup.push(file);
        });
    });

    // Pick few random ones from `srcs_dup`

Requirements:

1. The selected random files should be unique
2. The code is still working if the folder contains less files than expected
3. As fast as possible

• javascript
• node.js

Share Improve this question Follow edited Feb 14, 2017 at 3:42 XYZ asked Feb 14, 2017 at 3:08 XYZXYZ 27.5k1515 gold badges9494 silver badges167167 bronze badges

Add a ment  | 

4 Answers 4

Sorted by: Reset to default 4

Well, readDir & readDirSync return an array. You could avoid mapping through the entire array of paths by using the length property. We can make a dynamic sample set using a percentage of the length, then store the samples in a new array.

const dirs = fs.readdirSync(IMAGE_BANK_FILEPATH);
const length = dirs.length;
const sampleSet = 25/100 * length;
const getRandomIndex = length => Math.floor( Math.random() * length );

let samples = [];
let usedIndices = [];
let randomIndex = undefined;

for (let i = 0; i < sampleSet; i++){
  do {
    randomIndex = getRandomIndex( length );
  }
  while ( usedIndices.includes( randomIndex ) );

  usedIndices.push( randomIndex );
  samples.push( dirs[randomIndex] );
}

Basically in the below code, I created randomIndex() which grabs a random file index. After you get the list of files. I do a while loop to grab a random file from the directory list and add it to the array.

  //Grabs a random index between 0 and length
  function randomIndex(length) {
    return Math.floor(Math.random() * (length));
  }

  //Read the directory and get the files
  const dirs = fs.readdirSync(IMAGE_BANK_SRC)
    .map(file => {
      return path.join(IMAGE_BANK_SRC, file);
    });

  const srcs_dup = [];
  const hashCheck = {}; //used to check if the file was already added to srcs_dup
  var numberOfFiles = dirs.length / 10; //OR whatever # you want

  //While we haven't got the number of files we want. Loop.
  while (srcs_dup.length < numberOfFiles) {
    var fileIndex = randomIndex(dirs.length-1);

    //Check if the file was already added to the array
    if (hashCheck[fileIndex] == true) {
      continue; //Already have that file. Skip it
    }

    //Add the file to the array and object
    srcs_dup.push(dirs[fileIndex]);
    hashCheck[fileIndex] = true;
  }

  console.log(srcs_dup); //The list of your files

If this doesn't work. Let me know.

Here's a simplistic implementation. You should also consider using the path.resolve() method.

const dirs = fs.readdirSync(IMAGE_BANK_SRC)
  .map((e) => { return path.join(IMAGE_BANK_SRC, e); });

// New random list of dirs
const randomList = dirs.slice(0)
  .map((e) => { return Math.random() < .5 ? e : null; })
  .filter((e) => { return e != null; });

First, you no need to map to concat your directory path, this will loop through entire file 1 time. Second, just loop number of files you need

let result = []
let requiredCount = 3;

let files = fs.readdirSync(IMAGE_BANK_SRC)

while(requiredCount-- && files.length) {
    let length = files.length;
    let selectedIndex = Math.floor(Math.random() * length)
    let selected = files.splice(selectedIndex, 1);
    result.push(path.join(IMAGE_BANK_SRC, selected))
}