Consider the following code within gatsby-config.js:

module.exports = {
  plugins: [
    {
      resolve: `gatsby-source-fetch`,
      options: {
        name: `brands`,
        type: `brands`,
        url: `${dynamicURL}`, // This is the part I need to be dynamic at run/build time.
        method: `get`,
        axiosConfig: {
          headers: { Accept: "text/csv" },
        },
        saveTo: `${__dirname}/src/data/brands-summary.csv`,
        createNodes: false,
      },
    },
  ],
}

As you can see above, the URL for the source plugin is something that I need to be dynamic. The reason for this is that the file URL will change every time it's updated in the CMS. I need to query the CMS for that field and get its CDN URL before passing to the plugin.

I tried adding the following to the top of gatsby-config.js but I'm getting errors.

const axios = require("axios")

let dynamicURL = ""
const getBrands = async () => {
  return await axios({
    method: "get",
    url: "https://some-proxy-url-that-returns-json-with-the-csv-file-url",
  })
}

;(async () => {
  const brands = await getBrands()
  dynamicURL = brands.data.summary.url
})()

I'm assuming this doesn't work because the config is not waiting for the request above to resolve and therefore, all we get is a blank URL.

Is there any better way to do this? I can't simply supply the source plugin with a fixed/known URL ahead of time.

Any help greatly appreciated. I'm normally a Vue.js guy but having to work with React/Gatsby and so I'm not entirely familiar with it.

Consider the following code within gatsby-config.js:

module.exports = {
  plugins: [
    {
      resolve: `gatsby-source-fetch`,
      options: {
        name: `brands`,
        type: `brands`,
        url: `${dynamicURL}`, // This is the part I need to be dynamic at run/build time.
        method: `get`,
        axiosConfig: {
          headers: { Accept: "text/csv" },
        },
        saveTo: `${__dirname}/src/data/brands-summary.csv`,
        createNodes: false,
      },
    },
  ],
}

As you can see above, the URL for the source plugin is something that I need to be dynamic. The reason for this is that the file URL will change every time it's updated in the CMS. I need to query the CMS for that field and get its CDN URL before passing to the plugin.

I tried adding the following to the top of gatsby-config.js but I'm getting errors.

const axios = require("axios")

let dynamicURL = ""
const getBrands = async () => {
  return await axios({
    method: "get",
    url: "https://some-proxy-url-that-returns-json-with-the-csv-file-url",
  })
}

;(async () => {
  const brands = await getBrands()
  dynamicURL = brands.data.summary.url
})()

I'm assuming this doesn't work because the config is not waiting for the request above to resolve and therefore, all we get is a blank URL.

Is there any better way to do this? I can't simply supply the source plugin with a fixed/known URL ahead of time.

Any help greatly appreciated. I'm normally a Vue.js guy but having to work with React/Gatsby and so I'm not entirely familiar with it.

• javascript
• reactjs
• gatsby

Share Improve this question Follow asked Aug 20, 2020 at 15:35 Michael Giovanni PumoMichael Giovanni Pumo 14.8k1818 gold badges100100 silver badges145145 bronze badges

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 10

I had similar requirement where I need to set siteId of gatsby-plugin-matomo dynamically by fetching data from async api. After searching a lot of documentation of gatsby build lifecycle, I found a solution.

Here is my approach -

gatsby-config.js

module.exports = {
   siteMetadata: {
     ...
   },
   plugins: {
     {
        resolve: 'gatsby-plugin-matomo',
        options: {
           siteId: '',
           matomoUrl: 'MATOMO_URL',
           siteUrl: 'GATSBY_SITE_URL',
           dev: true
        }
      }
    }
};

Here siteId is blank because I need to put it dynamically.

gatsby-node.js

exports.onPreInit = async ({ actions, store }) => {
    const { setPluginStatus } = actions;
    const state = store.getState();
    const plugin = state.flattenedPlugins.find(plugin => plugin.name === "gatsby-plugin-matomo");
    if (plugin) {
        const matomo_site_id = await fetchMatomoSiteId('API_ENDPOINT_URL');
        plugin.pluginOptions = {...plugin.pluginOptions, ...{ siteId: matomo_site_id }};
        setPluginStatus({ pluginOptions: plugin.pluginOptions }, plugin);
    }
};

exports.createPages = async function createPages({ actions, graphql }) {
    /* Create page code */
};

onPreInit is a gatsby lifecycle method which is executing just after plugin loaded from config. onPreInit lifecycle hook has some built in methods.

store is the redux store where gatsby is storing all required information for build process.

setPluginStatus is a redux action by which plugin data can be modified in redux store of gatsby.

Here the important thing is onPreInit lifecycle hook has to be called in async way.

Hope this helps someone in future.

Another approach that may work for you is using environment variables as you said, the URL is known so, you can add them in a .env file rather than a CSV.

By default, Gatsby uses .env.development for gatsby develop and a .env.production for gatsby build mand. So you will need to create two files in the root of your project.

In your .env (both and .env.development and .env.production) just add:

DYNAMIC_URL: https://yourUrl.

Since your gatsby-config.js is rendered in your Node server, you don't need to prefix them by GATSBY_ as the ones rendered in the client-side needs. So, in your gatsby-config.js:

module.exports = {
  plugins: [
    {
      resolve: `gatsby-source-fetch`,
      options: {
        name: `brands`,
        type: `brands`,
        url: process.env.DYNAMIC_URL, // This is the part I need to be dynamic at run/build time.
        method: `get`,
        axiosConfig: {
          headers: { Accept: "text/csv" },
        },
        saveTo: `${__dirname}/src/data/brands-summary.csv`,
        createNodes: false,
      },
    },
  ],

It's important to avoid tracking those files in your Git repository since you don't want to expose this type of data.