Short version:

How do we manipulate the AST for the final output bundle, as well as the AST for a file from inside a loader? In both cases, I'd like to manipulate an existing AST rather than what I'm doing which is to parse the sources and make a new AST. What I'm doing is slow, and I know that Webpack must already have made an AST so I want to avoid duplicating efforts.

Long version:

For example, suppose I have a bunch of files written in a format similar to (but not quite) AMD modules:

module({
  Foo: '/path/to/Foo',
  Bar: '/path/to/Bar',
  Baz: '/path/to/Baz',
}, function(imports) {
  console.log(imports) // {Foo:..., Bar:... Baz:...}
})

The difference is that it is called module instead of define, the dependencies argument is a map of import names to module paths instead of an array of module paths, and the module body function receives an import object with all requested imports instead of one argument per requested import.

The above is similar to the following in AMD format, with the same output:

define([
  '/path/to/Foo',
  '/path/to/Bar',
  '/path/to/Baz',
], function(Foo, Bar, Baz) {
  console.log({Foo, Bar, Baz}) // {Foo:..., Bar:... Baz:...}
})

What is the remended way to hook into Webpack to make Webpack be able to understand the files (be able to know what dependencies the file has) in order to finally build a bundle with files that are written in this module() format?

I've already tried one approach: I made a custom loader that receives a file's source as a string, parses it and creates and AST, transform the AST, then outputs the code in AMD define() format, which Webpack understands.

However, I feel like this is slow, because if there are many files and if they are big, then parsing and making an AST from each files seem redundant, because I bet Webpack is already doing that to begin with. Is there some way to get the AST from Webpack and transform it before Webpack wants to scan it's dependencies, so that I can transform the AST into AMD format (or any recognized format for that matter), so that Webpack can finally work with the file? Is there another approach?

Short version:

How do we manipulate the AST for the final output bundle, as well as the AST for a file from inside a loader? In both cases, I'd like to manipulate an existing AST rather than what I'm doing which is to parse the sources and make a new AST. What I'm doing is slow, and I know that Webpack must already have made an AST so I want to avoid duplicating efforts.

Long version:

For example, suppose I have a bunch of files written in a format similar to (but not quite) AMD modules:

module({
  Foo: '/path/to/Foo',
  Bar: '/path/to/Bar',
  Baz: '/path/to/Baz',
}, function(imports) {
  console.log(imports) // {Foo:..., Bar:... Baz:...}
})

The difference is that it is called module instead of define, the dependencies argument is a map of import names to module paths instead of an array of module paths, and the module body function receives an import object with all requested imports instead of one argument per requested import.

The above is similar to the following in AMD format, with the same output:

define([
  '/path/to/Foo',
  '/path/to/Bar',
  '/path/to/Baz',
], function(Foo, Bar, Baz) {
  console.log({Foo, Bar, Baz}) // {Foo:..., Bar:... Baz:...}
})

What is the remended way to hook into Webpack to make Webpack be able to understand the files (be able to know what dependencies the file has) in order to finally build a bundle with files that are written in this module() format?

I've already tried one approach: I made a custom loader that receives a file's source as a string, parses it and creates and AST, transform the AST, then outputs the code in AMD define() format, which Webpack understands.

However, I feel like this is slow, because if there are many files and if they are big, then parsing and making an AST from each files seem redundant, because I bet Webpack is already doing that to begin with. Is there some way to get the AST from Webpack and transform it before Webpack wants to scan it's dependencies, so that I can transform the AST into AMD format (or any recognized format for that matter), so that Webpack can finally work with the file? Is there another approach?

• javascript
• node.js
• webpack

Share Improve this question Follow edited Feb 23, 2017 at 6:15 trusktr asked Feb 15, 2017 at 18:31 trusktrtrusktr 45.6k5858 gold badges210210 silver badges287287 bronze badges

Add a ment  | 

2 Answers 2

Sorted by: Reset to default 3

I think you will find that the loader is used during dependency parsing.

Basically the parser needs source code to be able to do its job. Therefore any import/require statement (a dependency) that is encountered during the current parse phase needs to: a. be resolved and: b. be loaded before it can be parsed. If you hook into the enhanced-resolve package's "resolve-step" you can console.log out the state transitions that the resolver transitions through typically culminating in the "create-module" plugins being fired.

Hook into "resolve-step":

piler.plugin('after-resolvers', (piler) => {
    piler.resolvers.normal.plugin('resolve-step', function (type, request){
        console.log("resolve-step type:["+type+"], 
            path:["+request.path+"], request:["+request.request+"]");   
        });
});

Hook into "create-module":

piler.plugin('pilation', (pilation, params) => {
    params.normalModuleFactory.plugin("create-module", (data) => {
        console.log('create-module: raw-request: '+data.rawRequest);
    }
}

Hope this helps.

I was looking for something like this, want to manipulate the ast but there is no example or useful documentation out there.

Googling for it just wasted 2 hours of my time but with the half pleted and inprehensible documentation I did e up with this (doesn't work though):

var acorn = require("acorn-dynamic-import").default;

function MyPlugin(options) {
  // Configure your plugin with options...
}

MyPlugin.prototype.apply = function(piler) {
  piler.plugin("pilation", function(pilation, data) {
    data.normalModuleFactory.plugin(
      "parser"
      , function(parser, options) {
        parser.plugin(
          [
            "statement if"
          ]
          ,(node)=>
            Object.assign(
              node
              ,{
                test:{
                  type:"Literal",
                  start: node.test.start,
                  end: node.test.start+4,
                  loc: {
                    start: {
                      line: 7,
                      column: 3
                    },
                    end: {
                      line: 7,
                      column: node.test.loc.start.column+4
                    }
                  },
                  range: [
                    node.test.range,
                    node.test.range+4
                  ],
                  value: true,
                  raw: "true"
                }
              }
            )
        ); 
      });
  });
};

module.exports = MyPlugin;

That would get me a node of an if statement, for other types of nodes you can look in Parser.js.

I try returning another node but creating a node is a lot of work and there doesn't seem to be an easy way to do this.

Why bother trying to do this in webpack anyway? The webpack contributors have made a nice product but nobody knows how it works because there is no documentation for many of it's features.

You're probably better off doing this as a babel plugin, that has well written and understandable documentation.

I got this working in about 20 minutes with babel:

module.exports = function ({ types: t }) {
    return {
        visitor: {
            IfStatement(path, file) {
        path.node.test = {
          "type": "BooleanLiteral",
          "start": path.node.test.start,
          "end": path.node.test.start+4,
          "loc": {
            "start": {
              "line": 7,
              "column": path.node.test.loc.start.column
            },
            "end": {
              "line": 7,
              "column": path.node.test.loc.start.column+4
            }
          },
          "value": true
        }
        // debugger;
                // path.unshiftContainer('body', t.expressionStatement(t.stringLiteral('use helloworld')));
      }
        }
  };
};

In webpack.config.js:

const path = require("path");
const webpack = require("webpack");

module.exports = {
  entry: "./src",
  output: {
    path: path.resolve(__dirname, "dist"),
    filename: "[name].chunk.js"
  },
  module:{
    rules: [
      {
        test: /\.html$/,
        use: [{ loader: './plugin/templateLoader' }]
      },
      {
        test: /\.js$/,
        exclude: /node_modules/,
        use: {
          loader: 'babel-loader',
          options: {
            presets: ['env'],
            plugins: [require("./plugin/myBabelPlugin")]
          }
        },
      }
    ]
  }
}