I'm using node.js 8.4.0 and I want to write a function that performs a HTTP request and returns the result from the server. This means the function should wait until the request is plete.

I'm new to Javascript, but I've read other answers about Javascript being asynchronous and about how I should change my mental model and start using callbacks. Please, don't bother restating that, because it's childish and myopic. My need is fully legitimate and also a very simple one. If there's no way I can write the program the way I want, I'll ditch the whole thing and use a different language, the only problem being a particular library which exists only for JS.

As much as possible, I'm looking for a simple, general, robust and portable way to wait for a callback. Ideally not just for http, but also for any other async stuff. I know asynchronous programming is a great thing, but only when you really need it. I don't.

EDIT: see below.

I'm using node.js 8.4.0 and I want to write a function that performs a HTTP request and returns the result from the server. This means the function should wait until the request is plete.

I'm new to Javascript, but I've read other answers about Javascript being asynchronous and about how I should change my mental model and start using callbacks. Please, don't bother restating that, because it's childish and myopic. My need is fully legitimate and also a very simple one. If there's no way I can write the program the way I want, I'll ditch the whole thing and use a different language, the only problem being a particular library which exists only for JS.

As much as possible, I'm looking for a simple, general, robust and portable way to wait for a callback. Ideally not just for http, but also for any other async stuff. I know asynchronous programming is a great thing, but only when you really need it. I don't.

EDIT: see below.

• javascript
• node.js
• asynchronous

Share Improve this question Follow edited Sep 9, 2017 at 11:36 jurez asked Sep 6, 2017 at 7:29 jurezjurez 4,66722 gold badges1414 silver badges2222 bronze badges 21

• 5 No. no no no. NO. How to hack a single-threaded asynchronous engine and force it to perform synchronous stuff = big red alert to me. Instead, embrace the power of asynchronism and learn how to deal with it. Or go back to PHP. – Jeremy Thille Commented Sep 6, 2017 at 7:31
• 3 I'm looking for a way to wait for a callback --> You don't wait for a callback. A callback is a function that gets triggered whenever the stuff you asked for is done. In the meantime, the script execution is not blocked. That's the very principle of asynchronism. – Jeremy Thille Commented Sep 6, 2017 at 7:35
• 2 If you google synchronous HTTP requests this is what you get first: npmjs./package/sync-request – Ray Toal Commented Sep 6, 2017 at 7:35
• 7 "Please, don't bother restating that, because it's childish and myopic." Well, that's a great way to inspire people to help you. – T.J. Crowder Commented Sep 6, 2017 at 7:36
• 2 Ditch the whole thing and use another language. I'm serious. Node.js and javascript is my go-to tool but you MUST change your mental model and learn async programming. If you refuse to do this then js is not for you. You can't really change how event-oriented single-threaded systems work. So you must change how programmers think about code. – slebetman Commented Sep 6, 2017 at 7:45

 |  Show 16 more ments

2 Answers 2

Sorted by: Reset to default 2

If you feel you must issue a synchronous HTTP request...

...as Ray Toal pointed out, there's an npm package that does that for you by offloading it to a child process and using spawnSync to synchronously wait for that process to plete.

It has negative implications, which is why the advice is not to use synchronous I/O requests in Node, even though the Node API provides a few (fileReadSync, etc.). Node uses a single JavaScript thread. By blocking that thread on a long-running process (an HTTP request), you prevent that thread from doing anything else while the request is being processed.

---

If you just want to write synchronous-looking code...

...I'd suggest using promises and async/await syntax instead. async/await syntax brings asynchronous processing to code that looks synchronous, providing the benefits of asynchronous processing with synchronous-like syntax. Recent versions of Node use recent versions of the V8 JavaScript engine, which supports async/await.

Node predates Promises, and uses its own API conventions for callbacks, but there are packages such as promisify that can convert Node-callback style APIs to Promise-based ones. promisify works at the API level, so with a couple of lines of code you can convert the entire fs module's API for instance. As time moves forward, we can expect promises to be built-in to new packages (and I suspect retrofitted to the standard Node API as well).

For http.request, it's a bit more plicated than just updating the API as there are events to respond to. But "a bit more" is all it is. Here's a quick-and-simple version which also adds automatic handling of the Content-Length header:

const requestPromise = (options, postData = null) => new Promise((resolve, reject) => {
  const isPostWithData = options && options.method === "POST" && postData !== null;
  if (isPostWithData && (!options.headers || !options.headers["Content-Length"])) {
    // Convenience: Add Content-Length header
    options = Object.assign({}, options, {
      headers: Object.assign({}, options.headers, {
        "Content-Length": Buffer.byteLength(postData)
      })
    });
  }
  const body = [];
  const req = http.request(options, res => {
    res.on('data', chunk => {
      body.push(chunk);
    });
    res.on('end', () => {
      res.body = Buffer.concat(body);
      resolve(res);
    });
  });

  req.on('error', e => {
    reject(e);
  });

  if (isPostWithData) {
    req.write(postData);
  }
  req.end();
});

With that in our toolkit, we can make an asynchronous request within an async function like this:

try {
    const res = await requestPromise(/*...options...*/, /*...data if needed...*/);
    console.log(res.body.toString("utf8"));
    // ...continue with logic...
} catch (e) {
    console.error(e);
}

As you can see, the logical flow of the code is as it would be for synchronous code. But the code is not synchronous. The JavaScript thread runs the code up to and including the expression that's the operand of the first await, then does other things while the asynchronous process runs; later when the asynchronous process is plete, it picks up where it left off, assigning the result to res and then executing the console.log, etc. — up until the next await (if any). If the result of the promise await consumes is a rejection, it's processed as an exception and passed to the catch in the try/catch.

Here's the example from http.request using our requestPromise above:

try {
  const res = await requestPromise(
    {
      hostname: 'www.google.',
      port: 80,
      path: '/upload',
      method: 'POST',
      headers: {
        'Content-Type': 'application/x-www-form-urlencoded'
      }
    },
    querystring.stringify({
      'msg': 'Hello World!'
    }
  );
  console.log(res.body.toString("utf8"));
} catch (e) {
  console.error(e);
}

To use await, you have to be in an async function. You'd probably just wrap your entire module code in one:

(async () => {
  // ...module code here...
})();

If there's any possibility of your code not catching an error, add a catch-all catch handler at the end:

(async () => {
  // ...module code here...
})().catch(e => { /* ...handle the error here...*/ });

For the reference and for anybody stumbling at this in the future, this is the explanation I was looking for:

There is no such thing as synchronous requests/join/waitFor in Javascript because everything runs on the same thread, even callbacks. Callbacks are added to event queue and processed after thread returns from outer scope. Waiting for whatever, had it been possible, would essentially cause a deadlock.

This video explains it nicely: Philip Roberts: What the heck is the event loop anyway?

Thanks guys and keep up the promises!