Does node.js use threads/thread pool internally?

Question

I've decided to familiarize myself with node.js and have read a several articles on the subject. What remained unclear to me is if node.js creates new threads and/or schedules tasks on threads from a thread pool when you call node.js functions.

For example if I call fs.readFile is it executed on a different thread?

If yes, [how] can I write my own function readFileCustomized or doLongOperation to run on a different thread?

Worth reading this: [Which would be better for concurrent tasks on node.js? Fibers? Web-workers? or Threads?](http://stackoverflow.com/questions/10773564/which-would-be-better-for-concurrent-tasks-on-node-js-fibers-web-workers-or-t) — jfriend00, Dec 03 '13 at 08:38
@jfriend00 haven't seen this one and it's definitely useful. Thank you for suggesting it. — Andrew Savinykh, Dec 03 '13 at 08:42
Just tried to find the source for everything that's `fs.readFile` related: `fs.readFile > fs.read > ASYNC_CALL(read(2)) (macro) > FSReqWrap > ReqWrap > BaseObject`. If you want to check the intrinsics of `fs.readFile` it will take a little while. — Zeta, Dec 03 '13 at 08:45

score 9 · Accepted Answer · answered Dec 03 '13 at 08:51

9

There is no async API for file operations so node.js uses a thread pool for that. You can see it in the code of libuv.

The pool can run 4 threads:

static uv_thread_t default_threads[4];

Blocking FS tasks are posted with uv__work_submit. For example, here's how read is implemented:

int uv_fs_read(uv_loop_t* loop, uv_fs_t* req,
               uv_file file,
               void* buf,
               size_t len,
               int64_t off,
               uv_fs_cb cb) {
  INIT(READ);
  req->file = file;
  req->buf = buf;
  req->len = len;
  req->off = off;
  POST;
}

...

#define POST                                                                  \
  do {                                                                        \
    if ((cb) != NULL) {                                                       \
      uv__work_submit((loop), &(req)->work_req, uv__fs_work, uv__fs_done);    \
      return 0;                                                               \
    }                                                                         \
    else {                                                                    \
      uv__fs_work(&(req)->work_req);                                          \
      uv__fs_done(&(req)->work_req, 0);                                       \
      return (req)->result;                                                   \
    }                                                                         \
  }                                                                           \
  while (0)

If you want to implement your own threads, you can check this great introduction.

answered Dec 03 '13 at 08:51

Laurent Perrin

14,671
5
50
49

I don't think any thread is blocked for the duration of the IO though. – usr Dec 03 '13 at 08:51
1

libuv ultimately uses stdio's read: https://github.com/joyent/libuv/blob/v0.10.19/src/unix/fs.c#L184 which is blocking. – Laurent Perrin Dec 03 '13 at 08:56
Thank you for this, I've already studied the last linked article, but it concerns libuv, I was wondering how much of that is exposed in node.js. It does not look like thread pool/threading is exposed at all. – Andrew Savinykh Dec 03 '13 at 08:57
I'm unqualified to read the C code. I wonder whether this code path is ever invoked. That would be very much against the purpose of libuv. Surprising to say the least. Care to look at how socket IO is done? – usr Dec 03 '13 at 09:03
You check check [node-lame's](https://github.com/TooTallNate/node-lame/blob/master/src/node_mpg123.cc) code. It uses threads and libuv. – Laurent Perrin Dec 03 '13 at 09:03
1

all network IO is fully async (it uses epoll for Linux for example). However, FS is always blocking in a thread pool. – Laurent Perrin Dec 03 '13 at 09:05
Ah yes, the example with node-lame makes sense. Although it's binding to some native code it showcases where threading would be most useful. I suspected that if you wanted to do threading you would have to write in in C/C++. It seems to be the case indeed. – Andrew Savinykh Dec 03 '13 at 09:45

Does node.js use threads/thread pool internally?

1 Answers1

Linked