2

In JavaScript there are currently 2 alternatives for streaming a File as blocks of Uint8Array:

  1. slice(): File.slice( start, end )

    This gives a sub-Blob, which can then be read via new FileReader().readAsArrayBuffer( blob ); the ArrayBuffer can be cast to Uint8Array.

  2. stream(): File.stream().getReader().read()

    This goes via ReadableStream -> ReadableStreamDefaultReader, and read()'s .value gives the Uint8Array.

(The motivation for this "chunked" reading is usually processing large Files which the user Drag-and-Dropped into the browser, without reading them into memory all at once.)

Some differences I found:

  • slice() is older, stream() is newer.
  • slice() allows specifying a buffer size, while stream() is fixed to 64 KiB in current browsers.

The older slice() seems more flexible on all axes, providing random access (permitting easy parallel processing) and allowing custom buffer sizes (libraries such as hash-wasm are much faster on larger buffers).

Question

Are there any benefits to using stream() for this type of chunked File reading?

Perhaps stream() can implement some optimisations, such as better read-ahead or better support/performance when loading data from a local network mount? Evidence would be appreciated.

nh2
  • 24,526
  • 11
  • 79
  • 128
  • What processing exactly do you want to do? – Bergi Mar 24 '23 at 16:27
  • @Bergi The type of processing does not really matter for this question. In-browser hashing, searching in files, compression, chunked uploading, are all applications I'm interested in. But the question is on chunked bytes reading with browser APIs, which is a prerequisite for all of those. – nh2 Mar 25 '23 at 11:46
  • I'd guess that libraries for some of those would take a `ReadableStream` as the input, so they can be used on any streamable source not just blobs. For them, the benefit would just be the generally fitting API interface. But if you're implementing something for yourself, and only need to read files, it probably doesn't matter. – Bergi Mar 25 '23 at 15:53

0 Answers0