10

I'm creating an Electron application and I want to stream an image to a file (so basically download it).

I want to use the native Fetch API because the request module would be a big overhead.

But there is no pipe method on the response, so I can't do something like

fetch('https://imageurl.jpg')
    .then(response => response.pipe(fs.createWriteStream('image.jpg')));

So how can I combine fetch and fs.createWriteStream?

Tamás
  • 900
  • 2
  • 10
  • 28

2 Answers2

15

I got it working. I made a function which transforms the response into a readable stream.

const responseToReadable = response => {
    const reader = response.body.getReader();
    const rs = new Readable();
    rs._read = async () => {
        const result = await reader.read();
        if(!result.done){
            rs.push(Buffer.from(result.value));
        }else{
            rs.push(null);
            return;
        }
    };
    return rs;
};

So with it, I can do

fetch('https://imageurl.jpg')
    .then(response => responseToReadable(response).pipe(fs.createWriteStream('image.jpg')));
Tamás
  • 900
  • 2
  • 10
  • 28
  • 1
    Careful! This loads the whole content of the file in memory ... Might not be ideal when downloading 2 GB documents ... your might run out of memory. – Karsten Daemen Oct 04 '19 at 17:40
  • 3
    @KarstenDaemen can you point out which line of code loads it all into memory? I think that line `const result = await reader.read()` only reads a chunk. – James Kerr Mar 31 '20 at 21:14
3

Fetch is not really able to work with nodejs Streams out of the box, because the Stream API in the browser differs from the one nodejs provides, i.e. you can not pipe a browser stream into a nodejs stream or vice versa.

The electron-fetch module seems to solve that for you. Or you can look at this answer: https://stackoverflow.com/a/32545850/2016129 to have a way of downloading files without the need of nodeIntegration.

There is also needle, a smaller alternative to the bulkier request, which of course supports Streams.

RoyalBingBong
  • 1,030
  • 7
  • 14
  • This is not really true. Have a look here https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Using_readable_streams – starsinmypockets Sep 20 '18 at 13:31
  • While the browser supports streams, they are not really compatible with the ones of nodejs. You have to use a use some kind of middleware to connect both. – RoyalBingBong Sep 21 '18 at 09:24