How to pipe to Next.js 13 api response?

Question

Earlier, I was able to pipe the response of another api call to Next.js api response as follows

export default async function (req, res) {
    // prevent same site/ obfuscate original API
    // some logic here 
    fetch(req.body.url).then(r => {
      r.body.pipe(res);
    }).catch(err => {
      console.log(err);
      res.status(500).send("Invalid Url");
    })
}

It worked fine. But now the response.body from fetch API does not have pipe method. Rather, it has pipeTo and pipeThrough methods. And the Next.js res:NextApiResponse, is not assignable to WritableStream.

I also tried creating blob (await r.blob()) and using res.send(blob) and res.send(blob.strem()). It seems to work at first but the data received by front end is not proper (Basically fetch().then((res) => res.blob()).then((blob) => URL.createObjectURL(blob))) will give corrupt result).

Klesun · Answer 1 · 2023-03-22T19:15:25.297

1

There exists the Readable.fromWeb() experimental API that transforms fetch() ReadableStream into nodejs stream. Seems to work in node v18.14.0.

import { Readable } from 'node:stream';

export default async function (req, res) {
    // prevent same site/ obfuscate original API
    // some logic here 
    fetch(req.body.url).then(r => {
      Readable.fromWeb(r.body).pipe(res);
    }).catch(err => {
      console.log(err);
      res.status(500).send("Invalid Url");
    })
}

edited Mar 22 '23 at 19:15

answered Mar 22 '23 at 19:04

Klesun

12,280
5
59
52

Thanks, this works great. Can you suggest something similar that can work with youtube videos? – Mayank Kumar Chaudhari Mar 23 '23 at 10:53
Hm, I would think that for normal video links fetched by `fetch` it would work same way through `Readable.fromWeb(r.body).pipe(res)`, though I think youtube probably has some protection from directly downloading their videos, otherwise there would not be need for tools like [youtube-dl](https://github.com/ytdl-org/youtube-dl). – Klesun Mar 23 '23 at 11:51

Mayank Kumar Chaudhari · Accepted Answer · 2023-03-23T07:45:02.923

Here's the trick that worked like a charm!

const https = require("https");

export default async function handler(req, res) {
   try {
      const url = `your-url`;
      res.status(200);
      https.get(url, (response) => {
        response.pipe(res);
        response.on("finish", res.end);
      });
    } catch (err) {
      console.log(err);
      res.status(500).send("Invalid Url");
    }
}

Update

The above trick works great for some simple cases but fails when there are redirects.

The Readable API works better than this as suggested by Klesum

import { Readable } from 'node:stream';

export default async function (req, res) {
    fetch(req.body.url).then(r => {
      Readable.fromWeb(r.body).pipe(res);
    })
}

TypeScript complains about this, but you can ignore ts warnings for that line // @ts-ignore and it would work fine.

This solution won't follow redirects though. – Klesun Mar 22 '23 at 19:06 — Klesun, Mar 22 '23 at 19:06

How to pipe to Next.js 13 api response?

2 Answers2

Update

Linked