So I am currently working on a web scraping project in NodeJS, using Puppeteer. I am needing to use and rotate proxies on each request so avoid being blocked by the website.
However, Puppeteer only allows one proxy per browser launch, which cannot be changed without relaunching the browser. Relaunching the browser on each request would add a huge amount of time to the script.
So I was wondering if it were possible and how I could write a local proxy server that would redirect each request to a random up address from a list. The language doesn’t have to be NodeJS for the proxy server of course. I was thinking maybe use Python and The Proxy Broker library, but I’m not sure on how to go about this.
