PhantomJs fail to visit a redirecting url through socks5 proxy

Question

I am writing a script using PhantomJs with socks5 proxy(for tor) to visit a url that redirects to some other url, it fails to do it.

But when I visit the same url with no proxy, it is successful.

Also, when I visit some other url which do not redirect, like whatismyipaddress.com through socks5 proxy, it is successful.

Any idea why when I combine socks5 proxy and visit redirecting url, it isn't happening, and is there any work around?

Thanks.

Can you post the specific error you are getting? I have a few recommendations but they could be wrong depending on the exact error you are receiving. It is always better to post your example code and error that the code is generated. — james-see, Sep 12 '15 at 17:56
Can you please share you source code? SOCKS5 are not working for me. — MrD, Apr 04 '16 at 13:37

score 1 · Answer 1 · edited May 23 '17 at 12:31

Here is what I recommend if this is written in Python and using Selenium & PhantomJS:

I would first import Selenium and install PhantomJS in node via npm, like the example I am pasting from this answer. (npm -g install phantomjs) Then:

Example using Selenium

from selenium import webdriver
service_args = [
'--proxy=127.0.0.1:9050',
'--proxy-type=socks5',
]
driver = webdriver.PhantomJS('../path_to/phantomjs',service_args=service_args)
driver.set_window_size(1024, 768) # optional
driver.get('https://google.com/')
driver.save_screenshot('screen.png') # save a screenshot to disk
sbtn = driver.find_element_by_css_selector('button.gbqfba')
sbtn.click()

Otherwise, if you want to try to use the urllib (Python 3) or urllib2 (Python 2.7x) packages then the following works:

Make sure you have the SocksPy package installed using pip install SocksPy or pip3 install SocksPy on Python 3.x.
import everything in: import socks, socket
Refer to this example code I provide here that binds your connection and your DNS lookups to go through the socks connection:

set up TOR connection

SOCKS_PORT = 9050

Set socks proxy and wrap the url lib module

socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, '127.0.0.1', SOCKS_PORT)

socket.socket = socks.socksocket

Perform DNS resolution through the socket

def getaddrinfo(*args): return [(socket.AF_INET, socket.SOCK_STREAM, 6,'', (args[0], args[1]))] socket.getaddrinfo = getaddrinfo

Then any urllib.request() you build will by default use the proxy you just setup.

I hope both of these options help point you in the right direction. I can update once you clarify the exact setup and issues you are facing.

PhantomJs fail to visit a redirecting url through socks5 proxy

1 Answers1