7

I am trying to redirect one page to another by using mitmproxy and Python. I can run my inline script together with mitmproxy without issues, but I am stuck when it comes to changing the URL to another URL. Like if I went to google.com it would redirect to stackoverflow.com

def response(context, flow):
        print("DEBUG")
        if flow.request.url.startswith("http://google.com/"):
            print("It does contain it")
            flow.request.url = "http://stackoverflow/"

This should in theory work. I see http://google.com/ in the GUI of mitmproxy (as GET) but the print("It does contain it") never gets fired.

When I try to just put flow.request.url = "http://stackoverflow.com" right under the print("DEBUG") it won't work neither.

What am I doing wrong? I have also tried if "google.com" in flow.request.url to check if the URL contains google.com but that won't work either.

Thanks

MortenMoulder
  • 6,138
  • 11
  • 60
  • 116
  • (1) Do you see the “DEBUG” output? (2) Is your request actually `http://` or is it `https://`? – Vasiliy Faronov May 09 '16 at 15:27
  • @VasiliyFaronov I see the "DEBUG" output yes (I have a separate print function, which writes to a file, which I then `tail -f` on, since mitmproxy's GUI fills the whole terminal). It is `http://` and I have carefully copy-pasted the entire URL from the mitmproxy GUI, so it matches 100%. Obviously google.com and stackoverflow.com are just fillers. – MortenMoulder May 09 '16 at 15:29
  • Alternative answer here: http://stackoverflow.com/questions/24886372/mitmproxy-smart-url-replacement/35058400#35058400 – Dr. Jekyll Jul 25 '16 at 09:49

3 Answers3

11

The following mitmproxy script will

  1. Redirect requesto from mydomain.com to newsite.mydomain.com
  2. Change the request method path (supposed to be something like /getjson? to a new one `/getxml
  3. Change the destination host scheme
  4. Change the destination server port

  5. Overwrite the request header Host to pretend to be the origi

    import mitmproxy
from mitmproxy.models import HTTPResponse
from netlib.http import Headers
def request(flow):

    if flow.request.pretty_host.endswith("mydomain.com"):
            mitmproxy.ctx.log( flow.request.path )
            method = flow.request.path.split('/')[3].split('?')[0]
            flow.request.host = "newsite.mydomain.com"
            flow.request.port = 8181
            flow.request.scheme = 'http'
            if method == 'getjson':
                flow.request.path=flow.request.path.replace(method,"getxml")
            flow.request.headers["Host"] = "newsite.mydomain.com"
loretoparisi
  • 15,724
  • 11
  • 102
  • 146
3

Setting the url attribute will not help you, as it is merely constructed from underlying data. [EDIT: I was wrong, see Maximilian’s answer. The rest of my answer should still work, though.]

Depending on what exactly you want to accomplish, there are two options.

(1) You can send an actual HTTP redirection response to the client. Assuming that the client understands HTTP redirections, it will submit a new request to the URL you give it.

from mitmproxy.models import HTTPResponse
from netlib.http import Headers

def request(context, flow):
    if flow.request.host == 'google.com':
        flow.reply(HTTPResponse('HTTP/1.1', 302, 'Found',
                                Headers(Location='http://stackoverflow.com/',
                                        Content_Length='0'),
                                b''))

(2) You can silently route the same request to a different host. The client will not see this, it will assume that it’s still talking to google.com.

def request(context, flow):
    if flow.request.url == 'http://google.com/accounts/':
        flow.request.host = 'stackoverflow.com'
        flow.request.path = '/users/'

These snippets were adapted from an example found in mitmproxy’s own GitHub repo. There are many more examples there.

For some reason, I can’t seem to make these snippets work for Firefox when used with TLS (https://), but maybe you don’t need that.

Community
  • 1
  • 1
Vasiliy Faronov
  • 11,840
  • 2
  • 38
  • 49
  • Thanks for the reply! All I need is make the content of X look like the content of Y. So if I enter `website.com` on the device connected, it should show the content of `another-website.com`. Changing the host as written in your other example won't work, as I need the whole URL to change. `google.com/account` works but `stackoverflow.com/account` doesn't. Get what I mean? I need to replace a whole URL to another URL. – MortenMoulder May 09 '16 at 16:12
  • @Snorlax Method (2) replaces the host in every requested URL. It does work for me. I’m using `example.com` and `stackoverflow.com`, and when I go to `http://example.com/users` in Firefox, I do actually see the Stack Overflow users page, with (relative) links pointing to other `example.com` pages. Maybe you can tell the actual sites you’re using? For example, if your `stackoverflow.com` depends on cookies, method (2) may not work, because the browser will not send `stackoverflow.com` cookies to `google.com`—but I don’t think you can do anything about that. – Vasiliy Faronov May 09 '16 at 16:21
  • Oh, do you mean that you need to change the path in addition to the host? That’s easy, just set `flow.request.path` to whatever you want. – Vasiliy Faronov May 09 '16 at 16:24
  • The problem with replacing `flow.request.host` is the paths need to be exactly the same. I don't want that. I want to be able to say `if url == "example.com": url = "new-example.com`. I know it's possible, I just don't know how. – MortenMoulder May 09 '16 at 16:24
  • Still not good enough. I need to change an EXACT URL to another EXACT URL. `anything-goes-here.com/anything-123.html -> google.com/account` for example. – MortenMoulder May 09 '16 at 16:27
  • @Snorlax An HTTP/1.1 request (to a server) does not include the full URL. It has a host and a path, separately. If you need a way to split `http://google.com/account` into a host and a path, then use [`urlparse.urlparse`](https://docs.python.org/2/library/urlparse.html#urlparse.urlparse). – Vasiliy Faronov May 09 '16 at 16:30
  • So basically I would check if `flow.request.host.get_url()` is equal to my X URL, then I would set the host and the path afterwards, which is essentially like doing X->Y? – MortenMoulder May 09 '16 at 16:32
  • @Snorlax Exactly. But, as I said, depending on the site, there may be caveats like cookies, `localStorage`, XHR requests becoming cross-origin, etc., that would prevent the resulting page from rendering correctly. – Vasiliy Faronov May 09 '16 at 16:33
  • This will not be an issue. All I basically need to do, is change one JSON page (`file.json`) into another JSON page, but it's a bit more advanced than **just** doing that, as I need to make it user based. – MortenMoulder May 09 '16 at 16:36
  • Oh yeah I will definitely do that, but I need to reinstall my server. For some reason, mitmproxy keeps screwing up. Like I can't find `mitmproxy.models`. I did do `sudo pip install mitmproxy`, but I guess I'll have to go and compile it all from source. So far I think this sounds like a working plan! Do you know `--hypasswd` works? I got it to work and all that, but do you know how to read the username? Like `if username == "something": set path to something else` – MortenMoulder May 09 '16 at 16:40
  • It works! Thanks a lot for the help! Do you know if it's possible to get the current user sending the request though? – MortenMoulder May 09 '16 at 19:01
  • @Snorlax I suggest you post that as a separate question, this comment thread is too long already. – Vasiliy Faronov May 09 '16 at 19:06
  • I agree. Thank you for the help. – MortenMoulder May 09 '16 at 19:08
3

You can set .url attribute, which will update the underlying attributes. Looking at your code, your problem is that you change the URL in the response hook, after the request has been done. You need to change the URL in the request hook, so that the change is applied before requesting resources from the upstream server.

Maximilian Hils
  • 6,309
  • 3
  • 27
  • 46