How can I unshorten a URL?

Question

I want to be able to take a shortened or non-shortened URL and return its un-shortened form. How can I make a python program to do this?

Additional Clarification:

Case 1: shortened --> unshortened
Case 2: unshortened --> unshortened

e.g. bit.ly/silly in the input array should be google.com in the output array
e.g. google.com in the input array should be google.com in the output array

Are you talking about a specific URL shortening service, and does this service have an API you can retrieve the info from? — JAL, Nov 17 '10 at 02:58
If you are in a hurry, you could also use this API https://rapidapi.com/logicione/api/url-expander1 — Watt, Feb 08 '20 at 10:28

score 40 · Accepted Answer · answered Nov 17 '10 at 03:20

40

Send an HTTP HEAD request to the URL and look at the response code. If the code is 30x, look at the Location header to get the unshortened URL. Otherwise, if the code is 20x, then the URL is not redirected; you probably also want to handle error codes (4xx and 5xx) in some fashion. For example:

# This is for Py2k.  For Py3k, use http.client and urllib.parse instead, and
# use // instead of / for the division
import httplib
import urlparse

def unshorten_url(url):
    parsed = urlparse.urlparse(url)
    h = httplib.HTTPConnection(parsed.netloc)
    h.request('HEAD', parsed.path)
    response = h.getresponse()
    if response.status/100 == 3 and response.getheader('Location'):
        return response.getheader('Location')
    else:
        return url

answered Nov 17 '10 at 03:20

Adam Rosenfield

390,455
97
512
589

ignores url query, better version here: http://stackoverflow.com/a/7153185/818634 – DmitrySandalov Jul 12 '13 at 21:06
5

do note when using above code does not unshorten recursively in case you want to obtain the actual URL. Try on `http://t.co/hAplNMmSTg`. You need to do `return unshorten_url(response.getheader('Location'))` for recursivity. – Andrei-Niculae Petre Jun 30 '14 at 11:48
1

Possibly also keep track of previous urls in a set to prevent cyclic recursion. – Herbert Feb 15 '17 at 14:15

score 34 · Answer 2 · answered Mar 07 '15 at 18:00

34

Using requests:

import requests

session = requests.Session()  # so connections are recycled
resp = session.head(url, allow_redirects=True)
print(resp.url)

answered Mar 07 '15 at 18:00

GermainZ

1,893
3
15
20

4

I like this solution, it automatically follows multiple redirects – neuhaus Nov 16 '16 at 09:33
I had to set `verify=False` as Requests could not validate the cert – Joe Jun 14 '22 at 19:32
Is there a way to have requests display the url of each redirect? – newcool Jul 02 '23 at 02:48

score 5 · Answer 3 · edited Feb 14 '16 at 13:29

5

Unshorten.me has an api that lets you send a JSON or XML request and get the full URL returned.

edited Feb 14 '16 at 13:29

Anup pokhrel

3
3

answered Nov 17 '10 at 03:00

user387049

6,647
8
53
55

score 5 · Answer 4 · answered May 04 '20 at 07:51

5

If you are using Python 3.5+ you can use the Unshortenit module that makes this very easy:

from unshortenit import UnshortenIt
unshortener = UnshortenIt()
uri = unshortener.unshorten('https://href.li/?https://example.com')

answered May 04 '20 at 07:51

fmarm

4,209
1
17
29

score 4 · Answer 5 · answered Nov 17 '10 at 03:19

4

Open the url and see what it resolves to:

>>> import urllib2
>>> a = urllib2.urlopen('http://bit.ly/cXEInp')
>>> print a.url
http://www.flickr.com/photos/26432908@N00/346615997/sizes/l/
>>> a = urllib2.urlopen('http://google.com')
>>> print a.url
http://www.google.com/

answered Nov 17 '10 at 03:19

hughdbrown

47,733
20
85
108

3

This does a GET of the whole page. If the page isn't a redirect and happens to be very large, you're wasting a huge amount of bandwidth just to determine that it's not a redirect. Much better to use a HEAD request instead. – Adam Rosenfield Nov 17 '10 at 03:22
1

@Adam Rosenfeld: It's probably an appropriate answer for a side project for someone beginning python. I don't recommend that Google or Yahoo spider pages like this to find the real URL. – hughdbrown Nov 18 '10 at 15:08
It is a **NOT GOOD IDEA** doing this. You wasting a lot of bandwidth. Just using http://unshort.me api is better and faster as @user387049 suggested – Cory Mar 26 '11 at 22:57

score 4 · Answer 6 · answered May 01 '17 at 00:03

4

To unshort, you can use requests. This is a simple solution that works for me.

import requests
url = "http://foo.com"

site = requests.get(url)
print(site.url)

answered May 01 '17 at 00:03

Daniel Cambría

217
1
6

score 1 · Answer 7 · answered Jul 12 '13 at 13:34

1

http://github.com/stef/urlclean

sudo pip install urlclean
urlclean.unshorten(url)

answered Jul 12 '13 at 13:34

DmitrySandalov

3,879
3
23
17

Unfortunately this is python 2 only, and why would one write unparenthised print's in python code in 2012 :( – Herbert Feb 15 '17 at 14:21

Amir Krifa · Answer 8 · 2015-07-16T18:40:31.170

Here a src code that takes into account almost of the useful corner cases:

set a custom Timeout.
set a custom User Agent.
check whether we have to use an http or https connection.
resolve recursively the input url and prevent ending within a loop.

The src code is on github @ https://github.com/amirkrifa/UnShortenUrl

comments are welcome ...

import logging
logging.basicConfig(level=logging.DEBUG)

TIMEOUT = 10
class UnShortenUrl:
    def process(self, url, previous_url=None):
        logging.info('Init url: %s'%url)
        import urlparse
        import httplib
        try:
            parsed = urlparse.urlparse(url)
            if parsed.scheme == 'https':
                h = httplib.HTTPSConnection(parsed.netloc, timeout=TIMEOUT)
            else:
                h = httplib.HTTPConnection(parsed.netloc, timeout=TIMEOUT)
            resource = parsed.path
            if parsed.query != "": 
                resource += "?" + parsed.query
            try:
                h.request('HEAD', 
                          resource, 
                          headers={'User-Agent': 'curl/7.38.0'}
                                   }
                          )
                response = h.getresponse()
            except:
                import traceback
                traceback.print_exec()
                return url

            logging.info('Response status: %d'%response.status)
            if response.status/100 == 3 and response.getheader('Location'):
                red_url = response.getheader('Location')
                logging.info('Red, previous: %s, %s'%(red_url, previous_url))
                if red_url == previous_url:
                    return red_url
                return self.process(red_url, previous_url=url) 
            else:
                return url 
        except:
            import traceback
            traceback.print_exc()
            return None

If I understand your flow correctly, you might want to put a cap on how many redirects you'll tolerate — Foon, Jul 16 '15 at 14:10
@Foon in some cases, the redirect points to the same previous url, so, to prevent the trap of an infinite loop, i propagate the previous url within the recusive call and if i end up with red_url == previous_url, i stop and return that url. Otherwise, in a normal case, at some iteration, the response.status will not be equal anymore to a redirection status, so, we return the retrieved url. — Amir Krifa, Jul 16 '15 at 17:19
@AmirKrifa does that handle link.foo which points to link.bar which points back to link.foo? (I don't know httplib to know if there's an option to follow redirects, in which case, this sort of link would throw an exception before you called the recursive call) — Foon, Jul 16 '15 at 17:26

score 1 · Answer 9 · answered Jun 17 '20 at 07:23

1

You can use geturl()

from urllib.request import urlopen
url = "bit.ly/silly"
unshortened_url = urlopen(url).geturl()
print(unshortened_url)
# google.com

answered Jun 17 '20 at 07:23

Tiago Martins Peres

14,289
18
86
145

score 0 · Answer 10 · answered Sep 03 '21 at 17:14

0

This Is very easy task you just need to add 4 lines of codes thats it :)

import requests
url = input('Enter url : ')
site = requests.get(url)
print(site.url)

just run this code you will successfully unshort the url.

answered Sep 03 '21 at 17:14

AYUSH KUMAR JAISWAL

9
2

It's the same as this answer: https://stackoverflow.com/a/43712633/2745495 – Gino Mempin Sep 05 '21 at 03:01

How can I unshorten a URL?

10 Answers10

Linked