How to extract top-level domain name (TLD) from URL

python url parsing dns extract
hoju picture hoju · Jul 1, 2009 · Viewed 65.4k times · Source

how would you extract the domain name from a URL, excluding any subdomains?

My initial simplistic attempt was:

'.'.join(urlparse.urlparse(url).netloc.split('.')[-2:])

This works for http://www.foo.com, but not http://www.foo.com.au. Is there a way to do this properly without using special knowledge about valid TLDs (Top Level Domains) or country codes (because they change).

thanks

Answer

Acorn picture Acorn · Sep 12, 2011

Here's a great python module someone wrote to solve this problem after seeing this question: https://github.com/john-kurkowski/tldextract

The module looks up TLDs in the Public Suffix List, mantained by Mozilla volunteers

Quote:

tldextract on the other hand knows what all gTLDs [Generic Top-Level Domains] and ccTLDs [Country Code Top-Level Domains] look like by looking up the currently living ones according to the Public Suffix List. So, given a URL, it knows its subdomain from its domain, and its domain from its country code.

Related questions

Retrieving parameters from a URL

Given a URL like the following, how can I parse the value of the query parameters? For example, in this case I want the value of def. /abc?def='ghi' I am using Django in my environment; is there a …

Read More
How to get everything after last slash in a URL?

How can I extract whatever follows the last slash in a URL in Python? For example, these URLs should return the following: URL: http://www.test.com/TEST1 returns: TEST1 URL: http://www.test.com/page/TEST2 returns: TEST2 URL: …

Read More
URL query parameters to dict python

Is there a way to parse a URL (with some python library) and return a python dictionary with the keys and values of a query parameters part of the URL? For example: url = "http://www.example.org/default.html?ct=32&…

Read More