How to normalize a URL in Java?

java url-rewriting
dfrankow picture dfrankow · Jun 8, 2010 · Viewed 44.4k times · Source

URL normalization (or URL canonicalization) is the process by which URLs are modified and standardized in a consistent manner. The goal of the normalization process is to transform a URL into a normalized or canonical URL so it is possible to determine if two syntactically different URLs are equivalent.

Strategies include adding trailing slashes, https => http, etc. The Wikipedia page lists many.

Got a favorite method of doing this in Java? Perhaps a library (Nutch?), but I'm open. Smaller and fewer dependencies is better.

I'll handcode something for now and keep an eye on this question.

EDIT: I want to aggressively normalize to count URLs as the same if they refer to the same content. For example, I ignore the parameters utm_source, utm_medium, utm_campaign. For example, I ignore subdomain if the title is the same.

Answer

Nitrodist picture Nitrodist · Jun 8, 2010

Have you taken a look at the URI class?

http://docs.oracle.com/javase/7/docs/api/java/net/URI.html#normalize()

Related questions

Best option for Session management in Java

Best way managing session in Java. I heard that cookies are not reliable option for this as they gets stored into browser and can be accessed later on? Is this correct? If possible please come up with the answers with …

Read More
Tomcat 8 URL Rewrite

I have an AngularJS webapp and Jersey backend. I need to setup URL rewriting, so everything except given exceptions will be rewritten to Angular's index.html. Eg.: http://my.domain.com/about will be rewritten http://my.domain.com/photos/…

Read More
Is there a url rewriting engine for Tomcat/Java?

How can we provide URL-Rewriting support for Tomcat/Java ? mod_rewrite provides this functionality for Apache and can modify URLs even before they reach the handler (e.g. PHP). Is there a similar tool in Java/Tomcat ? To be more …

Read More