C# and HtmlAgilityPack encoding problem

c# encoding html-agility-pack
August picture August · Aug 10, 2010 · Viewed 38.5k times · Source 
WebClient GodLikeClient = new WebClient();
HtmlAgilityPack.HtmlDocument GodLikeHTML = new HtmlAgilityPack.HtmlDocument();

GodLikeHTML.Load(GodLikeClient.OpenRead("www.alfa.lt");

So this code returns: "Skaitytojo klausimas psichologui: kas lemia homoseksualumÄ…? - Naujienų portalas Alfa.lt" instead of "Skaitytojo klausimas psichologui: kas lemia homoseksualumą? - Naujienų portalas Alfa.lt".

This webpage is encoded in 1257 (baltic), but textBox1.Text = GodLikeHTML.DocumentNode.OuterHtml; returns the distorted text - baltic diacritics are transformed into some weird several characters long strings :(

And yes, I've tried the HtmlAgilityPack forums. They do suck.

P.S. I'm no programmer, but I work on a community project and I really need to get this code working. Thanks ;}

Answer

Mikael Svenson picture Mikael Svenson · Aug 10, 2010

Actually the page is encoded with UTF-8.

GodLikeHTML.Load(GodLikeClient.OpenRead("http://www.alfa.lt"), Encoding.UTF8);

will work.

Or you could use the code in my SO answer which detects encoding from http headers or meta tags, en re-encodes properly. (It also supports gzip to minimize your download).

With the download class your code would look like:

HttpDownloader downloader = new HttpDownloader("http://www.alfa.lt",null,null);
GodLikeHTML.LoadHtml(downloader.GetPage());

Related questions

Converting string to byte array in C#

I'm converting something from VB into C#. Having a problem with the syntax of this statement: if ((searchResult.Properties["user"].Count > 0)) { profile.User = System.Text.Encoding.UTF8.GetString(searchResult.Properties["user"][0]); } I then see the following errors: Argument 1: cannot …

Read More
How can I convert a hex string to a byte array?

Can we convert a hex string to a byte array using a built-in function in C# or do I have to make a custom method for this?

Read More
How can I transform string to UTF-8 in C#?

I have a string that I receive from a third party app and I would like to display it correctly in any language using C# on my Windows Surface. Due to incorrect encoding, a piece of my string looks like …

Read More