Utf8_general_ci or utf8mb4 or...?

utf-8 localization utf-16 utf-32 utf8mb4
Wolfpack'08 picture Wolfpack'08 · Jul 18, 2012 · Viewed 30.5k times · Source

utf16 or utf32? I'm trying to store content in a lot of languages. Some of the languages use double-wide fonts (for example, Japanese fonts are frequently twice as wide as English fonts). I'm not sure which kind of database I should be using. Any information about the differences between these four charsets...

Answer

Ignacio Vazquez-Abrams picture Ignacio Vazquez-Abrams · Jul 18, 2012

MySQL's utf32 and utf8mb4 (as well as standard UTF-8) can directly store any character specified by Unicode; the former is fixed size at 4 bytes per character whereas the latter is between 1 and 4 bytes per character.

utf8mb3 and the original utf8 can only store the first 65,536 codepoints, which will cover CJVK (Chinese, Japanese, Vietnam, Korean), and use 1 to 3 bytes per character.

utf16 uses 2 bytes for the first 65,536 codepoints, and 4 bytes for everything else.

As for fonts, that's strictly a visual thing.

"The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)"

See also MySQL documentation for Unicode support.

Related questions

how to change source file encoding in csharp project (visual studio / msbuild machine)?

Is there a way to force VS to use Unicode always, instead of weird ISO-something? I'm working on a winapp csproject using Visual Studio 2008 on Vista (Polish locale). The problem appears when I build the project on Win Server 2003 (English …

Read More
Encode String to UTF-8

I have a String with a "ñ" character and I have some problems with it. I need to encode this String to UTF-8 encoding. I have tried it by this way, but it doesn't work: byte ptext[] = myString.getBytes(); String value = …

Read More
Working with UTF-8 encoding in Python source

Consider: $ cat bla.py u = unicode('d…') s = u.encode('utf-8') print s $ python bla.py File "bla.py", line 1 SyntaxError: Non-ASCII character '\xe2' in file bla.py on line 1, but no encoding declared; see http://…

Read More