mb_detect_encoding detects ASCII as UTF-8?

php string utf-8 ascii multibyte
Cobra_Fast picture Cobra_Fast · Apr 30, 2013 · Viewed 19.1k times · Source

I'm trying to automatically convert imported IPTC metadata from images to UTF-8 for storage in a database based on the PHP mb_ functions.

Currently it looks like this:

$val = mb_convert_encoding($val, 'UTF-8', mb_detect_encoding($val));

However, when mb_detect_encoding() is supplied an ASCII string (special characters in the Latin1-fields from 192-255) it detects it as UTF-8, hence in the following attempt to convert everything to proper UTF-8 all special characters are removed.

I tried writing my own method by looking for Latin1 values and if none occured I would go on to letting mb_detect_encoding decide what it is. But I stopped midway when I realized that I can't be sure that other encoding don't use the same byte values for other things.

So, is there a way to properly detect ASCII to feed to mb_convert_encoding as the source encoding?

Answer

Cobra_Fast picture Cobra_Fast · Apr 30, 2013

Specifying a custom order, where ASCII is detected first, works.

mb_detect_encoding($val, 'ASCII,UTF-8,ISO-8859-15');

For completeness, the list of available encodings is at http://www.php.net/manual/en/mbstring.supported-encodings.php

Related questions

php iconv translit for removing accents: not working as excepted?

consider this simple code: echo iconv('UTF-8', 'ASCII//TRANSLIT', 'è'); it prints `e instead of just e do you know what I am doing wrong? nothing changed after adding setlocale setlocale(LC_COLLATE, 'en_US.utf8'); echo …

Read More
PHP mb_substr() not working correctly?

This code print mb_substr('éxxx', 0, 1); prints an empty space :( It is supposed to print the first character, é. This seems to work however: print mb_substr('éxxx', 0, 2); But it's not right, because (0, 2) means 2 characters...

Read More
Multi-byte safe wordwrap() function for UTF-8

PHP's wordwrap() function doesn't work correctly for multi-byte strings like UTF-8. There are a few examples of mb safe functions in the comments, but with some different test data they all seem to have some problems. The function should take …

Read More