php readdir problem with japanese language file name

Question

I have the following code

<?php
if ($handle = opendir('C:/xampp/htdocs/movies')) {
    while (false !== ($file = readdir($handle))) {
        if ($file != "." && $file != "..") {
            echo $file."<br />\n";
        }
    }
    closedir($handle);
}
?>

When it does have mb language such as japanese, it doesn't display properly instead it display like kyuukyoku Choujin R ?????~? rather then kyuukyoku Choujin R 究極超人あ～る

Anyway to make it display the correct name or make it still download-able by others?

Thanks for helping me :)

Answer 1

I can't speak definitively for PHP, but I suspect it's the same basic problem as with Python 2 had (before later adding special support for Unicode string filenames).

My belief is that PHP is dealing with filenames using the standard C library ‘open’-et-al functions, which are byte-based. On Windows (NT) these try to encode the real Unicode filename using the system codepage. That might be cp1252 (similar to ISO-8859-1) for Western machines, or cp932 (similar to Shift-JIS) on Japanese machines. For any characters that don't exist in the system codepage you will get a ‘?’ character, and you'll be unable to refer to that file.

To get around this problem PHP would have to do the same as Python 3.0 and start using Unicode strings for filenames (and everything else), using the ‘_wopen’-et-al functions to get native-Unicode access to the filenames under Windows. I expect this will happen in PHP6, but for the moment you're probably pretty much stuffed. You could change the system codepage to cp932 to get access to the filenames, but you'd still get ‘?’ characters for any other Unicode characters not in Shift-JIS, and in any case you really don't want to make your application's internal strings all Shift-JIS as it's quite a horrible encoding.

If it's your own scripts choosing how to store files, I'd strongly suggest using simple primary-key-based filenames like ‘4356’ locally, putting the real filename in a database, and serving the files up using rewrites/trailing path parts in the URL. Keeping user-supplied filenames in your own local filenames is difficult and a recipe for security disasters even without having to worry about Unicode.