How do I check if a string is unicode or ascii?

python unicode encoding utf-8
TIMEX picture TIMEX · Feb 13, 2011 · Viewed 362.2k times · Source

What do I have to do in Python to figure out which encoding a string has?

Answer

Greg Hewgill picture Greg Hewgill · Feb 13, 2011

In Python 3, all strings are sequences of Unicode characters. There is a bytes type that holds raw bytes.

In Python 2, a string may be of type str or of type unicode. You can tell which using code something like this:

def whatisthis(s):
    if isinstance(s, str):
        print "ordinary string"
    elif isinstance(s, unicode):
        print "unicode string"
    else:
        print "not a string"

This does not distinguish "Unicode or ASCII"; it only distinguishes Python types. A Unicode string may consist of purely characters in the ASCII range, and a bytestring may contain ASCII, encoded Unicode, or even non-textual data.

Related questions

ElementTree and unicode

I have this char in an xml file: <data> <products> <color>fumè</color> </product> </data> I try to generate an instance of ElementTree with the following code: string_…

Read More
Python - dealing with mixed-encoding files

I have a file which is mostly UTF-8, but some Windows-1252 characters have also found their way in. I created a table to map from the Windows-1252 (cp1252) characters to their Unicode counterparts, and would like to use it to …

Read More
UnicodeDecodeError when performing os.walk

I am getting the error: 'ascii' codec can't decode byte 0x8b in position 14: ordinal not in range(128) when trying to do os.walk. The error occurs because some of the files in a directory have the 0x8b (non-utf8) …

Read More