Why should we NOT use sys.setdefaultencoding("utf-8") in a py script?

python encoding utf-8 python-2.x sys
mlzboy picture mlzboy · Sep 30, 2010 · Viewed 201.5k times · Source

I have seen few py scripts which use this at the top of the script. In what cases one should use it?

import sys
reload(sys)
sys.setdefaultencoding("utf-8")

Answer

pyfunc picture pyfunc · Sep 30, 2010

As per the documentation: This allows you to switch from the default ASCII to other encodings such as UTF-8, which the Python runtime will use whenever it has to decode a string buffer to unicode.

This function is only available at Python start-up time, when Python scans the environment. It has to be called in a system-wide module, sitecustomize.py, After this module has been evaluated, the setdefaultencoding() function is removed from the sys module.

The only way to actually use it is with a reload hack that brings the attribute back.

Also, the use of sys.setdefaultencoding() has always been discouraged, and it has become a no-op in py3k. The encoding of py3k is hard-wired to "utf-8" and changing it raises an error.

I suggest some pointers for reading:

Related questions

Working with UTF-8 encoding in Python source

Consider: $ cat bla.py u = unicode('d…') s = u.encode('utf-8') print s $ python bla.py File "bla.py", line 1 SyntaxError: Non-ASCII character '\xe2' in file bla.py on line 1, but no encoding declared; see http://…

Read More
How do I check if a string is unicode or ascii?

What do I have to do in Python to figure out which encoding a string has?

Read More
Changing default encoding of Python?

I have many "can't encode" and "can't decode" problems with Python when I run my applications from the console. But in the Eclipse PyDev IDE, the default character encoding is set to UTF-8, and I'm fine. I searched around for …

Read More