UnicodeEncodeError: 'utf-8' codec can't encode character '\udc43' in position 1: surrogates not allowed

python character-encoding surrogate-pairs
Mr Moose picture Mr Moose · Nov 22, 2017 · Viewed 8k times · Source

I have a list containing placenames and I want to create another array, initially empty, and then iterate the list of placenames and fill up my empty array with these placenames.

For example, my first location is 'CHARTRIDGE' and accessing this element in my LOCARY list via LOCARY[S[0][0]] I get: 'CHARTRIDGE'

I created an empty array: LOCLIST = np.empty([len(LOCARY),1])

I then wrote a for loop to fill it up with the items from LOCARY using:

for i in range(len(LOCARY)):
    LOCLIST[i] = LOCARY[S[i][0]]

But I get the error:

UnicodeEncodeError: 'utf-8' codec can't encode character '\udc43' in position 1: surrogates not allowed

I'm wondering if it doesn't like the characters ' in the placename.

Any help would be appreciated, thank you.

Answer

Related questions

Best way to convert string to bytes in Python 3?

There appear to be two different ways to convert a string to bytes, as seen in the answers to TypeError: 'str' does not support the buffer interface Which of these methods would be better or more Pythonic? Or is it …

Read More
Working with UTF-8 encoding in Python source

Consider: $ cat bla.py u = unicode('d…') s = u.encode('utf-8') print s $ python bla.py File "bla.py", line 1 SyntaxError: Non-ASCII character '\xe2' in file bla.py on line 1, but no encoding declared; see http://…

Read More
"for line in..." results in UnicodeDecodeError: 'utf-8' codec can't decode byte

Here is my code, for line in open('u.item'): #read each line whenever I run this code it gives the following error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2892: invalid continuation byte I tried to solve …

Read More