Why does str(list)
returns how we see list on the console? How does str(list)
work? (any reference to the CPython code for str(list)
)?
>>> x = ['abc', 'def', 'ghi']
>>> str(x)
"['abc', 'def', 'ghi']"
To get the original list back from the str(list)
I have to:
>>> from ast import literal_eval
>>> x = ['abc', 'def', 'ghi']
>>> str(x)
"['abc', 'def', 'ghi']"
>>> list(str(x))
['[', "'", 'a', 'b', 'c', "'", ',', ' ', "'", 'd', 'e', 'f', "'", ',', ' ', "'", 'g', 'h', 'i', "'", ']']
>>> literal_eval(str(x))
['abc', 'def', 'ghi']
Why doesn't list(str(list))
turns the str(list)
back to the original list?
Or I could use:
>>> eval(str(x))
['abc', 'def', 'ghi']
Is literal_eval
the same as eval
? Is eval
safe to use?
How many times can I do the following? Does the code break if it keep on doing str(list(str(list))))
? E.g.
>>> x = 'abc'
>>> list(x)
['a', 'b', 'c']
>>> str(list(x))
"['a', 'b', 'c']"
>>> list(str(list(x)))
['[', "'", 'a', "'", ',', ' ', "'", 'b', "'", ',', ' ', "'", 'c', "'", ']']
>>> str(list(str(list(x))))
'[\'[\', "\'", \'a\', "\'", \',\', \' \', "\'", \'b\', "\'", \',\', \' \', "\'", \'c\', "\'", \']\']'
>>> list(str(list(str(list(x)))))
['[', "'", '[', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", 'a', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", 'b', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", 'c', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ']', "'", ']']
>>> str(list(str(list(str(list(x))))))
'[\'[\', "\'", \'[\', "\'", \',\', \' \', \'"\', "\'", \'"\', \',\', \' \', "\'", \'a\', "\'", \',\', \' \', \'"\', "\'", \'"\', \',\', \' \', "\'", \',\', "\'", \',\', \' \', "\'", \' \', "\'", \',\', \' \', \'"\', "\'", \'"\', \',\', \' \', "\'", \'b\', "\'", \',\', \' \', \'"\', "\'", \'"\', \',\', \' \', "\'", \',\', "\'", \',\', \' \', "\'", \' \', "\'", \',\', \' \', \'"\', "\'", \'"\', \',\', \' \', "\'", \'c\', "\'", \',\', \' \', \'"\', "\'", \'"\', \',\', \' \', "\'", \']\', "\'", \']\']'
>>> list(str(list(str(list(str(list(x)))))))
['[', "'", '[', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", '[', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', "'", '"', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", '"', "'", ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", 'a', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', "'", '"', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", '"', "'", ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', "'", '"', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", '"', "'", ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", 'b', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', "'", '"', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", '"', "'", ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', "'", '"', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", '"', "'", ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", 'c', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', "'", '"', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", '"', "'", ',', ' ', "'", ',', "'", ',', ' ', "'", ' ', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ']', "'", ',', ' ', '"', "'", '"', ',', ' ', "'", ']', "'", ']']
Well you have a total of 4 questions, let us go one by one.
1. Why does
str(list)
returns how we seelist
on the console? How doesstr(list)
work?
str()
and __str__()
?The str()
callable is to return a printable form of the object only! From the docs
str(object)
does not always attempt to return a string that is acceptable toeval()
; its goal is to return a printable string.
The __str__()
function in a class is called whenever you call str()
on an object. Again from the documentation
object.__str__(self)
Called by the
str()
built-in function and by the
list
callable?The list()
callable is to create a list from an iterable passed as an argument. Again from the docs
Return a
list
whose items are the same and in the same order as iterable‘s items
Thus, str(list)
gives you a printable form and list(str(list))
will iterate over the string. That is list(str(list))
will give you a list of the individual characters of the printable form of the argument passed.
A small walk-through between the nested calls,
Given list, l = ['a','b']
(Apologies for taking a smaller example than that in your question).
When you call str(l)
, it returns a printable form of the list l
, that is "['a','b']"
.
Now you can see clearly that "['a','b']"
is a string and is indeed an iterable. Now when you call list
on this i.e. list("['a','b']")
you get a weird list like ['[', "'", 'a', "'", ',', "'", 'b', "'", ']']
. Why does this happen? This happens because the string iterates over its characters, you can test this by using a dummy string,
>>> 'dummy'
'dummy'
>>> list('dummy')
['d', 'u', 'm', 'm', 'y']
Thus when you call the list
on a string you get a list of character. Note that again here, when you call str()
on list('dummy')
, you will not get back your original string 'dummy'
, so again you will have to use join
! Thus recalling the same function will NOT get you back your original object!
So, Calling str()
over a list calls the builtin __str__()
method of the list?
The answer is NO!
str()
on a list?Whenever you call str()
on an list object, the steps followed are
repr()
of each of the list element.[
at the front and another ]
at the end of the list. As you can see from the source code of the list object in cpython on github. Going through the source code of cpython in hg.python, which is more clear, you can see the following three comments. (Thanks to Ashwini for the link on that particular code)
/* Do repr() on each element. Note that this may mutate the list, so must refetch the list size on each iteration. */ line (382) /* Add "[]" decorations to the first and last items. */ line (398) /* Paste them all together with ", " between. */ line (418)
These correspond to the points I mentioned above.
repr()
?repr()
prints the string representation of all the objects. Again from the documentation
Return a string containing a printable representation of an object.
and also note this sentence!
For many types, this function makes an attempt to return a string that would yield an object with the same value when passed to
eval()
, otherwise the representation is a string enclosed in angle brackets that contains the name of the type of the object together with additional information often including the name and address of the object.
And now your second question here,
2. Why doesn't
list(str(list))
turns thestr(list)
back to the original list?
Internally, str(list)
actually creates the repr()
representation of the list object. So to get back the list after calling str
on the list, you actually need to do eval
on it and not a list
call.
But we all know that eval
is evil, so what is/are the workaround(s)?
literal_eval
The first work-around would be to use ast.literal_eval
. That brings us to your 3rd question,
3. Is
literal_eval()
the same aseval()
? Iseval()
safe to use?
ast.literal_eval()
is safe unlike the eval()
function. The docs themselves mention that it is safe --
Safely evaluate an expression node or a string containing a Python literal or container display
Another workaround can be done using str.split()
>>> x = ['abc', 'def', 'ghi']
>>> a = str(x)
>>> a[2:-2].split("', '")
['abc', 'def', 'ghi']
This is just a simple way to do that for a list of strings. For a list of integers you will need map
.
>>> x = [1,2,3]
>>> a =str(x)
>>> list(map(int,a[1:-1].split(', '))) # No need for list call in Py2
[1, 2, 3]
Thus unlike literal_eval
these are simple hacks given that you know the elements of the list. If they are heterogeneous in nature like [1, "a", True]
then you will have to loop through the split list and discover the element type and then convert it and append the converted element to a final list.
Another place where this fails is when the string itself contains quote characters. As mentioned by nneonneo in a comment
The
str.split
solution is very fragile and will break if the input contains e.g. strings that contain", "
, or tuples, or other lists, ... It is much better to useast.literal_eval
because that will deal with all the subtleties of the syntax.
And for your final question,
4. Does the code break if you do
str(list(str(list))))
again and again?
Not really. The output will grow longer and longer as each time you are creating a list
of a str
and then again getting the printable version of it. The limitation is your physical machine's limitation only. (which will be soon reached as each step the string length is multiplied by 5.)