BeautifulSoup - TypeError: 'NoneType' object is not callable
I need to make my code backwards compatible with python2.6 and BeautifulSoup 3. My code was written using python2.7 and at this case using BS4. But when I try to run it at squeezy server, I get this error (it has python2.6 and bs3):
try:
from bs4 import BeautifulSoup
except ImportError:
from BeautifulSoup import BeautifulSoup
gmp = open(fname, 'r')
soup = BeautifulSoup(gmp)
p = soup.body.div.find_all('p')
p = soup.body.div.find_all('p')
TypeError: 'NoneType' object is not callable
If I change to:
p = soup.body.div.findAll('p')
then I get this error:
p = soup.body.div.findAll('p')
TypeError: 'NoneType' object is not callable
Update of thrown error
File "/home/user/openerp/7.0/addons/my_module/models/gec.py", line 401, in parse_html_data
p = soup.body.div.findAll('p') #used findAll instead of find_all for backwards compatability to bs3 version
TypeError: 'NoneType' object is not callable
Either way, both approaches work on my Ubuntu with python2.7 and bs4, but not on squeezy. Is there any other difference between those versions that I don't see/know and gives me this error?
Answer
You are using BeautifulSoup 3, but are using BeautifulSoup 4 syntax.
Your fallback is at fault here:
try:
from bs4 import BeautifulSoup
except ImportError:
from BeautifulSoup import BeautifulSoup
If you want to use either version 3 or 4, stick to version 3 syntax:
p = soup.body.div.findAll('p')
because find_all is not a valid method in BeautifulSoup 3, so it is instead interpreted as a tag search. There is no find_all tag in your HTML, so None is returned, which you then try to call.
Next, the parser used by BeautifulSoup 3 will respond differently to broken or incomplete HTML. If you have lxml installed on Ubuntu, then that'll be used as the default parser, and it'll insert a missing <body> tag for you. BeautifulSoup 3 may leave that out.
I strongly urge you to instead remove the fallback, and stick with BeautifulSoup version 4 only. Version 3 has been discontinued years ago, and contains unfixed bugs. BeautifulSoup 4 also offers additional features you may want to make use of.
BeautifulSoup is pure Python and easily installed into a virtual environment on any platform supported by Python. You are not tied to the system-supplied package here.
On Debian Squeezy for example, you'd be stuck with BeautifulSoup 3.1.0, and even the BeautifulSoup developers do not want you to use it!. Your problem with findAll almost certainly stems from using that release.