Scrapy spider not found error

python scrapy

user199421 · Mar 26, 2012 · Viewed 42.3k times · Source

This is Windows 7 with python 2.7

I have a scrapy project in a directory called caps (this is where scrapy.cfg is)

My spider is located in caps\caps\spiders\campSpider.py

I cd into the scrapy project and try to run

scrapy crawl campSpider -o items.json -t json

I get an error that the spider can't be found. The class name is campSpider

...
    spider = self.crawler.spiders.create(spname, **opts.spargs)
  File "c:\Python27\lib\site-packages\scrapy-0.14.0.2841-py2.7-win32.egg\scrapy\spidermanager.py", l
ine 43, in create
    raise KeyError("Spider not found: %s" % spider_name)
KeyError: 'Spider not found: campSpider'

Am I missing some configuration item?

Answer

Make sure you have set the "name" property of the spider. Example:

class campSpider(BaseSpider):
   name = 'campSpider'

Without the name property, the scrapy manager will not be able to find your spider.

Scrapy spider not found error

Answer

Related questions