[已解决]网络爬虫

皮皮小发

C:\tutoria>scrapy crawl dmoz
Traceback (most recent call last):
  File "c:\python27\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "c:\python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\Scripts\scrapy.exe\__main__.py", line 9, in <module>
  File "c:\python27\lib\site-packages\scrapy\cmdline.py", line 148, in execute
    cmd.crawler_process = CrawlerProcess(settings)
  File "c:\python27\lib\site-packages\scrapy\crawler.py", line 243, in __init__
    super(CrawlerProcess, self).__init__(settings)
  File "c:\python27\lib\site-packages\scrapy\crawler.py", line 134, in __init__
    self.spider_loader = _get_spider_loader(settings)
  File "c:\python27\lib\site-packages\scrapy\crawler.py", line 330, in _get_spider_loader
    return loader_cls.from_settings(settings.frozencopy())
  File "c:\python27\lib\site-packages\scrapy\spiderloader.py", line 61, in from_settings
    return cls(settings)
  File "c:\python27\lib\site-packages\scrapy\spiderloader.py", line 25, in __init__
    self._load_all_spiders()
  File "c:\python27\lib\site-packages\scrapy\spiderloader.py", line 47, in _load_all_spiders
    for module in walk_modules(name):
  File "c:\python27\lib\site-packages\scrapy\utils\misc.py", line 71, in walk_modules
    submod = import_module(fullpath)
  File "c:\python27\lib\importlib\__init__.py", line 37, in import_module
    __import__(name)
  File "C:\tutoria\tutoria\spiders\dmoz_spider.py", line 4, in <module>
    class DmozSpider(scrapy.spider):
AttributeError: 'module' object has no attribute 'spider'
在Windows的命令窗口输入scrapy crawl dmoz 为什么会显示上面这个，scarpy爬不了，求指教

最佳答案

月排行榜 / 总排行榜

ba21

2017-11-6 22:23:36

  class DmozSpider(scrapy.spider):
AttributeError: 'module' object has no attribute 'spider'


scrapy.Spider

跳转到最佳答案楼层

ba21

  class DmozSpider(scrapy.spider):
AttributeError: 'module' object has no attribute 'spider'


scrapy.Spider

皮皮小发

ba21 发表于 2017-11-6 22:23
class DmozSpider(scrapy.spider):
AttributeError: 'module' object has no attribute 'spider'

2017-11-06 23:47:22 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: tutoria)
2017-11-06 23:47:22 [scrapy.utils.log] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutoria.spiders', 'SPIDER_MODULES': ['tutoria.spiders'], 'ROBOTSTXT_OBEY': True, 'BOT_NAME': 'tutoria'}
2017-11-06 23:47:22 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.logstats.LogStats',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.corestats.CoreStats']
Unhandled error in Deferred:
2017-11-06 23:47:22 [twisted] CRITICAL: Unhandled error in Deferred:

2017-11-06 23:47:22 [twisted] CRITICAL:
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\twisted\internet\defer.py", line 1386, in _inlineCallbacks
    result = g.send(result)
  File "c:\python27\lib\site-packages\scrapy\crawler.py", line 95, in crawl
    six.reraise(*exc_info)
  File "c:\python27\lib\site-packages\scrapy\crawler.py", line 77, in crawl
    self.engine = self._create_engine()
  File "c:\python27\lib\site-packages\scrapy\crawler.py", line 102, in _create_engine
    return ExecutionEngine(self, lambda _: self.stop())
  File "c:\python27\lib\site-packages\scrapy\core\engine.py", line 69, in __init__
    self.downloader = downloader_cls(crawler)
  File "c:\python27\lib\site-packages\scrapy\core\downloader\__init__.py", line 88, in __init__
    self.middleware = DownloaderMiddlewareManager.from_crawler(crawler)
  File "c:\python27\lib\site-packages\scrapy\middleware.py", line 58, in from_crawler
    return cls.from_settings(crawler.settings, crawler)
  File "c:\python27\lib\site-packages\scrapy\middleware.py", line 34, in from_settings
    mwcls = load_object(clspath)
  File "c:\python27\lib\site-packages\scrapy\utils\misc.py", line 44, in load_object
    mod = import_module(module)
  File "c:\python27\lib\importlib\__init__.py", line 37, in import_module
    __import__(name)
  File "c:\python27\lib\site-packages\scrapy\downloadermiddlewares\retry.py", line 20, in <module>
    from twisted.web.client import ResponseFailed
  File "c:\python27\lib\site-packages\twisted\web\client.py", line 42, in <module>
    from twisted.internet.endpoints import HostnameEndpoint, wrapClientTLS
  File "c:\python27\lib\site-packages\twisted\internet\endpoints.py", line 41, in <module>
    from twisted.internet.stdio import StandardIO, PipeAddress
  File "c:\python27\lib\site-packages\twisted\internet\stdio.py", line 30, in <module>
    from twisted.internet import _win32stdio
  File "c:\python27\lib\site-packages\twisted\internet\_win32stdio.py", line 9, in <module>
    import win32api
ImportError: No module named win32api
这个又怎么弄

ba21

皮皮小发 发表于 2017-11-6 23:48
2017-11-06 23:47:22 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: tutoria)
2017-11-06 23:47 ...

安装 win32api 模块