Scrapy 未自动初始化sel变量，如何手动
网上都说 response会自动初始化一个sel变量，但是我的没有 提示NameError: name 'sel' is not definedpython 3.8.4 pip安装的scrapy
具体代码
>>> sel.xpath("//ul/li/h4/text()").extract()
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'sel' is not defined、
请发完整代码 C:\Users\Desktop\tutorial>scrapy shell "https://www.dmozdir.org/Category/?SmallPath=20"
2020-08-22 16:19:48 INFO: Scrapy 2.3.0 started (bot: tutorial)
2020-08-22 16:19:48 INFO: Versions: lxml 4.5.2.0, libxml2 2.9.5, cssselect 1.1.0, parsel 1.6.0, w3lib 1.22.0, Twisted 20.3.0, Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) , pyOpenSSL 19.1.0 (OpenSSL 1.1.1g21 Apr 2020), cryptography 3.0, Platform Windows-10-10.0.18362-SP0
2020-08-22 16:19:48 DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor
2020-08-22 16:19:48 INFO: Overridden settings:
{'BOT_NAME': 'tutorial',
'DUPEFILTER_CLASS': 'scrapy.dupefilters.BaseDupeFilter',
'LOGSTATS_INTERVAL': 0,
'NEWSPIDER_MODULE': 'tutorial.spiders',
'ROBOTSTXT_OBEY': True,
'SPIDER_MODULES': ['tutorial.spiders']}
2020-08-22 16:19:48 INFO: Telnet Password: 6fbd425e0ac85fd6
2020-08-22 16:19:48 INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.telnet.TelnetConsole']
2020-08-22 16:19:49 INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2020-08-22 16:19:49 INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2020-08-22 16:19:49 INFO: Enabled item pipelines:
[]
2020-08-22 16:19:49 INFO: Telnet console listening on 127.0.0.1:6023
2020-08-22 16:19:49 INFO: Spider opened
2020-08-22 16:19:49 DEBUG: Crawled (200) <GET https://www.dmozdir.org/robots.txt> (referer: None)
2020-08-22 16:19:49 DEBUG: Crawled (200) <GET https://www.dmozdir.org/Category/?SmallPath=20> (referer: None)
Available Scrapy objects:
scrapy scrapy module (contains scrapy.Request, scrapy.Selector, etc)
crawler <scrapy.crawler.Crawler object at 0x00000275FC858280>
item {}
request <GET https://www.dmozdir.org/Category/?SmallPath=20>
response <200 https://www.dmozdir.org/Category/?SmallPath=20>
settings <scrapy.settings.Settings object at 0x00000275FC8580A0>
spider <DmozSpider 'dmoz' at 0x275fcd18a90>
Useful shortcuts:
fetch(url[, redirect=True]) Fetch URL and update local objects (by default, redirects are followed)
fetch(req) Fetch a scrapy.Request and update local objects
shelp() Shell help (print this help)
view(response) View response in a browser
>>> sel.xpath("//ul/li/h4/text()").extract()
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'sel' is not defined
>>> python3.8.4不是有问题的么？
不兼容中文，现在问题解决了么？
很多人遇到这个问题，现在都升级到3.8.5或者降级到其他版本了。你没遇到这个问题？ >>> response.xpath("//title")
[<Selector xpath='//title' data='<title>视频影视-娱乐休闲-目录分类-DMOZ中文网站分类目录</t...'>]
>>> response.xpath("//title/text()").extract
<bound method SelectorList.getall of [<Selector xpath='//title/text()' data='视频影视-娱乐休闲-目录分类-DMOZ中文网站分类目录'>]>
>>> response.xpath("//title/text()").extract()
['视频影视-娱乐休闲-目录分类-DMOZ中文网站分类目录']
>>> response.body
>>> sel.xpath("//ul/li/text()").extract()
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'sel' is not defined
>>> sel.xpath()
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'sel' is not defined Twilight6 发表于 2020-8-22 16:55
请发完整代码
代码在楼下 sunrise085 发表于 2020-8-22 17:01
python3.8.4不是有问题的么？
不兼容中文，现在问题解决了么？
很多人遇到这个问题，现在都升级到3.8.5或 ...
我好像没遇到太多问题 就没升级 我升级一下试试 sunrise085 发表于 2020-8-22 17:01
python3.8.4不是有问题的么？
不兼容中文，现在问题解决了么？
很多人遇到这个问题，现在都升级到3.8.5或 ...
看错了 我的就是3，85 感谢各位大佬已解决手动sel=response 就搞定了
页:
[1]