|
马上注册,结交更多好友,享用更多功能^_^
您需要 登录 才可以下载或查看,没有账号?立即注册
x
# dmoztools_spider.py
import scrapy
class DmoztoolsSpider(scrapy.Spider):
name = 'dmoztools'
allowed_domains = ['dmoztools.net']
start_urls = [
'http://dmoztools.net/Computers/Programming/Languages/Python/Books/',
'http://dmoztools.net/Computers/Programming/Languages/Python/Resources/'
]
def parse(self, response):
sel = scrapy.selector.Selcetor(response) # 初始化变量sel,类初始化成一个selector对象(在shell中shell帮我们初始化好了变sel量,在代码中需要我们自己初始化)
sites = sel.xpath('//ul[@class="direcotry-url"]/li')
for site in sites:
title = site.xpath('a/text()').extract()
link = site.xpath('a/@href').extract()
desc = site.xpath('text()').extract()
print(title, link, desc)
红色是小甲鱼在视频中的一句话,求大神指点帮助理解,谢谢! |
|