关于第63课scrapy

holistic杀手

1. import scrapy
   
2. 
   
3. class DmozSpider(scrapy.Spider):
   
4.     name = "dmoz"
   
5.     allowed_domains = ['http://dmoztools.net/']#爬取的范围
   
6.     start_urls = [
   
7.         'http://dmoztools.net/Computers/Programming/Languages/Python/Books/',
   
8.         'http://dmoztools.net/Computers/Programming/Languages/Python/Resources/'
   
9.         ]
   
10.     def parse(self,response):
    
11.         sel = scrapy.selector.Selector(response)
    
12.         sites = sel.xpath('//div[@class="title-and-desc"]/a')
    
13.         for site in sites:
    
14.             title = site.xpath('div/text()').extract()
    
15.             link = site.xpath('@href').extract()
    
16.             desc = site.xpath('text()').extract()
    
17.             print(title,link,desc)

复制代码

抓标题的标签为什么前面还要加div?
desc是什么?

ba21

百度xpath的用法先

holistic杀手

ba21 发表于 2018-1-7 12:18
百度xpath的用法先