定位不了需要的属性，用selenium得到的html和网页看的源码也不一样
# usr/bin/env python# -*- coding:utf-8 -*-
#需求爬取该代码中url对应网页中每个黄花菜资料的详情文本和图片
importrequests
from lxml importetree
import os
if not os.path.exists('./黄花'):
os.mkdir('./黄花')
url= 'http://www.qdtcp.com/story/huanghuacai/'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36 Edg/100.0.1185.50'
}
html = requests.get(url=url,headers=headers).text.encode('utf-8')
tree = etree.HTML(html)
div_list = tree.xpath('//*[@id="wp"]/div/div/div/div/div')
print(div_list)
for div in div_list:
href = div.xpath('./div/dl/dt/a/@href')
print(href)
xpath表达式得到的列表是空的
试过用selenium，但是page_source得到的网页 和浏览器上面的源码不同
蹲一个大佬解惑 靠，我把那个.encode('utf-8') 去掉就好了
页:
[1]