[已解决]xpath 爬取知，乎 ，的专栏标题，输出列表为空

lqq123

结果是个空列表，xpath定位浏览器提取的和自己写的都试了，结果是空列表

import requests
from lxml import etree

url = 'https://zhuanlan.zhihu.com/pypcfx'
headers = {'user-agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36'}
html = requests.get(url, headers=headers)
data = etree.HTML(html.text)
title = data.xpath('//div[@class="Card css-ny4o71"]/div[@class="css-8txec3"]//h2/a/text()')
print(title)

最佳答案

月排行榜 / 总排行榜

wp231957

2021-5-31 16:42:20

这玩意是通过AJAX传过来的  你试试

url = 'https://www.zhihu.com/api/v4/columns/pypcfx/pinned-items'
headers = {'user-agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36'}
html = requests.get(url, headers=headers).json()["data"][0]["content"]
print(html)

跳转到最佳答案楼层

wp231957

这玩意是通过AJAX传过来的  你试试

url = 'https://www.zhihu.com/api/v4/columns/pypcfx/pinned-items'
headers = {'user-agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36'}
html = requests.get(url, headers=headers).json()["data"][0]["content"]
print(html)