[已解决]xpath 爬取知，乎 ，的专栏标题，输出列表为空

lqq123

结果是个空列表，xpath定位浏览器提取的和自己写的都试了，结果是空列表

1. import requests
   
2. from lxml import etree
   
3. 
   
4. url = 'https://zhuanlan.zhihu.com/pypcfx'
   
5. headers = {'user-agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36'}
   
6. html = requests.get(url, headers=headers)
   
7. data = etree.HTML(html.text)
   
8. title = data.xpath('//div[@class="Card css-ny4o71"]/div[@class="css-8txec3"]//h2/a/text()')
   
9. print(title)
   

复制代码

最佳答案

月排行榜 / 总排行榜

wp231957

2021-5-31 16:42:20

这玩意是通过AJAX传过来的  你试试

1. url = 'https://www.zhihu.com/api/v4/columns/pypcfx/pinned-items'
   
2. headers = {'user-agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36'}
   
3. html = requests.get(url, headers=headers).json()["data"][0]["content"]
   
4. print(html)
   

复制代码

跳转到最佳答案楼层

wp231957

这玩意是通过AJAX传过来的  你试试

1. url = 'https://www.zhihu.com/api/v4/columns/pypcfx/pinned-items'
   
2. headers = {'user-agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36'}
   
3. html = requests.get(url, headers=headers).json()["data"][0]["content"]
   
4. print(html)
   

复制代码