Doze~Z 发表于 2022-5-8 17:45:12

python爬虫动态网页

各位鱼友们：
        怎么才能爬取像这种动态网页的https://www.lixiang.com/app/service.html里的内容，尝试过selenium，但是无法定位它的元素，不知道怎么搞，请问鱼友们有会的吗？感谢指教

kerln888 发表于 2022-5-9 08:30:33

帮顶{:10_256:}{:10_256:}{:10_256:}

YunGuo 发表于 2022-5-13 21:41:17

网页数据没有在html页面，抓包请求数据，写代码请求这个url就能拿到数据：https://api-web.lixiang.com/mall-unit-api/v1-0/service-centers?types=RETAIL%2CDELIVER%2CAFTERSALE%2CSPRAY%2CTEMPORARY_EXHIBITION%2CTEMPORARY_AFTERSALE_SUPPORT&sortType=CITY&storeEffectiveStatus=
import requests

url = 'https://api-web.lixiang.com/mall-unit-api/v1-0/service-centers?types=RETAIL%2CDELIVER%2CAFTERSALE%2CSPRAY%2CTEMPORARY_EXHIBITION%2CTEMPORARY_AFTERSALE_SUPPORT&sortType=CITY&storeEffectiveStatus='
headers = {
    'accept': 'application/json, text/javascript, */*; q=0.01',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36',
}
response = requests.get(url, headers=headers)
print(response.json())

17860303637 发表于 2022-5-14 08:19:13

帮顶{:5_109:}
页: [1]
查看完整版本: python爬虫动态网页