大佬们,本人萌新一枚,最近跟着小甲鱼老师的python爬虫课程爬取网易云评论时,发现request库不大好使了,爬出来会报错:{"msg":"参数错误","code":400},请问这是什么原因导致的错误呢?莫非是网易云的params和encSEcKey参数加密了?代码如下,感觉代码应该没啥问题呢。import requests
def get_comments(url):
name_id = url.split("=")[1]
headers = {"user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36",
"referer":"https://music.163.com/song?id={}".format(name_id),
"origin":"https://music.163.com"}
params=
"PaXPWdb3bwNmS9m9VvBhueMpH6Lg7Xr2x2pU0hAv3M5Luh5ff/HfMArdNMJbSu9T3l6KdCMWR5/4oFDcGS09vxJOcHDB/kFUQ8cftlgeiSivjTE0YMYWNXifGuBARt5GvJlS5COncvZKZhDyov+IKuO/dRWGDbzoOdn+tGjRsXTsozm3A/eKk9rUVqLf5oELBfwVx5D2pJR3diMrrug7+sG64tAZfjQMoYw8gEmxnfLyUGqOaVlEELVZJun3Ue0gDSgCOCuuLZGHd3zttqPdlg=="
encSEcKey = "7bbe72cb0f192959ff6bd3809cd492a9e274eef1b05c4c7cadb451a6abebca7fae6e0ea60d1842c19757e931b4996d16cff09201a112d6b95cf349502338662a7ca9fa0ad2a67e07bb15cb2dcf7ec732053fc3a41302d69be0a528558920beaea595fa3ba908f08b9fac0948cde63f6eeff582c6555d17388bf2a954d71af5b2"
data = {"params":params,"encSEcKey":encSEcKey}
target_url = "https://music.163.com/api/comment/resource/comments/get?csrf_token="
res = requests.post(target_url, headers=headers,data=data)
return res
def main():
url = input("请输入链接地址:")
res = get_comments(url)
#print(res.status_code)
with open("res.txt","w",encoding="utf-8") as f:
f.write(res.text)
if __name__ == "__main__":
main()
然后我又尝试使用selenium库来实现,代码如下:from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
service=Service('/usr/local/bin/chromedriver')
driver = webdriver.Chrome(service=service)
target_url = 'https://music.163.com/#/song?id=4466775'
driver.get(target_url)
#切换到嵌套网页
driver.switch_to.frame(0)
#css选择器 根据标签属性提取内容
lis = driver.find_elements(By.CSS_SELECTOR,'.itm')
for each in lis:
try:
comment_elements = each.find_elements(By.CSS_SELECTOR,'.cnt')
for comment_element in comment_elements:
comment = comment_element.text
print(comment)
except Exception as e:
print("异常:",e)
driver.close()
然后就成功的爬到了评论,但是贼卡,要等好久才出来。所以这个是不是意味着selenium比request好用呀?这两个库的优缺点分别是啥捏? |