爬虫 http error 401求助
爬取内容的网站request header如上图所示。
def url_open(url):
req = urllib.request.Request(url)
req.add_header('user-agent',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36')
req.add_header('cookie',
'''sajssdk_2015_cross_new_user=1; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%22176e5ddd74c256-0ac4fc023ec3e8-c791039-2073600-176e5ddd74d2a3%22%2C%22%24device_id%22%3A%22176e5ddd74c256-0ac4fc023ec3e8-c791039-2073600-176e5ddd74d2a3%22%2C%22props%22%3A%7B%22%24latest_referrer%22%3A%22%22%2C%22%24latest_referrer_host%22%3A%22%22%2C%22%24latest_traffic_source_type%22%3A%22%E7%9B%B4%E6%8E%A5%E6%B5%81%E9%87%8F%22%2C%22%24latest_search_keyword%22%3A%22%E6%9C%AA%E5%8F%96%E5%88%B0%E5%80%BC_%E7%9B%B4%E6%8E%A5%E6%89%93%E5%BC%80%22%7D%7D; api_server_id=1oxYuM0c; sensorsdata-token=5DKLZX4BEiDNPYHUzt0R0yIzmNTI9S7uNWEGR0JOi49aZYMpUtbOiP2tdKPm59k3V8V8MKthQDMxRyyziAHx93ftmLRzUkGeQbhBuQlD65VhK972HiBF7GRSqznPdWOj; sajssdk_2015_new_user_sensors-web=1; sa_jssdk_2015_sensors-web=%7B%22distinct_id%22%3A%22vandream%3A60%22%2C%22props%22%3A%7B%22customer_id%22%3A%22vandream%22%2C%22project_name%22%3A%22production%22%2C%22username%22%3A%22zhangyu%40vandream.com%22%2C%22isDemo%22%3Afalse%2C%22version1%22%3A%221.17%22%2C%22version2%22%3A%221.17.5036%22%7D%7D; csrftoken=oC7QFB15wR6THxWm84c8fMcLaYfVIXfc''')
......
因为该网站需要登录，所以添加了cookie作为身份验证。但是在运行代码时还是提示了401错误。
求大佬们帮忙看看是什么原因？ 1.打开时必需有headers={'User-Agent':'Mozilla/5.0 (Windows NT 6.3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4408.5 Safari/537.36'}
2可能有反爬虫 代码发全
页:
[1]