[已解决]爬虫412如何解决

a豪

爬取 阳光高考 网站 提示412

最佳答案

月排行榜 / 总排行榜

suchocolate

2022-12-11 10:01:53

f12观察了一下：
第一次访问时，网站返回412时，要求浏览器设置两个cookie项。
第二次访问时，却带了3个cookie项，估计得研究一下js是怎么生成的第3项。

跳转到最佳答案楼层

suchocolate

发一下你的代码

a豪

suchocolate 发表于 2022-12-9 14:07
发一下你的代码

import requests
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36',
    'Cookie': 'goaYXsyEWlxdO=5lfEO882FR5N405ZV8ZlXrz7QBaeaa1itSJTzCmPlhP5cNQF_nokgatyPjCjRDlr3lrYygFZPwhpU4ODz_OaO.A; zg_did=%7B%22did%22%3A%20%221849f5a46a25e-02e8dcc83e8f1b-3e604809-144000-1849f5a46a33f4%22%7D; _ga=GA1.3.451236072.1669120805; CHSICC01=!rusBFxxRpzo5SBnzYxYLahOzddj6Yw+Da909AHo+YWwAmp4KBwANcP/n9H9HH0SkgD9wyofcIjnuhg==; 35067579362dd9cd78=c58ea6476bd9c117ebaa07bb7def3eea; XSRF-CCKTOKEN=eb9bc4943bb2647da7e389116b3b930b; CHSICC_CLIENTFLAGGAOKAO=1d331ce10098465acd742c22f4f17cda; CHSICC_CLIENTFLAGCHSI=d7652cf64d259c53b2b39a438ad266cd; _gid=GA1.3.101851822.1670573755; _gat_gtag_UA_100524_6=1; JSESSIONID=CCFE1665C964CD5265D4B04FF019915C; zg_a5f21d6abc554e3c9e4c17aade025fd8=%7B%22sid%22%3A%201670573755568%2C%22updated%22%3A%201670573758940%2C%22info%22%3A%201670573755572%2C%22superProperty%22%3A%20%22%7B%7D%22%2C%22platform%22%3A%20%22%7B%7D%22%2C%22utm%22%3A%20%22%7B%7D%22%2C%22referrerDomain%22%3A%20%22gaokao.chsi.com.cn%22%7D; goaYXsyEWlxdP=53yhXXCi14.gqqqDleI1AQGLSbTU0LhwRWO1GiVKc4q32m6OF4pLVOfAHtadxs7cTqXt20sOGSKVkNUkwQ7AMy7qgzpXjGtmPAFSLkO7KqgSwsOSxxt8QmUDUsua.Sd7lQzYBOWVJLG9omBI6JIl7KOmKmikn5l.WkxL_LQuRJL4O9oxlt0zMieCubf2H5RkNDxf4iZcLUH23kCq5eSCNAbN4V3Z5mIwuahqGEnieVB7YAKnrx_NYyhjmxDSPsaIaHuNJrKLZPJwPlRLXNDEAmo'
}


url = 'https://gaokao.chsi.com.cn/sch/schoolInfoMain--schId-1.dhtml'
resp = requests.get(url=url,headers=headers)

a豪

suchocolate 发表于 2022-12-9 14:07
发一下你的代码

import requests
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36',
    'Cookie': 'goaYXsyEWlxdO=5lfEO882FR5N405ZV8ZlXrz7QBaeaa1itSJTzCmPlhP5cNQF_nokgatyPjCjRDlr3lrYygFZPwhpU4ODz_OaO.A; zg_did=%7B%22did%22%3A%20%221849f5a46a25e-02e8dcc83e8f1b-3e604809-144000-1849f5a46a33f4%22%7D; _ga=GA1.3.451236072.1669120805; CHSICC01=!rusBFxxRpzo5SBnzYxYLahOzddj6Yw+Da909AHo+YWwAmp4KBwANcP/n9H9HH0SkgD9wyofcIjnuhg==; 35067579362dd9cd78=c58ea6476bd9c117ebaa07bb7def3eea; XSRF-CCKTOKEN=eb9bc4943bb2647da7e389116b3b930b; CHSICC_CLIENTFLAGGAOKAO=1d331ce10098465acd742c22f4f17cda; CHSICC_CLIENTFLAGCHSI=d7652cf64d259c53b2b39a438ad266cd; _gid=GA1.3.101851822.1670573755; _gat_gtag_UA_100524_6=1; JSESSIONID=CCFE1665C964CD5265D4B04FF019915C; zg_a5f21d6abc554e3c9e4c17aade025fd8=%7B%22sid%22%3A%201670573755568%2C%22updated%22%3A%201670573758940%2C%22info%22%3A%201670573755572%2C%22superProperty%22%3A%20%22%7B%7D%22%2C%22platform%22%3A%20%22%7B%7D%22%2C%22utm%22%3A%20%22%7B%7D%22%2C%22referrerDomain%22%3A%20%22gaokao.chsi.com.cn%22%7D; goaYXsyEWlxdP=53yhXXCi14.gqqqDleI1AQGLSbTU0LhwRWO1GiVKc4q32m6OF4pLVOfAHtadxs7cTqXt20sOGSKVkNUkwQ7AMy7qgzpXjGtmPAFSLkO7KqgSwsOSxxt8QmUDUsua.Sd7lQzYBOWVJLG9omBI6JIl7KOmKmikn5l.WkxL_LQuRJL4O9oxlt0zMieCubf2H5RkNDxf4iZcLUH23kCq5eSCNAbN4V3Z5mIwuahqGEnieVB7YAKnrx_NYyhjmxDSPsaIaHuNJrKLZPJwPlRLXNDEAmo'
}


url = 'https://gaokao.chsi.com.cn/sch/schoolInfoMain--schId-1.dhtml'
resp = requests.get(url=url,headers=headers)

suchocolate

f12观察了一下：
第一次访问时，网站返回412时，要求浏览器设置两个cookie项。
第二次访问时，却带了3个cookie项，估计得研究一下js是怎么生成的第3项。

a豪

suchocolate 发表于 2022-12-11 10:01
f12观察了一下：
第一次访问时，网站返回412时，要求浏览器设置两个cookie项。
第二次访问时，却带了 ...

好嘞 谢谢大哥