你是我的小可爱 发表于 2020-4-3 22:03:35

爬虫

from urllib import request,parse
url = 'https://www.lagou.com/jobs/positionAjax.json?needAddtionalResult=false'
headers = {
'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0',
'Referer' : 'https://www.lagou.com/jobs/list_python%E7%88%AC%E8%99%AB?oquery=python&fromSearch=true&labelWords=relative'
}
data = {
      'first' : 'true',
      'pn': 1,
      'kd': 'python爬虫',
}
req = request.Request(url,headers = headers,data = parse.urlencode(data).encode('utf-8'),method = "POST")
resq = request.urlopen(req)
print(resq.read().decode('utf-8'))



运行后结果是：
{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"183.160.22.8","state":2402}


请求头那里不是已经加了Referer吗，怎么还是操作太频繁啊
有哪位大佬解释一下

wp231957 发表于 2020-4-4 10:17:03

反爬
页: [1]
查看完整版本: 爬虫