[已解决]求助

黄jz

触发网页反爬了怎么解决，今天刚学习到实战那里，代码打好了执行以后就返回418的错误

最佳答案

月排行榜 / 总排行榜

suchocolate

2020-9-11 09:56:13

豆瓣418吧，改ua，默认ua会被反扒，要这样：

1. import requests
   
2. headers = {'user-agent': 'firefox'}   # 其实写成什么都可以，只要不是默认的python-requests/urllib，这里随便写的firefox。
   
3. r = requests.get('https://www.douban.com', headers=headers)
   
4. print(r.text)

复制代码

跳转到最佳答案楼层

疾风怪盗

能上个代码、错误提示么。。。。。。你这一句话，也不知道啥意思
一般加个headers，多几个参数，或者换手机端网页，能获取json就获取json更方便

黄jz

import urllib.request
from http.cookiejar import CookieJar

# 豆瓣的登录url
loginurl = 'https://accounts.douban.com/passport/login'
cookie = CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor)

data = {
    "form_email":"your email",
    "form_password":"your password",
    "source":"index_nav"
}
data = {}
data['form_email'] = '
data['form_password'] =
data['source'] = 'index_nav'

response = opener.open(loginurl, urllib.parse.urlencode(data).encode('utf-8'))

# 登录成功跳转至首页
if response.geturl() == "http://www.douban.com/":
     print('登录成功！')





Traceback (most recent call last):
  File "G:\新建文件夹 (4)\test2.py", line 20, in <module>
    response = opener.open(loginurl, urllib.parse.urlencode(data).encode('utf-8'))
  File "E:\python\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "E:\python\lib\urllib\request.py", line 640, in http_response
    response = self.parent.error(
  File "E:\python\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "E:\python\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
  File "E:\python\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 418:
>>>

黄jz

疾风怪盗 发表于 2020-9-10 12:59
能上个代码、错误提示么。。。。。。你这一句话，也不知道啥意思
一般加个headers，多几个参数 ...

import urllib.request
from http.cookiejar import CookieJar

# 豆瓣的登录url
loginurl = 'https://accounts.douban.com/passport/login'
cookie = CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor)

data = {
    "form_email":"your email",
    "form_password":"your password",
    "source":"index_nav"
}
data = {}
data['form_email'] = '
data['form_password'] =
data['source'] = 'index_nav'

response = opener.open(loginurl, urllib.parse.urlencode(data).encode('utf-8'))

# 登录成功跳转至首页
if response.geturl() == "http://www.douban.com/":
     print('登录成功！')




Traceback (most recent call last):
  File "G:\新建文件夹 (4)\test2.py", line 20, in <module>
    response = opener.open(loginurl, urllib.parse.urlencode(data).encode('utf-8'))
  File "E:\python\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "E:\python\lib\urllib\request.py", line 640, in http_response
    response = self.parent.error(
  File "E:\python\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "E:\python\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
  File "E:\python\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 418:
>>>

suchocolate

豆瓣418吧，改ua，默认ua会被反扒，要这样：

1. import requests
   
2. headers = {'user-agent': 'firefox'}   # 其实写成什么都可以，只要不是默认的python-requests/urllib，这里随便写的firefox。
   
3. r = requests.get('https://www.douban.com', headers=headers)
   
4. print(r.text)

复制代码