[已解决]关于代理爬虫问题

光宇

代码都是复制小甲鱼的，代理也换了，在百度找了十几个代理d网站，几十个代理都出问题了。
代码详细

1. 
   
2. import urllib.request
   
3. 
   
4. url = 'http://www.whatismyip.com.tw'
   
5. 
   
6. proxy_support = urllib.request.ProxyHandler({'http':'115.200.171.145:8123'})
   
7. 
   
8. opener = urllib.request.build_opener(proxy_support)
   
9. 
   
10. urllib.request.install_opener(opener)
    
11. 
    
12. response = urllib.request.urlopen(url)
    
13. 
    
14. html = reponse.read().decode('utf-8')
    
15. 
    
16. print(html)
    

复制代码

运行结果

情况1:无响应

1. 
   
2. Traceback (most recent call last):
   
3.   File "C:\Users\Administrator\Desktop\python\proxy_eg.py", line 13, in <module>
   
4.     html = reponse.read().decode('utf-8')
   
5. NameError: name 'reponse' is not defined
   

复制代码

情况2:请求超时

1. 
   
2. Traceback (most recent call last):
   
3.   File "C:\Users\Administrator\Desktop\python\proxy_eg.py", line 11, in <module>
   
4.     response = urllib.request.urlopen(url)
   
5.   File "C:\Python33\lib\urllib\request.py", line 156, in urlopen
   
6.     return opener.open(url, data, timeout)
   
7.   File "C:\Python33\lib\urllib\request.py", line 475, in open
   
8.     response = meth(req, response)
   
9.   File "C:\Python33\lib\urllib\request.py", line 587, in http_response
   
10.     'http', request, response, code, msg, hdrs)
    
11.   File "C:\Python33\lib\urllib\request.py", line 513, in error
    
12.     return self._call_chain(*args)
    
13.   File "C:\Python33\lib\urllib\request.py", line 447, in _call_chain
    
14.     result = func(*args)
    
15.   File "C:\Python33\lib\urllib\request.py", line 595, in http_error_default
    
16.     raise HTTPError(req.full_url, code, msg, hdrs, fp)
    
17. urllib.error.HTTPError: HTTP Error 504: Gateway Timeout
    

复制代码

情况3:主机无反应，链接失败

1. >>>
   
2. Traceback (most recent call last):
   
3.   File "C:\Python33\lib\urllib\request.py", line 1248, in do_open
   
4.     h.request(req.get_method(), req.selector, req.data, headers)
   
5.   File "C:\Python33\lib\http\client.py", line 1061, in request
   
6.     self._send_request(method, url, body, headers)
   
7.   File "C:\Python33\lib\http\client.py", line 1099, in _send_request
   
8.     self.endheaders(body)
   
9.   File "C:\Python33\lib\http\client.py", line 1057, in endheaders
   
10.     self._send_output(message_body)
    
11.   File "C:\Python33\lib\http\client.py", line 902, in _send_output
    
12.     self.send(msg)
    
13.   File "C:\Python33\lib\http\client.py", line 840, in send
    
14.     self.connect()
    
15.   File "C:\Python33\lib\http\client.py", line 818, in connect
    
16.     self.timeout, self.source_address)
    
17.   File "C:\Python33\lib\socket.py", line 435, in create_connection
    
18.     raise err
    
19.   File "C:\Python33\lib\socket.py", line 426, in create_connection
    
20.     sock.connect(sa)
    
21. TimeoutError: [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。
    
22. 
    
23. During handling of the above exception, another exception occurred:
    
24. 
    
25. Traceback (most recent call last):
    
26.   File "C:\Users\Administrator\Desktop\python\proxy_eg.py", line 11, in <module>
    
27.     response = urllib.request.urlopen(url)
    
28.   File "C:\Python33\lib\urllib\request.py", line 156, in urlopen
    
29.     return opener.open(url, data, timeout)
    
30.   File "C:\Python33\lib\urllib\request.py", line 469, in open
    
31.     response = self._open(req, data)
    
32.   File "C:\Python33\lib\urllib\request.py", line 487, in _open
    
33.     '_open', req)
    
34.   File "C:\Python33\lib\urllib\request.py", line 447, in _call_chain
    
35.     result = func(*args)
    
36.   File "C:\Python33\lib\urllib\request.py", line 1268, in http_open
    
37.     return self.do_open(http.client.HTTPConnection, req)
    
38.   File "C:\Python33\lib\urllib\request.py", line 1251, in do_open
    
39.     raise URLError(err)
    
40. urllib.error.URLError: <urlopen error [WinError 10060] 由于连接方在一段时间后没有正确答复或连接的主机没有反应，连接尝试失败。>

复制代码

情况4:软件中止了

1. >>>
   
2. Traceback (most recent call last):
   
3.   File "C:\\Users\\Administrator\\Desktop\\python\\proxy_eg.py", line 11, in <module>
   
4.     response = urllib.request.urlopen(url)
   
5.   File "C:\\Python33\\lib\\urllib\\request.py", line 156, in urlopen
   
6.     return opener.open(url, data, timeout)
   
7.   File "C:\\Python33\\lib\\urllib\\request.py", line 469, in open
   
8.     response = self._open(req, data)
   
9.   File "C:\\Python33\\lib\\urllib\\request.py", line 487, in _open
   
10.     '_open', req)
    
11.   File "C:\\Python33\\lib\\urllib\\request.py", line 447, in _call_chain
    
12.     result = func(*args)
    
13.   File "C:\\Python33\\lib\\urllib\\request.py", line 1268, in http_open
    
14.     return self.do_open(http.client.HTTPConnection, req)
    
15.   File "C:\\Python33\\lib\\urllib\\request.py", line 1253, in do_open
    
16.     r = h.getresponse()
    
17.   File "C:\\Python33\\lib\\http\\client.py", line 1143, in getresponse
    
18.     response.begin()
    
19.   File "C:\\Python33\\lib\\http\\client.py", line 354, in begin
    
20.     version, status, reason = self._read_status()
    
21.   File "C:\\Python33\\lib\\http\\client.py", line 316, in _read_status
    
22.     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    
23.   File "C:\\Python33\\lib\\socket.py", line 297, in readinto
    
24.     return self._sock.recv_into(b)
    
25. ConnectionAbortedError: [WinError 10053] 你的主机中的软件中止了一个已建立的连接。

复制代码

最佳答案

月排行榜 / 总排行榜

gopythoner

2017-4-14 15:46:54

免费IP大部分都是无效，小部分是偶尔有效，总之，免费代理IP根本不靠谱
然后，你的第一个报错并不是无响应，而是你打错一个变量名NameError: name 'reponse' is not defined
你把response打错了

跳转到最佳答案楼层

gopythoner

免费IP大部分都是无效，小部分是偶尔有效，总之，免费代理IP根本不靠谱
然后，你的第一个报错并不是无响应，而是你打错一个变量名NameError: name 'reponse' is not defined
你把response打错了