关于爬虫超时的问题。

fineconey

如题，在进行爬虫爬取的时候（单线程），一直提示

requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, '远程主机强迫关闭了一个现有的连接。', None, 10054, None))

期间，在request请求中，添加了timeout=25没有解决。
网上搜索，说是要引入import  socket  ，然后加上  socket.setdefaulttimeout(25)，设置超时时间。  也没有解决。

单次循环暂停time.sleep(25)，暂停25秒也未解决。
再后来，说是要保持窗口的cookies， 使用了   request.Session()  也依然没有解决。



暂时不知道怎么搞了。

求解决

1. Traceback (most recent call last):
   
2.   File "D:\Python38\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
   
3.     httplib_response = self._make_request(
   
4.   File "D:\Python38\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
   
5.     self._validate_conn(conn)
   
6.   File "D:\Python38\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
   
7.     conn.connect()
   
8.   File "D:\Python38\lib\site-packages\urllib3\connection.py", line 362, in connect
   
9.     self.sock = ssl_wrap_socket(
   
10.   File "D:\Python38\lib\site-packages\urllib3\util\ssl_.py", line 386, in ssl_wrap_socket
    
11.     return context.wrap_socket(sock, server_hostname=server_hostname)
    
12.   File "D:\Python38\lib\ssl.py", line 500, in wrap_socket
    
13.     return self.sslsocket_class._create(
    
14.   File "D:\Python38\lib\ssl.py", line 1040, in _create
    
15.     self.do_handshake()
    
16.   File "D:\Python38\lib\ssl.py", line 1309, in do_handshake
    
17.     self._sslobj.do_handshake()
    
18. ConnectionResetError: [WinError 10054] 远程主机强迫关闭了一个现有的连接。
    
19. 
    
20. During handling of the above exception, another exception occurred:
    
21. 
    
22. Traceback (most recent call last):
    
23.   File "D:\Python38\lib\site-packages\requests\adapters.py", line 439, in send
    
24.     resp = conn.urlopen(
    
25.   File "D:\Python38\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
    
26.     retries = retries.increment(
    
27.   File "D:\Python38\lib\site-packages\urllib3\util\retry.py", line 410, in increment
    
28.     raise six.reraise(type(error), error, _stacktrace)
    
29.   File "D:\Python38\lib\site-packages\urllib3\packages\six.py", line 734, in reraise
    
30.     raise value.with_traceback(tb)
    
31.   File "D:\Python38\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    
32.     httplib_response = self._make_request(
    
33.   File "D:\Python38\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    
34.     self._validate_conn(conn)
    
35.   File "D:\Python38\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    
36.     conn.connect()
    
37.   File "D:\Python38\lib\site-packages\urllib3\connection.py", line 362, in connect
    
38.     self.sock = ssl_wrap_socket(
    
39.   File "D:\Python38\lib\site-packages\urllib3\util\ssl_.py", line 386, in ssl_wrap_socket
    
40.     return context.wrap_socket(sock, server_hostname=server_hostname)
    
41.   File "D:\Python38\lib\ssl.py", line 500, in wrap_socket
    
42.     return self.sslsocket_class._create(
    
43.   File "D:\Python38\lib\ssl.py", line 1040, in _create
    
44.     self.do_handshake()
    
45.   File "D:\Python38\lib\ssl.py", line 1309, in do_handshake
    
46.     self._sslobj.do_handshake()
    
47. urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(10054, '远程主机强迫关闭了一个现有的连接。', None, 10054, None))
    
48. 
    
49. During handling of the above exception, another exception occurred:
    
50. 
    
51. Traceback (most recent call last):
    
52.   File "E:/MyPythonFiles/PT/mteam.py", line 21, in <module>
    
53.     res=requests.get(url,headers=headers,timeout=25)
    
54.   File "D:\Python38\lib\site-packages\requests\api.py", line 76, in get
    
55.     return request('get', url, params=params, **kwargs)
    
56.   File "D:\Python38\lib\site-packages\requests\api.py", line 61, in request
    
57.     return session.request(method=method, url=url, **kwargs)
    
58.   File "D:\Python38\lib\site-packages\requests\sessions.py", line 530, in request
    
59.     resp = self.send(prep, **send_kwargs)
    
60.   File "D:\Python38\lib\site-packages\requests\sessions.py", line 643, in send
    
61.     r = adapter.send(request, **kwargs)
    
62.   File "D:\Python38\lib\site-packages\requests\adapters.py", line 498, in send
    
63.     raise ConnectionError(err, request=request)
    
64. requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, '远程主机强迫关闭了一个现有的连接。', None, 10054, None))
    
65. 
    
66. Process finished with exit code 1
    

复制代码

不能懒

提供一下源代码

wp231957

这是遭遇反爬