Event loop is closed,Python交流,编程语言专区,鱼C论坛

一只魈咸鱼 发表于 2021-8-13 14:09:40

Event loop is closed

这段代码为什么会报题目的错误呢
import asyncio
import aiohttp
from lxml import etree
import requests
headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
}
#/html/body/div/div/div/dl/dd/a/img
async def download(url):
name = url.rsplit('/',1)
async with aiohttp.ClientSession() as session:
   async with session.get(url) as res:
         with open(f'彼岸桌面壁纸下载/{name}','wb') as f:
            f.write(await res.content.read())
async def main():
tasks=[]
for url in urls:
   tasks.append(download(url))
await asyncio.wait(tasks)
if __name__ == '__main__':
url0 = 'https://www.enterdesk.com/zhuomianbizhi/'
text = requests.get(url0).text
tree = etree.HTML(text)
urls = tree.xpath('/html/body/div/div/div/dl/dd/a/img/@src')
asyncio.run(main())

大马强 发表于 2021-8-13 14:29:17

需要加上这句tasks = []
for url in urls:
task = asyncio.create_task(download(url))
tasks.append(task)
await asyncio.wait(tasks)

kogawananari 发表于 2021-8-13 14:57:00

asyncio.run是这样的改成3.8之前的写法可以不报错。。还有就是请求http而不是https也不会报这个错{:10_292:}bug属于是

一只魈咸鱼 发表于 2021-8-13 15:17:58

大马强发表于 2021-8-13 14:29
需要加上这句

我一开始就是用的create_task，报的也是这个错

一只魈咸鱼 发表于 2021-8-13 15:18:28

kogawananari 发表于 2021-8-13 14:57
asyncio.run是这样的改成3.8之前的写法可以不报错。。还有就是请求http而不是https也不会报这个错{:10_2 ...

{:10_266:}抱歉我没看懂，能详细点嘛

大马强 发表于 2021-8-13 15:22:51

一只魈咸鱼发表于 2021-8-13 15:17
我一开始就是用的create_task，报的也是这个错

这样应该可以了
import asyncio
import aiohttp
from lxml import etree
import requests
headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
}
# /html/body/div/div/div/dl/dd/a/img

async def download(url):
name = url.rsplit('/', 1)
async with aiohttp.ClientSession() as session:
   async with session.get(url) as res:
         async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
            await f.write(await res.content.read())

async def main():
tasks = []
for url in urls:
   task = asyncio.create_task(download(url))
   tasks.append(task)
await asyncio.wait(tasks)
if __name__ == '__main__':
url0 = 'https://www.enterdesk.com/zhuomianbizhi/'
text = requests.get(url0).text
tree = etree.HTML(text)
urls = tree.xpath('/html/body/div/div/div/dl/dd/a/img/@src')
asyncio.run(main())

大马强 发表于 2021-8-13 15:23:22

一只魈咸鱼发表于 2021-8-13 15:17
我一开始就是用的create_task，报的也是这个错

在修改这两句
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
await f.write(await res.content.read())

一只魈咸鱼 发表于 2021-8-13 15:31:19

大马强发表于 2021-8-13 15:22
这样应该可以了

还是不行，这是报错：
Task exception was never retrieved
future: <Task finished name='Task-16' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-15' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-14' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-13' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-12' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-11' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-10' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-9' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-8' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-7' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-6' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-5' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-4' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-3' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Task exception was never retrieved
future: <Task finished name='Task-17' coro=<download() done, defined at D:/MyPcharmProject/爬虫/爬取唯美壁纸.py:11> exception=AttributeError('__aexit__')>
Traceback (most recent call last):
File "D:/MyPcharmProject/爬虫/爬取唯美壁纸.py", line 15, in download
async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
AttributeError: __aexit__
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x00000198DF849430>
Traceback (most recent call last):
File "C:\Users\lenovo\AppData\Local\Programs\Python\Python38\lib\asyncio\proactor_events.py", line 116, in __del__
self.close()
File "C:\Users\lenovo\AppData\Local\Programs\Python\Python38\lib\asyncio\proactor_events.py", line 108, in close
self._loop.call_soon(self._call_connection_lost, None)
File "C:\Users\lenovo\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 711, in call_soon
self._check_closed()
File "C:\Users\lenovo\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 504, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

kogawananari 发表于 2021-8-13 16:45:35

import asyncio
import aiohttp
headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
}

async def download(url):
async with aiohttp.ClientSession() as session:
   async with session.get(url=url, headers=headers) as response:
         page_text = await response.text()
         print(page_text)

async def main():
tasks = []
for url in urls:
   task = asyncio.create_task(download(url))
   tasks.append(task)
await asyncio.wait(tasks)
if __name__ == '__main__':
urls = [
   'https://fishc.com.cn/forum.php?mod=viewthread&tid=200766&page=1#pid5515666',
   'https://fishc.com.cn/forum.php?mod=viewthread&tid=200766&page=1#pid5515666'
]
asyncio.run(main())

你试试看哪怕是请求你这个贴子页面依然会报错的但是我开个http 127.0.0.1的服务去自己请求自己就不会报错 99%是aiohttp这个库有问题不能请求https

kogawananari 发表于 2021-8-13 16:56:30

本帖最后由 kogawananari 于 2021-8-13 16:57 编辑

import asyncio,requests,aiohttp
from lxml import etree
from threading import Thread
header = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36'
}

def start_loop(loop):
asyncio.set_event_loop(loop)
loop.run_forever()

async def end_loop():
loop = asyncio.get_event_loop()
loop.stop()

async def download(url):
name = url.rsplit('/', 1)
async with aiohttp.ClientSession() as session:
   async with session.get(url) as res:
         async with open(f'彼岸桌面壁纸下载/{name}', 'wb') as f:
            await f.write(await res.content.read())

if __name__ == '__main__':
url0 = 'https://www.enterdesk.com/zhuomianbizhi/'
text = requests.get(url0).text
tree = etree.HTML(text)
urls = tree.xpath('/html/body/div/div/div/dl/dd/a/img/@src')
new_loop = asyncio.new_event_loop()
t = Thread(target=start_loop, args=(new_loop,))
t.start()
for url in urls:
   asyncio.run_coroutine_threadsafe(download(url),new_loop)
t.join()

我给你甩另一个线程里面貌似就可以了

一只魈咸鱼 发表于 2021-8-13 17:25:05

kogawananari 发表于 2021-8-13 16:45
你试试看哪怕是请求你这个贴子页面依然会报错的但是我开个http 127.0.0.1的服务去自己请求自己就不 ...

那对于https咋请求哎，现在感觉大多数都是https

kogawananari 发表于 2021-8-13 17:28:45

一只魈咸鱼发表于 2021-8-13 17:25
那对于https咋请求哎，现在感觉大多数都是https

试一下我刚才写的甩给另一个线程好像就不报错了

一只魈咸鱼 发表于 2021-8-13 17:30:40

kogawananari 发表于 2021-8-13 17:28
试一下我刚才写的甩给另一个线程好像就不报错了

{:10_266:}嗯是的，其实我还想知道https的该怎么搞，这么整也太麻烦了

kogawananari 发表于 2021-8-13 17:31:51

一只魈咸鱼发表于 2021-8-13 17:30
嗯是的，其实我还想知道https的该怎么搞，这么整也太麻烦了

那就甩到其他线程嘛{:10_293:}

一只魈咸鱼 发表于 2021-8-13 17:32:35

kogawananari 发表于 2021-8-13 16:56
我给你甩另一个线程里面貌似就可以了

不过为什么会想到再开一个线程呢，主线程也没被占用呀

kogawananari 发表于 2021-8-13 17:34:22

一只魈咸鱼发表于 2021-8-13 17:32
不过为什么会想到再开一个线程呢，主线程也没被占用呀

join比较稳主线程有提前结束的嫌疑

一只魈咸鱼 发表于 2021-8-13 17:36:10

kogawananari 发表于 2021-8-13 17:34
join比较稳主线程有提前结束的嫌疑

{:10_266:}好滴吧我现在是听不懂了，不过还是谢谢你！

kogawananari 发表于 2021-8-13 18:20:39

一只魈咸鱼发表于 2021-8-13 17:36
好滴吧我现在是听不懂了，不过还是谢谢你！

就是asyncio.run这个还是里面自动get loop 和 close loop 。它close的时候https请求还没完就close就报这个错。而asyncio.run是py3.8以上的我在另一个线程会被迫用的3.8以下的写法 loop run forever 这个方法是永远等待你去抛给它新任task，所以它不会自动调用loop close。所以更稳。所以我说99%是aiohttp问题 1%是3.8以上语法问题。不用其他线程也可以 loop拿着了自己close就行

一只魈咸鱼 发表于 2021-8-13 20:13:13

kogawananari 发表于 2021-8-13 18:20
就是asyncio.run这个还是里面自动get loop 和 close loop 。它close的时候https请求还没完就close就报这 ...

意思asyncio.run有bug吗，它close早了是吗

kogawananari 发表于 2021-8-13 22:48:14

一只魈咸鱼发表于 2021-8-13 20:13
意思asyncio.run有bug吗，它close早了是吗

这俩都有bug 不然也不会是https才会close早

页: [1] 2

鱼C论坛's Archiver

Event loop is closed