huyanmin 发表于 2022-7-29 15:34:54

python爬虫,成功安装bs4,执行程序却保错:import requests from bs4 import Beaut...


#爬取三国演义小说
import requests
from bs4 import BeautifulSoup      #程序报错:ModuleNotFoundError: No module named 'html.entities'; 'html' is not a package, 这是怎么回事,该怎么解决

url='http://sanguo.5000yan.com/'
headers ={'User-Agent':'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Mobile Safari/537.36'}

response = requests.get(url=url,headers=headers)
response.encoding='utf-8'
page_text=response.text

soup = BeautifulSoup(page_text,'lxml')
#print(soup)
li_list = soup.select(".sidamingzhu-list-mulu>ul>li")
#print(li_list)
fp =open('./sanguo.txt','w',encoding='utf-8')#放到循环上面,保证文件被打开一次,这样所有章节都会被追加写入
for li in li_list:
    #解析标题和正文
    title =li.a.string
    detail_url =li.a['href']#一个IP就是对应一个网页或者一张图片
    response =requests.get(url=detail_url,headers=headers)
    response.encoding='utf-8'
    detail_url_text = response.text
    #解析正文
    detail_soup =BeautifulSoup(detail_url_text,'lxml')
    detail_tag =detail_soup.find('div',class_="grap")
    #print(detail_tag)
    content =detail_tag.text
    #fp =open('./sanguo.txt','w',encoding='utf-8')#放到此处每循环一次,文件就会被打开一次,这样只有最后一次写入被保存
   
   
    fp.write(title +':::::::'+content+'\n')
    print(title,'下载成功')

url='http://sanguo.5000yan.com/'
headers ={'User-Agent':'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Mobile Safari/537.36'}

response = requests.get(url=url,headers=headers)
response.encoding='utf-8'
page_text=response.text

soup = BeautifulSoup(page_text,'lxml')
#print(soup)
li_list = soup.select(".sidamingzhu-list-mulu>ul>li")
#print(li_list)
fp =open('./sanguo.txt','w',encoding='utf-8')#放到循环上面,保证文件被打开一次,这样所有章节都会被追加写入
for li in li_list:
    #解析标题和正文
    title =li.a.string
    detail_url =li.a['href']#一个IP就是对应一个网页或者一张图片
    response =requests.get(url=detail_url,headers=headers)
    response.encoding='utf-8'
    detail_url_text = response.text
    #解析正文
    detail_soup =BeautifulSoup(detail_url_text,'lxml')
    detail_tag =detail_soup.find('div',class_="grap")
    #print(detail_tag)
    content =detail_tag.text
    #fp =open('./sanguo.txt','w',encoding='utf-8')#放到此处每循环一次,文件就会被打开一次,这样只有最后一次写入被保存
   
   
    fp.write(title +':::::::'+content+'\n')
    print(title,'下载成功')

临时号 发表于 2022-7-29 16:29:33

本帖最后由 临时号 于 2022-7-29 16:31 编辑

你的py文件存放路径下可能有一个文件的名字将html.py,把它换一个名字

huyanmin 发表于 2022-7-29 17:02:44

临时号 发表于 2022-7-29 16:29
你的py文件存放路径下可能有一个文件的名字将html.py,把它换一个名字

非常感谢,困惑了我一下午

huyanmin 发表于 2022-7-29 17:05:20

huyanmin 发表于 2022-7-29 17:02
非常感谢,困惑了我一下午

我的html.py和谁冲突了
页: [1]
查看完整版本: python爬虫,成功安装bs4,执行程序却保错:import requests from bs4 import Beaut...