关于 BeautifulSoup 对象的.prettify() 方法
本帖最后由 gux 于 2021-7-11 22:42 编辑以下程序生成了一个BeautifulSoup对象soup, 并打印 soup.prettify():
from urllib.request import urlopen
from urllib.error import HTTPError
from bs4 import BeautifulSoup
from urllib.parse import urljoin
from urllib.request import pathname2url
def path2url(path):
return urljoin('file:', pathname2url(path))
html = urlopen(path2url('D:\\Webscraping\\Mathematics authors_titles \'new\'.html'), 'r')
soup = BeautifulSoup(html, 'html.parser')
print(soup.prettify())
运行结果的前几行为
i
</span>
<span style="display: inline-block; width: 0px; height: 3.993em;">
</span>
</span>
</span>
</span>
</span>
</span>
<span style="display: inline-block; width: 0px; height: 3.993em;">
</span>
soup.prettify() 是一个字符串对象：
print(type(soup.prettify()))
的运行结果为
<class 'str'>
然而，当我试图把 soup.prettify() 写入一个txt文件时，发生了异常：
from urllib.request import pathname2url
def path2url(path):
return urljoin('file:', pathname2url(path))
html = urlopen(path2url('D:\\Webscraping\\Mathematics authors_titles \'new\'.html'), 'r')
soup = BeautifulSoup(html, 'html.parser')
with open('arxiv_new.txt', 'w') as arxiv_new:
arxiv_new.write(soup.prettify())
运行时出现异常
Traceback (most recent call last):
File "D:\Webscraping\arxiv_math_new.py", line 18, in <module>
arxiv_new.write(soup.prettify())
UnicodeEncodeError: 'gbk' codec can't encode character '\u22ca' in position 28183: illegal multibyte sequence
为什么一个字符串对象能被打印，却不能写入文件？
网页获取的字符串编码和你 txt 文件的编码不同呗，编码不相同自然写入要么会报错、要么乱码
字符串是 utf-8 编码的可能，你 open 那加上 encoding = 'utf-8' 应该就可以了
参考代码：
with open('arxiv_new.txt', 'w'， encoding='utf-8') as arxiv_new:
arxiv_new.write(soup.prettify()) Twilight6 发表于 2021-7-11 22:57
网页获取的字符串编码和你 txt 文件的编码不同呗，编码不相同自然写入要么会报错、要么乱码
字符串是...
非常感谢！
页:
[1]