|
|
马上注册,结交更多好友,享用更多功能^_^
您需要 登录 才可以下载或查看,没有账号?立即注册
x
在爬虫的第一节课课后题里有如下代码- import urllib.request
- import chardet
- def main():
- i = 0
-
- with open("urls.txt", "r") as f:
- # 读取待访问的网址
- # 由于urls.txt每一行一个URL
- # 所以按换行符'\n'分割
- urls = f.read().splitlines()
-
- for each_url in urls:
- response = urllib.request.urlopen(each_url)
- html = response.read()
- # 识别网页编码
- encode = chardet.detect(html)['encoding']
- if encode == 'GB2312':
- encode = 'GBK'
-
- i += 1
- filename = "url_%d.txt" % i
- [color=Red]with open(filename, "w", encoding=encode) as each_file:
- each_file.write(html.decode(encode, "ignore"))
- [/color]
- if __name__ == "__main__":
- main()
复制代码
其中open()函数带了encoding 参数。这个参数有什么效果?
我在不带这个参数时报错了- UnicodeEncodeError: 'gbk' codec can't encode character '\xd3' in position 256: illegal multibyte sequence
复制代码
|
|