[已解决]html页面的代码保存，怎么才能正常地保存呢

jfjjx

url = "https://cbsprodcenter.noahwm.com/productcenter/pcIndex.jsp"
rget =  requests.get( url , params = query_string_parameters )
txt = rget.text
with open("d:\\html_code.txt","w") as f:
    f.write( str(txt) )
print(txt)
输出结果：UnicodeEncodeError: 'gbk' codec can't encode character '\xa9' in position 7793: illegal multibyte sequence


url = "https://cbsprodcenter.noahwm.com/productcenter/pcIndex.jsp"
rget =  requests.get( url , params = query_string_parameters )
txt = rget.text.encode('utf-8')
with open("d:\\html_code.txt","w") as f:
    f.write( str(txt) )
print(txt)

最佳答案

月排行榜 / 总排行榜

流氓大叔

2016-8-20 21:39:14

import requests
url = "https://cbsprodcenter.noahwm.com/productcenter/pcIndex.jsp"
rget =  requests.get( url)
txt = rget.content#这里用字节集
with open("d:\\html_code.txt","wb") as f:
    f.write(txt)
#print(txt)

跳转到最佳答案楼层

SixPy

rget.encoding = 'utf-8'
txt = rget.text

SixPy

爬虫，分析网页字符集和字符编码的基本技巧。
http://bbs.fishc.com/thread-74916-1-1.html

jfjjx

SixPy 发表于 2016-8-20 11:28
rget.encoding = 'utf-8'
txt = rget.text

输出结果还是这个：UnicodeEncodeError: 'gbk' codec can't encode character '\xa9' in position 7793: illegal multibyte sequence
网页的response headers是这样的，貌似就是utf-8的：
Content-Type:text/html;charset=UTF-8
Date:Sat, 20 Aug 2016 04:09:14 GMT
Server:Apache-Coyote/1.1
Transfer-Encoding:chunked
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~这咋回事。。。

SixPy

http://bbs.fishc.com/thread-74332-1-1.html

流氓大叔

import requests
url = "https://cbsprodcenter.noahwm.com/productcenter/pcIndex.jsp"
rget =  requests.get( url)
txt = rget.content#这里用字节集
with open("d:\\html_code.txt","wb") as f:
    f.write(txt)
#print(txt)

jfjjx

流氓大叔 发表于 2016-8-20 21:39
import requests
url = "https://cbsprodcenter.noahwm.com/productcenter/pcIndex.jsp"
rget =  request ...

完美
原理是啥，可以分享一下吗~~

流氓大叔

jfjjx 发表于 2016-8-21 13:41
完美
原理是啥，可以分享一下吗~~

content返回的是二进制的数据，wb写出的也是二进制的数据，二进制的字节集不需要转码，直接写出。然后你打开的时候再由打开的编辑器进行转码。

jfjjx

流氓大叔 发表于 2016-8-21 17:04
content返回的是二进制的数据，wb写出的也是二进制的数据，二进制的字节集不需要转码，直接写出。然后你 ...

学习了