爬取新浪微博图片关于UnicodeDecodeError报错的问题

三林小猎人

1. import urllib.request
   
2. import re
   
3. 
   
4. 
   
5. def open_url(url):
   
6.   req = urllib.request.Request(url)
   
7.   req.add_header('User-Agent','Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.116 UBrowser/5.6.11466.7 Safari/537.36')
   
8.   
   
9.   page = urllib.request.urlopen(req)
   
10.   html = page.read().decode('utf-8')
    
11. 
    
12.   return html
    
13. 
    
14. def get_img(html):
    
15.   p = r'<img node-type="imgShow" src="(^http+\.jpg)"'
    
16.   imglist = re.findall(p,html)
    
17. 
    
18. 
    
19.   for each in imglist:
    
20.     filename = each.split('/')[-1]
    
21.     urllib.request.urlretrieve(each,filename,None)
    
22.   
    
23. 
    
24. 
    
25. if __name__ == '__main__':
    
26.   url = 'http://weibo.com/u/3640184915/home?wvr=5'
    
27.   get_img(open_url(url))
    

复制代码

Traceback (most recent call last):
  File "C:\Users\dell\Desktop\weibotupian.py", line 27, in <module>
    get_img(open_url(url))
  File "C:\Users\dell\Desktop\weibotupian.py", line 10, in open_url
    html = page.read().decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xca in position 319: invalid continuation byte
查看网页确实是utf-8编码格式呀！！！！

blurrye

试下decode('utf-8', 'ignore')

有时候网页里有些奇怪的字符

三林小猎人

blurrye 发表于 2016-4-8 20:02
试下decode('utf-8', 'ignore')

有时候网页里有些奇怪的字符

好的，谢谢啦！！！