53讲

水继续吹

2. 写一个程序，依次访问文件中指定的站点，并将每个站点返回的内容依次存放到不同的文件中。
演示：
urls.txt 文件存放需要访问的 ULR：
http://www.fishc.com
http://www.baidu.com
http://www.douban.com
http://www.zhihu.com
http://www.taobao.com

1. import urllib.request as url
   
2. import chardet
   
3. import os
   
4. file1 = open('C:\\Users\\Administrator\\Desktop\\work\\urls.txt' , 'r' , errors = 'ignore')
   
5. for x in file1:
   
6.     num = 1
   
7.     file_url = url.urlopen(x).read()
   
8.     file_encoding = chardet.detect(file_url)['encoding']
   
9.     if file_encoding == 'GB2312':
   
10.         file_encoding = 'GBK'
    
11.     file_in = file_url.decode(file_encoding)
    
12.     with open('C:\\Users\\Administrator\\Desktop\\work\\url_' + str(num) , '+w') as f:
    
13.         f.write(file_in)
    
14.     num += 1
    
15. file1.close()

复制代码

执行后提示出错：
UnicodeEncodeError: 'gbk' codec can't encode character '\xbb' in position 29793: illegal multibyte sequence
这都什么鬼阿

wp231957

你能确定一定是gbk吗

水继续吹

它出来什么编码，我就用什么啊
如果是GB2312，我才转GBK

水继续吹

有点错误，改正了一下，但还是提示出错

1. import urllib.request as url
   
2. import chardet
   
3. import os
   
4. file1 = open('C:\\Users\\Administrator\\Desktop\\work\\urls.txt' , 'r' , errors = 'ignore')
   
5. num = 1
   
6. for x in file1:
   
7.     file_url = url.urlopen(x).read()
   
8.     file_encoding = chardet.detect(file_url)['encoding']
   
9.     if file_encoding == 'GB2312':
   
10.         file_encoding = 'GBK'
    
11.     file_in = file_url.decode(file_encoding)
    
12.     with open('C:\\Users\\Administrator\\Desktop\\work\\url_' + str(num) + '.txt' , '+w') as f:
    
13.         f.write(file_in)
    
14.     num += 1
    
15. file1.close()

复制代码

zltzlt

这样试试：

1. import urllib.request as url
   
2. import chardet
   
3. import os
   
4. file1 = open('urls.txt' , 'r' , errors = 'ignore')
   
5. num = 1
   
6. for x in file1:
   
7.     file_url = url.urlopen(x).read()
   
8.     file_encoding = chardet.detect(file_url)['encoding']
   
9.     if file_encoding == 'GB2312':
   
10.         file_encoding = 'GBK'
    
11.     file_in = file_url.decode(file_encoding)
    
12.     with open('url_' + str(num) + '.txt' , 'w+', encoding=file_encoding) as f:
    
13.         f.write(file_in)
    
14.     num += 1
    
15. file1.close()

复制代码

水继续吹

试了，更多的出错信息

Traceback (most recent call last):
  File "C:\Users\Administrator\Desktop\work\53C.py", line 7, in <module>
    file_url = url.urlopen(x).read()
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python37-32\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 418: