[已解决]高手帮看一下，使用python3编写的网络爬虫哪里有问题

monkeyjz

帮看一下，写的爬虫，不能用，自己新手不是很熟练



import re
import urllib.request

def getHtml(url):
    page = urllib .request.urlopen(url)
    html = page.read()
    print(html)
    return html

def getImg(html):
    reg = r'src="(.*?\.jpg)"'
    imgre = re.compile(reg)
    imglist = re.findall(imgre,html)

    for imgurl in imglist:
        urllib.urlretrieve(imgurl,'1.jpg')

html = getHtml("https://tieba.baidu.com/p/3740796143#!/l/p1")
print(getImg(html))

最佳答案

月排行榜 / 总排行榜

时光不老

2018-1-5 09:32:21

py3环境

1. import re
   
2. import urllib.request
   
3. 
   
4. def getHtml(url):
   
5.     page = urllib .request.urlopen(url)
   
6.     html = page.read()
   
7.     # print(html)
   
8.     return html
   
9. 
   
10. def getImg(html):
    
11.     # reg = 'src="(.*?\.jpg)"'
    
12.     # imgre = re.compile(reg)
    
13.     imglist = re.findall('src="(.*?\.jpg)"',html.decode('utf-8'))
    
14. 
    
15.     for imgurl in imglist:
    
16.         urllib.request.urlretrieve(imgurl,'1.jpg')
    
17. 
    
18. html = getHtml("https://tieba.baidu.com/p/3740796143#!/l/p1")
    
19. print(getImg(html))

复制代码

遇到两个报错：
AttributeError: module 'urllib' has no attribute 'urlretrieve'
通过 urllib.request.urlretrieve() 解决
TypeError: cannot use a string pattern on a bytes-like object
通过 html.decode('utf-8') 解决

跳转到最佳答案楼层

时光不老

py3环境

1. import re
   
2. import urllib.request
   
3. 
   
4. def getHtml(url):
   
5.     page = urllib .request.urlopen(url)
   
6.     html = page.read()
   
7.     # print(html)
   
8.     return html
   
9. 
   
10. def getImg(html):
    
11.     # reg = 'src="(.*?\.jpg)"'
    
12.     # imgre = re.compile(reg)
    
13.     imglist = re.findall('src="(.*?\.jpg)"',html.decode('utf-8'))
    
14. 
    
15.     for imgurl in imglist:
    
16.         urllib.request.urlretrieve(imgurl,'1.jpg')
    
17. 
    
18. html = getHtml("https://tieba.baidu.com/p/3740796143#!/l/p1")
    
19. print(getImg(html))

复制代码

遇到两个报错：
AttributeError: module 'urllib' has no attribute 'urlretrieve'
通过 urllib.request.urlretrieve() 解决
TypeError: cannot use a string pattern on a bytes-like object
通过 html.decode('utf-8') 解决

monkeyjz

谢谢帮助