爬虫爬不出图片
用小甲鱼教的方法去爬取图片，但是爬了半天什么图片都没有代码是在不停的跑，但是就是跑不出东西
有没有大佬帮忙看下是哪里出了问题
import urllib
import os
import random
def open_url(url):
head = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'}
req = urllib.request.Request(url, head)
response = urllib.request.urlopen(url)
return response.read()
def find_imgs(url):
html = open_url(url).decode('utf_8', 'ignore')
img_addrs = []
a = html.find('img src')+8
while a != -1:
b = html.find('.jpg', a, a+255)+4
if b != -1:
img_addr = 'https://pic.netbian.com'+html
img_addrs.append(img_addr)
else:
b = a+9
a = html.find('img src', b)+8
return img_addrs
def save_imgs(img_addrs):
for each in img_addrs:
img = open_url(each)
img_name = each.split('/')[-1]
with open(img_name, 'wb') as f:
f.write(img)
def download_mm(pages=10):
os.mkdir(r'D:\python\mm')
os.chdir(r'D:\python\mm')
url = "https://pic.netbian.com/4kmeinv/"
page_num = 1
for i in range(pages):
page_num += 1
page_url = url+'index_'+str(page_num)+'.html'
img_addrs = find_imgs(page_url)
save_imgs(img_addrs)
if __name__ == '__main__':
download_mm() 3个问题吧：
1）req这么用：def open_url(url):
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) '
'Chrome/91.0.4472.124 Safari/537.36'}
req = urllib.request.Request(url, headers=headers)
response = urllib.request.urlopen(req)
return response.read()
2）代码里组装的url，手工用浏览器访问回404，是不是你的URL看错了。
https://pic.netbian.com/4kmeinv/index_1.html
你把你想爬取的内容说一下，我去那个网站没找到index url。
3）decode应该这么写：open_url(url).decode('utf-8', 'ignore')
页:
[1]