[已解决]煎蛋网的妹子图爬不了了

苏绛雪 · 发表于 2020-1-7 17:06:06

我上次用了小甲鱼的爬取煎蛋网妹子图的代码，发现无法下载图片了

，怎么办啊

最佳答案

月排行榜 / 总排行榜

__doc__

2020-1-7 17:06:07

import urllib.request
import os
def get_page(url):
'''抓取网页地址(从当前url中的'下一页'开始)
返回一个url'''
html = url_open(url).decode('utf-8')
a = html.find('current-comment-page')
a = html.find('jandan.net/ooxx/',a)
b = html.find('#comments',a) + 9
return 'http://' + html[a:b]
def find_imgs(url):
'''获取当前url中的jpg源地址,返回一个列表,包含了当前页面的几乎所有jpg'''
img_addrs = []
html = url_open(url).decode('utf-8')
a = 0
b = 0
while True:
a = html.find('li id="comment',a)
if a == -1:
break
a = html.find('img src=',a) + 11
if a == 10:
break
b = html.find('.jpg',a, a + 255) + 4
if b == 3:
break
img_addrs.append(html[a:b])
a = b
print(img_addrs)
return img_addrs
def save_imgs(folder, img_addrs):
'''将图像保存到文件夹'''
for each in img_addrs:
filename = each.split('/')[-1]
with open(filename, 'wb') as f:
img = url_open('http://' + each)
f.write(img)
def url_open(url):
'''打开url,返回未解码的html'''
req = urllib.request.Request(url)
req.add_header('User-Agent','Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0')
'''
如需代理则uncomment
proxies = []
proxy = random.choice(proxies)
proxy_support = urllib.request.ProxyHandler({'http':proxy})
opener = urllib.request.build_opener(proxy_support)
urllib.request.install_opener(opener)
'''
response = urllib.request.urlopen(req)
html = response.read()
return html
def download_mm(folder='ooxx',pages = 10):
os.mkdir(folder)
os.chdir(folder)
url = 'http://jandan.net/ooxx/'
next_page = get_page(url)
for page in range(pages):
print(next_page)
save_imgs(folder, find_imgs(next_page))
next_page = get_page(next_page)
if __name__ == '__main__' :
download_mm('ooxx2',pages=5)

复制代码

跳转到最佳答案楼层

__doc__ · 发表于 2020-1-7 17:06:07

这个最佳答案由 __doc__ 给出，感谢 __doc__ 的回答。

单击隐藏图章

import urllib.request
import os
def get_page(url):
'''抓取网页地址(从当前url中的'下一页'开始)
返回一个url'''
html = url_open(url).decode('utf-8')
a = html.find('current-comment-page')
a = html.find('jandan.net/ooxx/',a)
b = html.find('#comments',a) + 9
return 'http://' + html[a:b]
def find_imgs(url):
'''获取当前url中的jpg源地址,返回一个列表,包含了当前页面的几乎所有jpg'''
img_addrs = []
html = url_open(url).decode('utf-8')
a = 0
b = 0
while True:
a = html.find('li id="comment',a)
if a == -1:
break
a = html.find('img src=',a) + 11
if a == 10:
break
b = html.find('.jpg',a, a + 255) + 4
if b == 3:
break
img_addrs.append(html[a:b])
a = b
print(img_addrs)
return img_addrs
def save_imgs(folder, img_addrs):
'''将图像保存到文件夹'''
for each in img_addrs:
filename = each.split('/')[-1]
with open(filename, 'wb') as f:
img = url_open('http://' + each)
f.write(img)
def url_open(url):
'''打开url,返回未解码的html'''
req = urllib.request.Request(url)
req.add_header('User-Agent','Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Firefox/68.0')
'''
如需代理则uncomment
proxies = []
proxy = random.choice(proxies)
proxy_support = urllib.request.ProxyHandler({'http':proxy})
opener = urllib.request.build_opener(proxy_support)
urllib.request.install_opener(opener)
'''
response = urllib.request.urlopen(req)
html = response.read()
return html
def download_mm(folder='ooxx',pages = 10):
os.mkdir(folder)
os.chdir(folder)
url = 'http://jandan.net/ooxx/'
next_page = get_page(url)
for page in range(pages):
print(next_page)
save_imgs(folder, find_imgs(next_page))
next_page = get_page(next_page)
if __name__ == '__main__' :
download_mm('ooxx2',pages=5)

复制代码

空青 · 发表于 2020-1-7 17:07:39

前两天我让别人去爬，貌似没有加密了，可以直接看到图片地址

eachill · 发表于 2020-1-7 17:16:19

好歹怎么下载不了也给个截图哇

溯影 · 发表于 2020-1-7 17:48:12

可能煎蛋网的反爬机制升级了。

qq532401019 · 发表于 2020-1-7 23:51:41

网页地址改了，不是数字的那种，变成字母了

XiaoPaiShen · 发表于 2020-1-8 04:00:08

把代码贴出来，才好知道有什么问题。

Timeload · 发表于 2020-1-8 09:31:23

我的这篇文章解决了这个问题
https://fishc.com.cn/forum.php?m ... p;page=1#pid4266356

林思橙 · 发表于 2020-1-8 12:21:20

meizi网可以下载，这里给出我写的一个代码
import urllib.request
import re
import time
import os

hostreferer={
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36',
'Referer':'https://www.mzitu.com'}
hostreferer2={
'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36',
'Referer':'https://i.meizitu.net'}

def gethtml(url,host):
p=urllib.request.Request(url,headers=host)
response=urllib.request.urlopen(p)
html=response.read()
return html

def getpicture(html,host2):
html=html.decode('utf-8')
image=re.findall(r"data-original='(https:[^']+\.jpg)'",html)
for each in image:
      filename = each.split('/')[-1]
      with open(filename, 'wb') as f:
         img = gethtml(each,host2)
         f.write(img)
def getpageurl(url,num):
url=url+'page/'+str(num)+'/'
return url

if __name__=='__main__':
n=int(input('请输入需要下载mm图片的页数：'))
url_origin='https://www.mzitu.com/xinggan/'
url='https://www.mzitu.com/mm/'
os.mkdir('mm')
os.chdir('mm')
for i in range(1,n+1):
      print('正在下载第%d页mm图片' %i)
      getpicture(gethtml(url,hostreferer),hostreferer2)
      url=getpageurl(url_origin,i)
      time.sleep(1)

momo1232115 · 发表于 2020-1-8 13:32:44

这地址的图片尺寸有点大啊

__doc__ · 发表于 2020-1-8 15:31:30

可以参考我空间里的代码,网站有变化,我稍微改了一下

账号		自动登录	找回密码
密码			立即注册