huangdongdong 发表于 2021-5-26 15:30:51

爬妹子图

本帖最后由 huangdongdong 于 2021-5-26 15:36 编辑

import urllib.request
import os
import base64

def url_open(url):
    req = urllib.request.Request(url)
    req.add_header('User-Agent','Mozilla / 5.0（Windows NT 10.0; WOW64）AppleWebKit / 537.36（KHTML，like Gecko）Chrome / 86.0.4240.198 Safari / 537.36')
    response = urllib.request.urlopen(req)
    html = response.read()
    return html

def get_page(url):
    html = url_open(url).decode('UTF-8')
    a = html.find('current-comment-page') + 23
    b = html.find(']',a)
   
    return html

def find_addrs(url):
    html = url_open(url).decode('UTF-8')
    img_addrs = []

    a = html.find('img src =')

    while a != -1:
      b = html.find('.jpg',a,a+255)
      if b != -1:
            img_addrs.append(html)
      else:
            b = a+9

      a = html.find('img src =',b)

    return img_addrs


def save_imgs(folder,img_addrs):
    for each in img_addrs:
      filename = each.split('/')[-1]
      with open(filename,'wb') as f:
            img = url_open(each)
            f.write(img)

def download_num(folder = 'ooxx',pages = 10):
    os.mkdir(folder)
    os.chdir(folder)

    url = 'http://jandan.net/ooxx/'
    page_num = int(get_page(url))

    for i in range(pages):
      page_num-= 1
      target_num = '20210114' + str(page_num)
      target_num = base64.b64encode(target_num.encode('UTF-8'))

      page_url = url + '/' + str(target_num,'UTF -8') + '#comment'
      img_addrs = find_imgs(page_url)
      save_imgs(folder,img_addrs)
      
if __name__ == '__main__':
    download_num()
UnicodeEncodeError: 'latin-1' codec can't encode character '\uff08' in position 13: ordinal not in range(256)
>>>
如何解决？？？

路神 发表于 2021-5-26 15:30:52

'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 '
                  'Safari/537.36'

huangdongdong 发表于 2021-5-27 14:32:11

路神 发表于 2021-5-26 22:00
'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 '
...

ValueError: unknown url type: '//wx3.sinaimg.cn/mw600/0089jzyPly1gqwwi3t18vj30u01hcx6q.jpg'

路神 发表于 2021-5-27 16:54:10

huangdongdong 发表于 2021-5-27 14:32
ValueError: unknown url type: '//wx3.sinaimg.cn/mw600/0089jzyPly1gqwwi3t18vj30u01hcx6q.jpg'

https:

huangdongdong 发表于 2021-5-27 17:06:46

路神 发表于 2021-5-27 16:54
https:

程序走完但文件夹为空

路神 发表于 2021-5-27 19:35:37

huangdongdong 发表于 2021-5-27 17:06
程序走完但文件夹为空

a = html.find('img src =')这句全是-1当然没文件，这是你自己写的？还有这爬的也不是妹图

huangdongdong 发表于 2021-5-28 15:14:50

路神 发表于 2021-5-27 19:35
a = html.find('img src =')这句全是-1当然没文件，这是你自己写的？还有这爬的也不是妹图

找到解决办法了。只要在with ....of下加一句each= 'http'+ each即可
页: [1]
查看完整版本: 爬妹子图