新手爬百度贴吧图片，报ValueError: unknown url type错误

紫枫醉听雨 · 发表于 2017-1-10 00:42:18

马上注册，结交更多好友，享用更多功能^_^

您需要登录才可以下载或查看，没有账号？立即注册

x

本帖最后由紫枫醉听雨于 2017-1-10 13:55 编辑

代码如下：
import urllib.request
import os
import re

def url_open(url):
req = urllib.request.Request(url)
req.add_header('User-Agent','Mozilla/5.0 (Windows NT 6.1; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0')
response = urllib.request.urlopen(req)
html = response.read()
return html

def get_imgs(url):
html = url_open(url).decode('utf-8')
p = r'<img class="BDE_Image" src="(.*?\.jpg)"'
img_adders = re.findall(p,html)
return img_adders

def download(folder='xxoo'):
os.mkdir(folder)
os.chdir(folder)
url = 'http://tieba.baidu.com/p/4913057524'
page_url = url_open(url)
img_adders = get_imgs(page_url)
print(img_adders)
for each in img_adders:
      filename = each.split('/')[-1]
      urllib.request.urlretrieve(each,filename)

if __name__ == '__main__':
download()

报错信息：
Traceback (most recent call last):
  File "C:/Users/Administrator/AppData/Local/Programs/Python/Python36-32/shunvba.py", line 31, in <module>
download()
  File "C:/Users/Administrator/AppData/Local/Programs/Python/Python36-32/shunvba.py", line 24, in download
img_adders = get_imgs(page_url)
  File "C:/Users/Administrator/AppData/Local/Programs/Python/Python36-32/shunvba.py", line 14, in get_imgs
html = url_open(url).decode('utf-8')
  File "C:/Users/Administrator/AppData/Local/Programs/Python/Python36-32/shunvba.py", line 7, in url_open
req = urllib.request.Request(url)
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 329, in __init__
self.full_url = url
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 355, in full_url
self._parse()
  File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 384, in _parse
raise ValueError("unknown url type: %r" % self.full_url)
ValueError: unknown url type: 'b\'\\n<!DOCTYPE html><html><head><meta name="keywords" content="\\xe7\\x99\

SixPy · 发表于 2017-1-10 08:02:06

def get_imgs(url):
html = url.decode('utf-8')
……

玄夜Python之路 · 发表于 2017-1-10 09:30:00

函数调用是出现错误了：
1、page_url = get_imgs(url)此处传入的是URL地址而不是html源码
2、if __name__ == '__main__': download()此时函数调用是没有传参，虽说程序能够完成，但是不符合函数调用的语法

紫枫醉听雨 · 发表于 2017-1-10 11:03:54

SixPy 发表于 2017-1-10 08:02
def get_imgs(url):
html = url.decode('utf-8')
……

谢谢，可以跑起来了，不过没弄明白，为什么这个不需要自己定义的url_open(url)函数呢，而是直接用url,我看视频教学里的都是需要这么一步操作的呀

紫枫醉听雨 · 发表于 2017-1-10 13:46:36

SixPy 发表于 2017-1-10 08:02
def get_imgs(url):
html = url.decode('utf-8')
……

明白了，page_url = url_open(url)就已经使用过url_open()函数了，再在get_imgs()函数中用url_open()函数就不对了

jiamin_2010 · 发表于 2017-8-23 19:46:20

Traceback (most recent call last):
File "C:/Users/Administrator/AppData/Local/Programs/Python/randomsmileys.py", line 4, in <module>
t.speed(1)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python36-32\lib\turtle.py", line 2167, in speed
return self._speed
AttributeError: 'int' object has no attribute '_speed'

jiamin_2010 · 发表于 2017-8-23 19:47:58

jiamin_2010 发表于 2017-8-23 19:46
Traceback (most recent call last):
File "C:/Users/Administrator/AppData/Local/Programs/Python/ran ...

这是什么错误啊，求解啊

账号		自动登录	找回密码
密码			立即注册