[已解决]求助python 3.8 爬百度贴吧不报错没反应的问题

ijeju · 发表于 2020-4-7 23:45:33

马上注册，结交更多好友，享用更多功能^_^

您需要登录才可以下载或查看，没有账号？立即注册

x

按照🐟c的方法写了爬虫，也按照弹幕网友的建议调试了各种方式，却仍没反应，把代码贴在下面了，请各路大神帮忙看下，调了一晚上了。。。

import urllib.request
import re

def open_url(url):
req = urllib.request.Request(url)
req.add_header('Referer', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18362')
page = urllib.request.urlopen(req)
html = page.read().decode('utf-8')

return html

def get_img(html):
p = r'<img width="[\d]+" height=“[\d]+" class="BDE_Image" src="([^"]+\.jpg)"'

imglist = re.findall(p, html)

for each in imglist:
      pic = each
      print(pic)
      '''
      filename = each.split("/")[-1]
      urllib.request.urlretrieve(each, filename, None)
      '''

if __name__ == '__main__':
url = 'https://tieba.baidu.com/p/6577032706'
get_img(open_url(url))

最佳答案

月排行榜 / 总排行榜

LuLD

2020-4-8 11:20:11

我看了下实际网页上图片的
<img class="BDE_Image" src="http://tiebapic.baidu.com/forum/w%3D580/sign=b550f3b9bc64034f0fcdc20e9fc27980/f89992eef01f3a29d35ee8aa8e25bc315d607c09.jpg" size="1776777" changedsize="true" width="560" height="420">

你这个 :p = r'<img width="[\d]+" height=“[\d]+" class="BDE_Image" src="([^"]+\.jpg)"'

要改下 :p = r'<img class="BDE_Image" src="([^"]+\.jpg)"'

====
p = <img class="BDE_Image" src="([^"]+\.jpg)"

"""
imglist : ['http://tiebapic.baidu.com/forum/w%3D580/sign=b550f3b9bc64034f0fcdc20e9fc27980/f89992eef01f3a29d35ee8aa8e25bc315d607c09.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=40b49f7fc30735fa91f04eb1ae500f9f/202a30d12f2eb93870fc4796c2628535e4dd6f0a.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=2c9ec0c1c643ad4ba62e46c8b2035a89/e42270c6a7efce1b68bbf5aeb851f3deb58f650b.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=0443a330144f78f0800b9afb49300a83/e023f61f3a292df51f509171ab315c6035a87314.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=0afa28f0512309f7e76fad1a420f0c39/4d16b0fd5266d0164c94a34c802bd40734fa3515.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=628a53e1deea15ce41eee00186013a25/0c3df136afc379316be4bb4dfcc4b74542a91116.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=ab3da8b81ffa513d51aa6cd60d6c554c/a0f28194a4c27d1ee8a906a00cd5ad6edcc43817.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=cd93449f6ccb0a4685228b315b62f63e/a0ca33fae6cd7b895f319ce6182442a7d8330e47.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=4db63e3bcac451daf6f60ce386fc52a5/d86228dda3cc7cd9db7392bc2e01213fb90e9142.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=0b7aa52fcdb44aed594ebeec831d876a/c8f3b73533fa828b2b7c3b86ea1f4134960a5a41.jpg']

登录/注册后可看大图

"""

跳转到最佳答案楼层

编程鱼C · 发表于 2020-4-8 06:56:17

可能少了python中的pygame

LuLD · 发表于 2020-4-8 11:06:19

本帖最后由 LuLD 于 2020-4-8 11:13 编辑

====
p = <img width="[\d]+" height=“[\d]+" class="BDE_Image" src="([^"]+\.jpg)"

imglist : []
>>>

是利用正则来查找图片的吧

在返回的网页中没有查找到值

LuLD · 发表于 2020-4-8 11:20:11

我看了下实际网页上图片的
<img class="BDE_Image" src="http://tiebapic.baidu.com/forum/w%3D580/sign=b550f3b9bc64034f0fcdc20e9fc27980/f89992eef01f3a29d35ee8aa8e25bc315d607c09.jpg" size="1776777" changedsize="true" width="560" height="420">

你这个 :p = r'<img width="[\d]+" height=“[\d]+" class="BDE_Image" src="([^"]+\.jpg)"'

要改下 :p = r'<img class="BDE_Image" src="([^"]+\.jpg)"'

====
p = <img class="BDE_Image" src="([^"]+\.jpg)"

"""
imglist : ['http://tiebapic.baidu.com/forum/w%3D580/sign=b550f3b9bc64034f0fcdc20e9fc27980/f89992eef01f3a29d35ee8aa8e25bc315d607c09.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=40b49f7fc30735fa91f04eb1ae500f9f/202a30d12f2eb93870fc4796c2628535e4dd6f0a.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=2c9ec0c1c643ad4ba62e46c8b2035a89/e42270c6a7efce1b68bbf5aeb851f3deb58f650b.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=0443a330144f78f0800b9afb49300a83/e023f61f3a292df51f509171ab315c6035a87314.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=0afa28f0512309f7e76fad1a420f0c39/4d16b0fd5266d0164c94a34c802bd40734fa3515.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=628a53e1deea15ce41eee00186013a25/0c3df136afc379316be4bb4dfcc4b74542a91116.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=ab3da8b81ffa513d51aa6cd60d6c554c/a0f28194a4c27d1ee8a906a00cd5ad6edcc43817.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=cd93449f6ccb0a4685228b315b62f63e/a0ca33fae6cd7b895f319ce6182442a7d8330e47.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=4db63e3bcac451daf6f60ce386fc52a5/d86228dda3cc7cd9db7392bc2e01213fb90e9142.jpg', 'http://tiebapic.baidu.com/forum/w%3D580/sign=0b7aa52fcdb44aed594ebeec831d876a/c8f3b73533fa828b2b7c3b86ea1f4134960a5a41.jpg']

登录/注册后可看大图

"""

ijeju · 发表于 2020-4-8 19:33:31

LuLD 发表于 2020-4-8 11:20
我看了下实际网页上图片的

感谢大神，更改之后好使了！[code]但是我打开的html内容是这样的
<img width="560" height="568" class="BDE_Image" src="https://imgsa.baidu.com/forum/w%3D580/sign=da263e7098dda144da096cba82b6d009/c49cfe198618367a0c45e5be21738bd4b11ce5d7.jpg" size="197081" changedsize="true">
在img 和 class 之前的width和height不需要进行表达么？？？这个点有点没想明白，请求指教！

账号		自动登录	找回密码
密码			立即注册