君子好逑 发表于 2020-8-19 18:53:14

爬虫

import urllib.request
from bs4 import BeautifulSoup
import re


url = 'https://www.biduo.cc/biquge/39_39888/c13353637.html'

headers = {
'Accept-Language': 'zh-CN',
'Cache-Control': 'no-cache',
'Connection': 'Keep-Alive',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363'
}

res = urllib.request.Request(url=url,headers=headers)

response = urllib.request.urlopen(res)
html = response.read().decode("gbk",'ignore')

regular = re.compile(".*?<br><br>&nbsp;&nbsp;&nbsp;&nbsp;")



m = regular.findall(html)

for each in m:
    each = each[:-32]
    print('   ',end='')
    if '&nbsp;&nbsp;&nbsp;&nbsp;' in each :
      print(each.split(';')[-1])
    else:
      print(each)
搞了个爬虫爬了一张小说，有没有大佬给改得更精简好用让我观摩一下

zltzlt 发表于 2020-8-19 18:55:57

import urllib.request
import re

url = 'https://www.biduo.cc/biquge/39_39888/c13353637.html'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
                  '(KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363'
}

response = urllib.request.urlopen(urllib.request.Request(url=url, headers=headers))
html = response.read().decode("gbk", 'ignore')

for each in re.findall(".*?<br><br>&nbsp;&nbsp;&nbsp;&nbsp;", html):
    each = each[:-32]
    print('    ', (each.split(';')[-1] if '&nbsp;&nbsp;&nbsp;&nbsp;' in each else each))

小甲鱼的铁粉 发表于 2020-8-19 19:44:42

zltzlt 发表于 2020-8-19 18:55


{:10_275:}

zltzlt 发表于 2020-8-19 19:50:40

小甲鱼的铁粉 发表于 2020-8-19 19:44


{:10_278:}

君子好逑 发表于 2020-8-19 21:14:31

给大佬疯狂扣6{:10_256:}

君子好逑 发表于 2020-8-19 23:36:04

zltzlt 发表于 2020-8-19 18:55


大佬，我想问一下.*和.*？有啥区别，啥时候用.*啥时候用.*?

zltzlt 发表于 2020-8-20 07:25:49

君子好逑 发表于 2020-8-19 23:36
大佬，我想问一下.*和.*？有啥区别，啥时候用.*啥时候用.*?

https://zhidao.baidu.com/question/297472922.html

君子好逑 发表于 2020-8-20 12:13:35

zltzlt 发表于 2020-8-20 07:25
https://zhidao.baidu.com/question/297472922.html

{:10_256:}
页: [1]
查看完整版本: 爬虫