[已解决]爬虫

君子好逑

import urllib.request
from bs4 import BeautifulSoup
import re


url = 'https://www.biduo.cc/biquge/39_39888/c13353637.html'

headers = {
'Accept-Language': 'zh-CN',
'Cache-Control': 'no-cache',
'Connection': 'Keep-Alive',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363'
}

res = urllib.request.Request(url=url,headers=headers)

response = urllib.request.urlopen(res)
html = response.read().decode("gbk",'ignore')

regular = re.compile(".*?<br><br>    ")



m = regular.findall(html)

for each in m:
    each = each[:-32]
    print('     ',end='')
    if '    ' in each :
        print(each.split(';')[-1])
    else:
        print(each)

搞了个爬虫爬了一张小说，有没有大佬给改得更精简好用让我观摩一下

最佳答案

月排行榜 / 总排行榜

zltzlt

2020-8-19 18:55:57

import urllib.request
import re

url = 'https://www.biduo.cc/biquge/39_39888/c13353637.html'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
                  '(KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363'
}

response = urllib.request.urlopen(urllib.request.Request(url=url, headers=headers))
html = response.read().decode("gbk", 'ignore')

for each in re.findall(".*?<br><br>    ", html):
    each = each[:-32]
    print('    ', (each.split(';')[-1] if '    ' in each else each))

跳转到最佳答案楼层

zltzlt

import urllib.request
import re

url = 'https://www.biduo.cc/biquge/39_39888/c13353637.html'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
                  '(KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363'
}

response = urllib.request.urlopen(urllib.request.Request(url=url, headers=headers))
html = response.read().decode("gbk", 'ignore')

for each in re.findall(".*?<br><br>    ", html):
    each = each[:-32]
    print('    ', (each.split(';')[-1] if '    ' in each else each))

小甲鱼的铁粉

zltzlt 发表于 2020-8-19 18:55

zltzlt

小甲鱼的铁粉 发表于 2020-8-19 19:44

君子好逑

给大佬疯狂扣6

君子好逑

zltzlt 发表于 2020-8-19 18:55

大佬，我想问一下.*和.*？有啥区别，啥时候用.*啥时候用.*?

zltzlt

君子好逑 发表于 2020-8-19 23:36
大佬，我想问一下.*和.*？有啥区别，啥时候用.*啥时候用.*?

https://zhidao.baidu.com/question/297472922.html

君子好逑

zltzlt 发表于 2020-8-20 07:25
https://zhidao.baidu.com/question/297472922.html