[已解决]爬虫

君子好逑 · 发表于 2020-8-19 18:53:14

您需要登录才可以下载或查看，没有账号？立即注册

x

import urllib.request
from bs4 import BeautifulSoup
import re
url = 'https://www.biduo.cc/biquge/39_39888/c13353637.html'
headers = {
'Accept-Language': 'zh-CN',
'Cache-Control': 'no-cache',
'Connection': 'Keep-Alive',
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363'
}
res = urllib.request.Request(url=url,headers=headers)
response = urllib.request.urlopen(res)
html = response.read().decode("gbk",'ignore')
regular = re.compile(".*?     ")
m = regular.findall(html)
for each in m:
each = each[:-32]
print(' ',end='')
if '    ' in each :
print(each.split(';')[-1])
else:
print(each)

复制代码

搞了个爬虫爬了一张小说，有没有大佬给改得更精简好用让我观摩一下

最佳答案

zltzlt

2020-8-19 18:55:57

import urllib.request
import re
url = 'https://www.biduo.cc/biquge/39_39888/c13353637.html'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363'
}
response = urllib.request.urlopen(urllib.request.Request(url=url, headers=headers))
html = response.read().decode("gbk", 'ignore')
for each in re.findall(".*?     ", html):
each = each[:-32]
print(' ', (each.split(';')[-1] if '    ' in each else each))

复制代码

zltzlt · 发表于 2020-8-19 18:55:57

import urllib.request
import re
url = 'https://www.biduo.cc/biquge/39_39888/c13353637.html'
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 '
'(KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.18363'
}
response = urllib.request.urlopen(urllib.request.Request(url=url, headers=headers))
html = response.read().decode("gbk", 'ignore')
for each in re.findall(".*?     ", html):
each = each[:-32]
print(' ', (each.split(';')[-1] if '    ' in each else each))

复制代码

小甲鱼的铁粉 · 发表于 2020-8-19 19:44:42

zltzlt 发表于 2020-8-19 18:55

zltzlt · 发表于 2020-8-19 19:50:40

小甲鱼的铁粉发表于 2020-8-19 19:44

君子好逑 · 发表于 2020-8-19 21:14:31

给大佬疯狂扣6

君子好逑 · 发表于 2020-8-19 23:36:04

zltzlt 发表于 2020-8-19 18:55

大佬，我想问一下.*和.*？有啥区别，啥时候用.*啥时候用.*?

zltzlt · 发表于 2020-8-20 07:25:49

君子好逑发表于 2020-8-19 23:36
大佬，我想问一下.*和.*？有啥区别，啥时候用.*啥时候用.*?

君子好逑 · 发表于 2020-8-20 12:13:35

zltzlt 发表于 2020-8-20 07:25
https://zhidao.baidu.com/question/297472922.html

账号		自动登录	找回密码
密码			立即注册