爬虫标签
<div class="p-price"><strong class="J_100012820022" data-done="1">
<em>￥</em><i>3298.00</i>
</strong>
</div>
上面这个标签应该怎么找到3298.00，并且可以跳过<strong class="J_100012820022" data-done="1">？
price=[]
price=soup.find_all('div',class_="p-price")
for pri in price:
price.append(pri.i)
我是找到<div class="p-price">这个标签，然后pri.i直接找i这个节点，但是会报错AttributeError: 'NoneType' object has no attribute 'i'
price.append(pri.i.text)这样也会报错AttributeError: 'NoneType' object has no attribute 'text'
但是直接print(pri.i)却是有结果？
想知道原因？以及解决方法？
月魔同学 发表于 2020-8-9 17:25
用 try-except 过滤下 None 的即可
import requests
import bs4
import time
def open_url(url):
headers = {
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3706.400 SLBrowser/10.0.4040.400'}
res = requests.get(url, headers=headers)
return res
def find_phone(res):
soup = bs4.BeautifulSoup(res.text, 'html.parser')
# 手机名
names = []
name = soup.find_all('div', class_="p-name p-name-type-3")
for each in name:
names.append(each.a.em.text)
# 价格
price = []
price = soup.find_all('div', class_="p-price")
for pri in price:
try:
price.append(pri.i.text)
except:
pass
# 将找到的每一页的内容放进result中
result = []
length = len(names)
for i in range(length):
result.append(names + ':' + '\n' + str(price) + '\n')
return result
def main():
result = []
url = 'https://list.jd.com/list.html?cat=9987%2C653%2C655&page=1&s=1&click=0'
res = open_url(url)
result.extend(find_phone(res))
with open("E:\京东手机.txt", "w", encoding="utf-8") as f:
for each in result:
f.write(each)
time.sleep(1)
if __name__ == "__main__":
main() 本帖最后由 Twilight6 于 2020-8-9 17:23 编辑
加上 .text 是没错的，你报错的原因是因为有的没有爬到数据，所以返回 None ，而你对 None 直接 .text 肯定会导致报错的
发个完整代码看看吧
Twilight6 发表于 2020-8-9 17:22
加上 .text 是没错的，你报错的原因是因为有的没有爬到数据，所以返回 None ，而你对 None 直接 .text...
import requests
import bs4
import time
def open_url(url):
headers = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.25 Safari/537.36 Core/1.70.3706.400 SLBrowser/10.0.4040.400'}
res = requests.get(url, headers=headers)
returnres
def find_phone(res):
soup=bs4.BeautifulSoup(res.text, 'html.parser')
#手机名
names=[]
name=soup.find_all('div',class_="p-name p-name-type-3")
for each in name:
names.append(each.a.em.text)
#价格
price=[]
price=soup.find_all('div',class_="p-price")
for pri in price:
price.append(pri.i.text)
#将找到的每一页的内容放进result中
result = []
length = len(names)
for i in range(length):
result.append(names+':'+'\n'+price + '\n')
return result
def main():
result=[]
url='https://list.jd.com/list.html?cat=9987%2C653%2C655&page=1&s=1&click=0'
res = open_url(url)
result.extend(find_phone(res))
with open("E:\京东手机.txt", "w", encoding="utf-8") as f:
for each in result:
f.write(each)
time.sleep(1)
if __name__ == "__main__":
main()
Twilight6 发表于 2020-8-9 17:36
用 try-except 过滤下 None 的即可
是因为找到的有几个是None吗？所以过滤掉报错的信息 月魔同学 发表于 2020-8-9 17:42
是因为找到的有几个是None吗？所以过滤掉报错的信息
嗯 None 不能 .text
页:
[1]