|
马上注册,结交更多好友,享用更多功能^_^
您需要 登录 才可以下载或查看,没有账号?立即注册
x
求助各位大佬啦,拜托
定向爬取网页上的内容生成一个列表可是老是提示list index out of range啥情况啊,发现要爬的table标签下每个数据在tr标签下的td标签里,但是tr标签下有时是6个td优势是5个td因此写了一个if语句可是还是提示list index out of range
我的代码import requests
from bs4 import BeautifulSoup
import bs4
def getHTML(url):
try:
r = requests.get(url,timeout=30)
r.raise_for_status()
r.encoding = r.apparent_encoding
return r.text
except:
return ""
def fillTable(table,html):
soup = BeautifulSoup(html,"html.parser")
for tr in soup.find('table').children:
if isinstance(tr,bs4.element.Tag):
tds = tr('td')
if len(table) == 6:
table.append([tds[0].string,tds[1].string,tds[2].string,tds[3].string,tds[4].string,tds[5].string])
else :
table.append([tds[0].string,tds[1].string,tds[2].string,tds[3].string,tds[4].string])
print(tds)
def printTable(table):
print("{:^10}\t{:^10}\t{:^10}\t{:^6}\t{:^6}\t{:^6}".format("1","2","3","4","5","6"))
def main():
tableinfo = []
url = "http://www.cde.org.cn/guide.do?method=getIchTypeList&type1=1"
html = getHTML(url)
fillTable(tableinfo,html)
printTable(tableinfo)
main()
下面是一个6个td标签的tr,url网页在上面也有大家可以看一下源代码
|
|