Python爬虫response不是标准Json格式，怎么解析？

15882459495

在爬取http://vip.win007.com/count/goal ... 1988099&cid=115，图1中红色方框内数据

网页是动态网页，但是response对象不是标准的json格式，如图2所示，有前缀，
怎么把前面的
“var goalPageInfo = [5,1];
var goalPageData =”
信息去掉，留下后面的内容？


import json
import requests

url = 'http://vip.win007.com/count/goalData.aspx?p=1&sp=1&t=5&time=2021/5/28%2019:00:00&hw=2.3&gw=3&g=3&cid=115&r=1&t2=1&scid=466&flesh=0.22358079357785554'
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36"
    }
#r = requests.get(url=url,headers=headers).json()

#print(r)

r = requests.get(url=url,headers=headers)
# 调用get方法，下载
#print(type(r.content.decode('gbk')))
#print(r.content.decode('gbk'))

ret = r.content.decode('gbk')
print(ret)
result = json.loads(ret)
print(result)

代码会报错

wp231957

这就是去掉前面部分也只是列表啊，和json也不沾边

笨鸟学飞

试试ujson库

wp231957

1. import requests
   
2. 
   
3. url = 'http://vip.win007.com/count/goalData.aspx?p=1&sp=1&t=5&time=2021/5/28%2019:00:00&hw=2.3&gw=3&g=3&cid=115&r=1&t2=1&scid=466&flesh=0.22358079357785554'
   
4. headers = {
   
5.     "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36"
   
6.     }
   
7. r = requests.get(url=url,headers=headers)
   
8. r.encoding="gbk"
   
9. lst=r.text.split("=")[2][:-2]
   
10. for x in eval(lst):
    
11.     print(x)

复制代码

YunGuo

1. import requests
   
2. import re
   
3. 
   
4. url = 'http://vip.win007.com/count/goalData.aspx?p=1&sp=1&t=5&time=2021/5/28%2019:00:00&hw=2.3&gw=3&g=3&cid=115&r=1&t2=1&scid=466&flesh=0.22358079357785554'
   
5. headers = {
   
6.     'User-Agent': 'Mozilla/5.0'
   
7. }
   
8. res = requests.get(url, headers=headers)
   
9. datas = eval(re.findall('Data = (.*?);', res.content.decode('gbk'))[0])
   
10. for data in datas:
    
11.     print(data)

复制代码

kaohsing

1. import re
   
2. import requests
   
3. 
   
4. res = requests.get(
   
5.     url='http://vip.win007.com/count/goalData.aspx?p=1&sp=1&t=5&time=2021/5/28%2019:00:00&hw=2.3&gw=3&g=3&cid=115&r=1&t2=1&scid=466',
   
6.     headers={  'User-Agent': 'Mozilla/5.0' }
   
7. )
   
8. 
   
9. m = re.findall("(\[\[.*?\]\])",res.content.decode('gbk'), re.S|re.M)[0]
   
10. for  l in eval(m):
    
11.     print(l)
    

复制代码