|
10鱼币
[
{
"a" : "aa" ,
"b" : "bb" ,
},
{
"a" : "cc",
"b" : "dd",
}
]
[
{
"a" : "11" ,
"b" : "22" ,
},
{
"a" : "33",
"b" : "44",
},
{
"a" : "55",
"b" : "66"
}
]
爬取到的格式如上(a和b不变,只有后边跟的数据变),实际远远比上边的数组多。
如何在pycharm中改格式才能方便把json数组里的数据存入到mysql中?
求有经验的大佬们指点指点
- import requests
- import pymysql
- def main():
- # 准备数据库连接,假如已有数据库test,其中已有stock这个表
- db = pymysql.connect(host='localhost', user='root', password='123456', port=3306, db='test')
- cursor = db.cursor()
- sql = 'INSERT INTO stock(id, name, symbol) values(%s, %s, %s)'
- # 获取数据
- url = 'http://vip.stock.finance.sina.com.cn/quotes_service/api/json_v2.php/Market_Center.getHQFuturesData?page=1&num=5&sort=position&asc=0&node=dlm_qh&base=futures'
- headers = {'user-agent': 'firefox'}
- r = requests.get(url, headers=headers)
- j_data = r.json()
- # 处理数据,拿name和symbol这2个值做例子
- for n, i in enumerate(j_data):
- nm = i['name']
- sy = i['symbol']
- print(nm)
- print(sy)
- # 写数据库
- try:
- cursor.execute(sql, (n, nm, sy))
- db.commit()
- except:
- db.rollback()
- db.close()
- if __name__ == '__main__':
- main()
复制代码
|
|