Mcltui 发表于 2020-5-10 16:46:45

Python 爬取豆瓣TOP250 电影整理到EXCEL

Python 爬取豆瓣TOP250 电影整理到EXCELimport re
import requests
import xlsxwriter
from bs4 import BeautifulSoup


str1 = '\S+'
headers = {"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) "
                         "Chrome/83.0.4103.14 Safari/537.36 Edg/83.0.478.13"}

rank = []
for i in range(1, 251):
    rank.append('第' + str(i))

workbook = xlsxwriter.Workbook('豆瓣电影top250.xlsx')
worksheet = workbook.add_worksheet('sheet1')
heading = ['排名', '电影名']# 设置表头
worksheet.write_row('A1', heading)

list1 = []
for i in range(0, 250, 25):
    html = requests.get('https://movie.douban.com/top250?start=' + str(i) + '&filter=', headers=headers).text
    soup = BeautifulSoup(html, "html.parser")
    ol = soup.find('ol', class_='grid_view')
    a = ol.find_all('a')
    for each in a:
      name = re.findall(str1, each.text)
      if len(name) != 0:
            print(name)
            list1.append(name)

worksheet.write_column('B2', list1)# 为excel表的B列从B2开始写入数据
worksheet.write_column('A2', rank)
workbook.close()

乘号 发表于 2020-5-10 16:54:02

支持

Mcltui 发表于 2020-5-10 17:19:42

哈哈哈。没加COOkies 被限制啦
页: [1]
查看完整版本: Python 爬取豆瓣TOP250 电影整理到EXCEL