论坛 › Python交流 › 爬取豆瓣电影信息

摆动的呆毛 发表于 2020-8-7 21:53:41

爬取豆瓣电影信息

import requests
import bs4
import time

with open('./豆瓣.txt',mode='w+',encoding='utf-8') as f:
    headers = {
            'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
      }
    q = input("输入要爬几页")
    for i in range(int(q)):
      url = "https://movie.douban.com/top250"
      params = {q : (i * 25)}
      res = requests.get(url=url, headers=headers, params=params)
      soup = bs4.BeautifulSoup(res.text, "html.parser")
      targets = soup.find_all("div", class_="hd")
      for each in targets:
            f.write(each.a.text)
      time.sleep(1)


为什么无法爬取下一页，一直在第一页里循环

qiuyouzhi 发表于 2020-8-7 22:01:17

import requests
import bs4
import time

with open('./豆瓣.txt',mode='w+',encoding='utf-8') as f:
    headers = {
            'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
      }
    q = input("输入要爬几页")
    for i in range(int(q)):
      url = f"https://movie.douban.com/top250?start={i * 25}&filter="
      res = requests.get(url=url, headers=headers)
      soup = bs4.BeautifulSoup(res.text, "html.parser")
      targets = soup.find_all("div", class_="hd")
      for each in targets:
            f.write(each.a.text)
      time.sleep(1)

1q23w31 发表于 2020-8-7 22:07:38

你一直请求第一页当然不变

陈尚涵 发表于 2020-8-8 17:03:50

一直是第1页{:10_257:}

心驰神往 发表于 2020-11-3 11:13:55

鱼币

a~qie 发表于 2020-11-17 21:30:49

qiuyouzhi 发表于 2020-8-7 22:01


现在没用了吗？

查看完整版本: 爬取豆瓣电影信息