[已解决]爬取豆瓣电影信息

摆动的呆毛

1. import requests
   
2. import bs4
   
3. import time
   
4. 
   
5. with open('./豆瓣.txt',mode='w+',encoding='utf-8') as f:
   
6.     headers = {
   
7.             'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
   
8.         }
   
9.     q = input("输入要爬几页")
   
10.     for i in range(int(q)):
    
11.         url = "https://movie.douban.com/top250"
    
12.         params = {q : (i * 25)}
    
13.         res = requests.get(url=url, headers=headers, params=params)
    
14.         soup = bs4.BeautifulSoup(res.text, "html.parser")
    
15.         targets = soup.find_all("div", class_="hd")
    
16.         for each in targets:
    
17.             f.write(each.a.text)
    
18.         time.sleep(1)

复制代码

为什么无法爬取下一页，一直在第一页里循环

最佳答案

月排行榜 / 总排行榜

qiuyouzhi

2020-8-7 22:01:17

1. import requests
   
2. import bs4
   
3. import time
   
4. 
   
5. with open('./豆瓣.txt',mode='w+',encoding='utf-8') as f:
   
6.     headers = {
   
7.             'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
   
8.         }
   
9.     q = input("输入要爬几页")
   
10.     for i in range(int(q)):
    
11.         url = f"https://movie.douban.com/top250?start={i * 25}&filter="
    
12.         res = requests.get(url=url, headers=headers)
    
13.         soup = bs4.BeautifulSoup(res.text, "html.parser")
    
14.         targets = soup.find_all("div", class_="hd")
    
15.         for each in targets:
    
16.             f.write(each.a.text)
    
17.         time.sleep(1)

复制代码

跳转到最佳答案楼层

qiuyouzhi

1. import requests
   
2. import bs4
   
3. import time
   
4. 
   
5. with open('./豆瓣.txt',mode='w+',encoding='utf-8') as f:
   
6.     headers = {
   
7.             'User-Agent':'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'
   
8.         }
   
9.     q = input("输入要爬几页")
   
10.     for i in range(int(q)):
    
11.         url = f"https://movie.douban.com/top250?start={i * 25}&filter="
    
12.         res = requests.get(url=url, headers=headers)
    
13.         soup = bs4.BeautifulSoup(res.text, "html.parser")
    
14.         targets = soup.find_all("div", class_="hd")
    
15.         for each in targets:
    
16.             f.write(each.a.text)
    
17.         time.sleep(1)

复制代码

1q23w31

你一直请求第一页当然不变

陈尚涵

一直是第1页

心驰神往

鱼币

a~qie

qiuyouzhi 发表于 2020-8-7 22:01

现在没用了吗？