[已解决]爬虫

kygschp

求救各位大神，下面的代码运行后，啥东西都没打印出来

import requests
import bs4

response = requests.get('https://movie.douban.com/typerank?type_name=%E5%89%A7%E6%83%85&type=11&interval_id=100:90&action=')
soup = bs4.BeautifulSoup(response.text, 'html.parser')
targets = soup.find_all('div', class_='movie-name')

for each in targets:
    print(each.span.a.text)

最佳答案

月排行榜 / 总排行榜

suchocolate

2021-7-17 14:13:32

加header，不加默认是python-requests，会被反扒的。

import requests
import bs4

headers = {'user-agent': 'Mozilla'}
response = requests.get('https://movie.douban.com/typerank?type_name=%E5%89%A7%E6%83%85&type=11&interval_id=100:90&action=', headers=headers)
soup = bs4.BeautifulSoup(response.text, 'html.parser')
targets = soup.find_all('div', class_='movie-name')

for each in targets:
    print(each.span.a.text)

跳转到最佳答案楼层

suchocolate

加header，不加默认是python-requests，会被反扒的。

import requests
import bs4

headers = {'user-agent': 'Mozilla'}
response = requests.get('https://movie.douban.com/typerank?type_name=%E5%89%A7%E6%83%85&type=11&interval_id=100:90&action=', headers=headers)
soup = bs4.BeautifulSoup(response.text, 'html.parser')
targets = soup.find_all('div', class_='movie-name')

for each in targets:
    print(each.span.a.text)