川A2580 发表于 2021-3-7 23:28:46

python爬虫爬取网页信息采集

import requests
if __name__ == "__main__":
    headers = {
      'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36 Edg/88.0.705.81'
      }
    url = 'https://www.baidu.com/s?'
    kw = input('enter a word:')
    param = {
      'wd':kw
    }
    response = requests.get(url = url, params = param,headers = headers)

    page_text = response.text
    fileName = kw+'.html'
    with open(fileName, 'w', encoding='utf-8') as fp:
      fp.write(page_text)
    print(fileName,'保存成功！！！')

#跑程序之后出现这样的报错
enter a word:小甲鱼
Traceback (most recent call last):
File "E:/python_game/爬取研究生.py", line 12, in <module>
    response = requests.get(url = url, params = param,headers = headers)
File "C:\Users\westbrook\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 76, in get
    return request('get', url, params=params, **kwargs)
File "C:\Users\westbrook\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
File "C:\Users\westbrook\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 528, in request
    prep = self.prepare_request(req)
File "C:\Users\westbrook\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 456, in prepare_request
    p.prepare(
File "C:\Users\westbrook\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\models.py", line 317, in prepare
    self.prepare_headers(headers)
File "C:\Users\westbrook\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\models.py", line 449, in prepare_headers
    for header in headers.items():
AttributeError: 'set' object has no attribute 'items'

有点混乱，请问大佬这是哪出问题了呢？

suchocolate 发表于 2021-3-8 17:03:11

    headers = {'user-agent':
      'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36 Edg/88.0.705.81'
      }

川A2580 发表于 2021-3-11 20:38:34

suchocolate 发表于 2021-3-8 17:03


是因为前面没有加user-agent吗

suchocolate 发表于 2021-3-12 00:35:36

川A2580 发表于 2021-3-11 20:38
是因为前面没有加user-agent吗

对。没有对应键值，headers那句就成了集合, 所以报错提示说set对象没有属性items。

川A2580 发表于 2021-3-12 21:29:05

suchocolate 发表于 2021-3-12 00:35
对。没有对应键值，headers那句就成了集合, 所以报错提示说set对象没有属性items。

明白了谢谢
页: [1]
查看完整版本: python爬虫爬取网页信息采集