[已解决]爬虫

哈岁NB · 发表于 2023-1-17 11:06:34

马上注册，结交更多好友，享用更多功能^_^

您需要登录才可以下载或查看，没有账号？立即注册

x

import requests
from lxml import html
etree = html.etree

url = 'http://fund.eastmoney.com/data/rankhandler.aspx'
header = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/5.36"

}

parms = {
    "op": "ph",
    "dt": "kf",
    "ft": "zq",
    "rs":"",
    "gs": "0",
    "sc": "1nzf",
    "st": "desc",
    "sd": "2022-01-17",
    "ed": "2023-01-17",
    "qdii": "041|",
    "tabSubtype": "041,,,,,",
    "pi": "1",
    "pn": "50",
    "dx": "1",
    "v": "0.5745516544026257"

}

response = requests.get(url=url,headers=header,params=parms)

page = response.text
print(page)

大佬们。为什么这个一爬取就显示var rankData ={ErrCode:-999,Data:"无访问权限"}

最佳答案

月排行榜 / 总排行榜

isdkz

2023-1-17 11:34:41

import requests from lxml import html etree = html.etree url = 'http://fund.eastmoney.com/data/rankhandler.aspx' header = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/5.36", 'Referer':'http://fund.eastmoney.com/data/fundranking.html' # 加上这个，说明你是从哪个链接发起请求的，有一些url只允许从特定的链接发起请求 } parms = { "op": "ph", "dt": "kf", "ft": "zq", "rs":"", "gs": "0", "sc": "1nzf", "st": "desc", "sd": "2022-01-17", "ed": "2023-01-17", "qdii": "041|", "tabSubtype": "041,,,,,", "pi": "1", "pn": "50", "dx": "1", "v": "0.5745516544026257" } response = requests.get(url=url,headers=header,params=parms) page = response.text print(page)

跳转到最佳答案楼层

tommyyu · 发表于 2023-1-17 11:14:09

本帖最后由 tommyyu 于 2023-1-17 11:17 编辑

这个的页面是不是就只有“var rankData ={ErrCode:-999,Data:"无访问权限"}”，为啥我点开看也是这样

isdkz · 发表于 2023-1-17 11:14:59

你这个链接不是还带了别的参数吗？在请求的时候 url 带上完整的参数，直接在那里右键复制url

哈岁NB · 发表于 2023-1-17 11:17:35

isdkz 发表于 2023-1-17 11:14
你这个链接不是还带了别的参数吗？在请求的时候 url 带上完整的参数，直接在那里右键复制url

参数都带了，还是显示var rankData ={ErrCode:-999,Data:"无访问权限"}

哈岁NB · 发表于 2023-1-17 11:17:52

tommyyu 发表于 2023-1-17 11:14
这个的页面是不是就只有“var rankData ={ErrCode:-999,Data:"无访问权限"}”，为啥我访问也是这样{:10_282 ...

我也不知道

isdkz · 发表于 2023-1-17 11:21:51

哈岁NB 发表于 2023-1-17 11:17
参数都带了，还是显示var rankData ={ErrCode:-999,Data:"无访问权限"}

你这个请求是打开哪个链接出来的，我帮你调试看看

isdkz · 发表于 2023-1-17 11:33:17

加上 Referer，这个字段是说明你是从哪个链接发起请求的，这是一种防盗链的手段，只允许你从特定的链接发起请求
import requests from lxml import html etree = html.etree url = 'http://fund.eastmoney.com/data/rankhandler.aspx' header = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/5.36", 'Referer':'http://fund.eastmoney.com/data/fundranking.html' # 加上这个 } parms = { "op": "ph", "dt": "kf", "ft": "zq", "rs":"", "gs": "0", "sc": "1nzf", "st": "desc", "sd": "2022-01-17", "ed": "2023-01-17", "qdii": "041|", "tabSubtype": "041,,,,,", "pi": "1", "pn": "50", "dx": "1", "v": "0.5745516544026257" } response = requests.get(url=url,headers=header,params=parms) page = response.text print(page)

isdkz · 发表于 2023-1-17 11:34:41

import requests from lxml import html etree = html.etree url = 'http://fund.eastmoney.com/data/rankhandler.aspx' header = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/5.36", 'Referer':'http://fund.eastmoney.com/data/fundranking.html' # 加上这个，说明你是从哪个链接发起请求的，有一些url只允许从特定的链接发起请求 } parms = { "op": "ph", "dt": "kf", "ft": "zq", "rs":"", "gs": "0", "sc": "1nzf", "st": "desc", "sd": "2022-01-17", "ed": "2023-01-17", "qdii": "041|", "tabSubtype": "041,,,,,", "pi": "1", "pn": "50", "dx": "1", "v": "0.5745516544026257" } response = requests.get(url=url,headers=header,params=parms) page = response.text print(page)

哈岁NB · 发表于 2023-1-17 11:40:29

isdkz 发表于 2023-1-17 11:34

好的好的，成功了

账号		自动登录	找回密码
密码			立即注册