[已解决]淘宝客的爬取，一直成功不了

jia3168 · 发表于 2017-4-16 16:18:42

马上注册，结交更多好友，享用更多功能^_^

您需要登录才可以下载或查看，没有账号？立即注册

x

import urllib.request,http.cookies,http.cookiejar
import os
import re
import gzip
def ungzip(data):
try:
print('正在解压.....')
data = gzip.decompress(data)
print('解压完毕!')
except:
print('未经压缩, 无需解压')
return data
def url_open(url):
req = urllib.request.Request(url,header)
response = urllib.request.urlopen(url)
html = response.read()
header={
"Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
"Accept-Encoding":"gzip, deflate, sdch",
"Accept-Language":"zh-CN,zh;q=0.8",
"Cache-Control":"max-age=0",
"Connection":"keep-alive",
"Cookie":"t=27d236359f46ea30e60db59c54ca2646; UM_distinctid=15b7195dc3ee99-0595dcceecedbe-31437652-1fa400-15b7195dc486fbe; account-path-guide-s1=true; pub-message-center=1; cookie2=68d900c3e873043561fb40c8b75c0bfe; v=0; _tb_token_=OMIt2M8EtsXq; cookie32=e9905dcb39a4085fdd37062f8f590fa8; cookie31=NDUyNjM3MDEsJUU2JUIxJTlGJUU1JTlDJUEzJUU1JUJFJUI3LDU2NDc0MjQzOUBxcS5jb20sVEI%3D; alimamapwag=TW96aWxsYS81LjAgKFdpbmRvd3MgTlQgNi4xOyBXT1c2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzUzLjAuMjc4NS4xMDQgU2FmYXJpLzUzNy4zNiBDb3JlLzEuNTMuMjU5NS40MDAgUVFCcm93c2VyLzkuNi4xMDg3Mi40MDA%3D; login=VFC%2FuZ9ayeYq2g%3D%3D; alimamapw=QHQhQyQgFXZyFnENEnEHQyIGbAZQAlFSUQIEA1INAAoABQcEBgVSVVYGAQICVQQJU1ZQ; cna=VlPIEAQQlAMCAbZWgx6zO5uo; l=Anh4kjqD1bYkHF5trROs0AtryCwK4dxr; isg=AmBg35RPLFCL65BVmwqrIFS4JG58kEQzWH9X_dpxLHsO1QD_gnkUwzbnG_ow",
"Host":"pub.alimama.com",
"Upgrade-Insecure-Requests":"1",
"User-Agent":"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.104 Safari/537.36 Core/1.53.2595.400 QQBrowser/9.6.10872.400",
}
'''
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip, deflate, sdch
Accept-Language:zh-CN,zh;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
Cookie:t=27d236359f46ea30e60db59c54ca2646; UM_distinctid=15b7195dc3ee99-0595dcceecedbe-31437652-1fa400-15b7195dc486fbe; account-path-guide-s1=true; pub-message-center=1; cookie2=68d900c3e873043561fb40c8b75c0bfe; v=0; _tb_token_=OMIt2M8EtsXq; cookie32=e9905dcb39a4085fdd37062f8f590fa8; cookie31=NDUyNjM3MDEsJUU2JUIxJTlGJUU1JTlDJUEzJUU1JUJFJUI3LDU2NDc0MjQzOUBxcS5jb20sVEI%3D; alimamapwag=TW96aWxsYS81LjAgKFdpbmRvd3MgTlQgNi4xOyBXT1c2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzUzLjAuMjc4NS4xMDQgU2FmYXJpLzUzNy4zNiBDb3JlLzEuNTMuMjU5NS40MDAgUVFCcm93c2VyLzkuNi4xMDg3Mi40MDA%3D; login=VFC%2FuZ9ayeYq2g%3D%3D; alimamapw=QHQhQyQgFXZyFnENEnEHQyIGbAZQAlFSUQIEA1INAAoABQcEBgVSVVYGAQICVQQJU1ZQ; cna=VlPIEAQQlAMCAbZWgx6zO5uo; l=Anh4kjqD1bYkHF5trROs0AtryCwK4dxr; isg=AmBg35RPLFCL65BVmwqrIFS4JG58kEQzWH9X_dpxLHsO1QD_gnkUwzbnG_ow
Host:pub.alimama.com
Referer:http://pub.alimama.com/promo/item/channel/index.htm?spm=a219t.7664554.1998457203.146.cnd11H&channel=9k9
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.104 Safari/537.36 Core/1.53.2595.400 QQBrowser/9.6.10872.400
'''
url='http://pub.alimama.com/promo/search/index.htm?'
html=url_open(url)
html=ungzip(html)
print('1')
html=html.decode('utf-8','ignore')
print('1')
print(html)

复制代码

是cookie的事还是什么？

最佳答案

月排行榜 / 总排行榜

ooxx7788

2017-4-16 17:49:25

本帖最后由 ooxx7788 于 2017-4-16 17:55 编辑

return html

复制代码

19行增加以上代码！没有返回值。

跳转到最佳答案楼层

Seboomm · 发表于 2017-4-16 17:11:17

淘宝做了限制吧，

ooxx7788 · 发表于 2017-4-16 17:49:25

这个最佳答案由 ooxx7788 给出，感谢 ooxx7788 的回答。

单击隐藏图章

本帖最后由 ooxx7788 于 2017-4-16 17:55 编辑

return html

复制代码

19行增加以上代码！没有返回值。

jia3168 · 发表于 2017-4-16 20:26:34

ooxx7788 发表于 2017-4-16 17:49
19行增加以上代码！没有返回值。

不过还是没用

<!DOCTYPE html>
<html>
<head>
<meta name="data-spm" content="a219t" />
<meta name="aplus-ajax" content="46807174">
<title>淘宝联盟</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width,initial-scale=1.0, user-scalable=no"/>
<meta name="renderer" content="webkit">
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"/>
<link rel="stylesheet" href="//g.alicdn.com/thx/cube/1.2.1/neat.css">
<link rel="stylesheet" href="//g.alicdn.com/thx/minecraft-animation/20151106.161602.306/css/animate-min.css">
<link rel="stylesheet" href="//g.alicdn.com/mm/pubplus/0.3.10/style/main.css">
<script src="//g.alicdn.com/thx/brix-release/1.0.0-beta.9/require-config-debug.js"></script>
<script src="//g.alicdn.com/mm/pubplus/0.3.10/app/aliww.js"></script>
<script src="//g.alicdn.com/crm/anywhere/1.0.88/lib/include.js"></script>
<script src="//g.alicdn.com/secdev/pointman/js/index.js" app="union-pub"></script>
<!--[if lte IE 7]>
<script src="//g.alicdn.com/mm/pubplus/0.3.10/app/exts/ieupdate/ieupdate.js"></script>
<![endif]-->
</head>
<body data-spm="7900221"><script>
with(document)with(body)with(insertBefore(createElement("script"),firstChild))setAttribute("exparams","category=&userid=&aplus&yunid=&&trid=&asid=AQAAAACeXPNYVYe5cQAAAACI0O+FcR2nUw==",id="tb-beacon-aplus",src=(location>"https"?"//g":"//g")+".alicdn.com/alilog/mlog/aplus_v2.js")
</script>
<script src="//g.alicdn.com/mm/pubplus/0.3.10/app/boot.js"></script>
</body>
</html>

复制代码

html是这个，进不去

账号		自动登录	找回密码
密码			立即注册

[已解决]淘宝客的爬取，一直成功不了

马上注册，结交更多好友，享用更多功能^_^

浏览过的版块