天天上头条 发表于 2017-11-28 14:56:26

关于爬取IP的问题，求大神指导

#coding=utf-8
import re
import urllib.request
import urllib.parse
import os
url='http://www.kuaidaili.com/ops/' #http://www.xicidaili.com/http://www.proxy360.cn/default.aspx
headers={'Accept':'text/html,application/xhtml+xm…plication/xml;q=0.9,*/*;q=0.8',
         'Accept-Encoding':'gzip,deflate',
         'Accept-Language':'zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2',
         'Host':'fs.xicidaili.com',
         'Cookie':'Hm_lvt_0cf76c77469e965d2957f0553e6ecf59=1511441689,1511784134,1511785760; Hm_lpvt_0cf76c77469e965d2957f0553e6ecf59=1511785760',
         'Connection':'keep-alive',
         'User-Agent':'Mozilla/5.0 (Windows NT 6.1; W…) Gecko/20100101 Firefox/57.0'
    }
headers=urllib.parse.urlencode(headers).encode('utf8')
rep=urllib.request.Request(url,headers)
reponse=urllib.request.urlopen(rep)
html=reponse.read().decode(encoding='UTF-8')
#print (html)
p=re.compile(r'((1\.)|(2\.)|(25\.)|(\.)|(\.)){3}((1)|(2)|(25)|()|())')
ip_list=p.search(html)
print(ip_list)
print(ip_list.group())

为什么我这只抓取了一个IP？

天天上头条 发表于 2017-11-30 19:19:55

#coding=utf-8
import re
import urllib.request
import urllib.parse
import os
from urllib.error import URLError,HTTPError
url='http://www.kuaidaili.com/ops/' #http://www.xicidaili.com/http://www.proxy360.cn/default.aspxhttp://www.kuaidaili.com/ops/
headers={'Accept':'text/html,application/xhtml+xm…plication/xml;q=0.9,*/*;q=0.8',
         'Accept-Encoding':'gzip,deflate',
         'Accept-Language':'zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2',
         'Host':'fs.xicidaili.com',
         'Cookie':'Hm_lvt_0cf76c77469e965d2957f0553e6ecf59=1511441689,1511784134,1511785760; Hm_lpvt_0cf76c77469e965d2957f0553e6ecf59=1511785760',
         'Connection':'keep-alive',
         'User-Agent':'Mozilla/5.0 (Windows NT 6.1; W…) Gecko/20100101 Firefox/57.0'
}
headers=urllib.parse.urlencode(headers).encode('utf8')
rep=urllib.request.Request(url,headers)
reponse=urllib.request.urlopen(rep)
html=reponse.read().decode(encoding='UTF-8')
reg=r'\b(?:(?:25|2|??)\.){3}(?:25|2|??)\b'
ip_list=re.findall(reg,html)
print(ip_list)
f=open('ip.txt','a')
f.write(str(ip_list)+'\n')
f.close()

天天上头条 发表于 2017-11-30 19:21:09

search只返回一个值
页: [1]
查看完整版本: 关于爬取IP的问题，求大神指导