[已解决]这个总统数据库怎么爬？

故里 · 发表于 2021-10-13 09:37:49

马上注册，结交更多好友，享用更多功能^_^

您需要登录才可以下载或查看，没有账号？立即注册

x

https://www.worldpresidentsdb.com/Yang-Shangkun/
https://www.worldpresidentsdb.com/list/gender/female/

最佳答案

月排行榜 / 总排行榜

suchocolate

2021-10-13 18:22:16

import requests
from lxml import etree


def main():
    base_url = 'https://www.worldpresidentsdb.com'
    url = 'https://www.worldpresidentsdb.com/list/gender/female/'
    headers = {'user-agent': 'firefox'}
    r = requests.get(url, headers=headers)
    html = etree.HTML(r.text)
    psts = html.xpath('//div[@class="list-group"]//@href')
    for pst in psts:
        url = f'{base_url}{pst}'
        r = requests.get(url, headers=headers)
        html = etree.HTML(r.text)
        info = html.xpath('//div[contains(@class,"col-md-8")]/p//text()')
        print(info)


if __name__ == '__main__':
    main()

跳转到最佳答案楼层

suchocolate · 发表于 2021-10-13 18:22:16

这个最佳答案由 suchocolate 给出，感谢 suchocolate 的回答。

单击隐藏图章

import requests
from lxml import etree


def main():
    base_url = 'https://www.worldpresidentsdb.com'
    url = 'https://www.worldpresidentsdb.com/list/gender/female/'
    headers = {'user-agent': 'firefox'}
    r = requests.get(url, headers=headers)
    html = etree.HTML(r.text)
    psts = html.xpath('//div[@class="list-group"]//@href')
    for pst in psts:
        url = f'{base_url}{pst}'
        r = requests.get(url, headers=headers)
        html = etree.HTML(r.text)
        info = html.xpath('//div[contains(@class,"col-md-8")]/p//text()')
        print(info)


if __name__ == '__main__':
    main()

账号		自动登录	找回密码
密码			立即注册