[已解决]抓取下来的网页内容为乱码（见正文），怎么转换成正常的可读模式？

阳崽崽 · 发表于 2021-6-20 00:41:48

马上注册，结交更多好友，享用更多功能^_^

您需要登录才可以下载或查看，没有账号？立即注册

x

目标网页：https://www.shicimingju.com/book/shiji.html
网页get下载之后，用notpad++打开的中文内容为：ç»¼åc2\xa6\xc2\x9c，尝试了很多方法都没办法转换成中文可读模式，求助大神帮忙解决

，python版本为3.9.4
内容截图：
QQ截图20210620003527.png

网页附件：

示例文件.zip (4.8 KB, 下载次数: 4)

最佳答案

月排行榜 / 总排行榜

fc5igm

2021-6-21 00:01:37

这网址没有任何问题啊
你一定没看小甲鱼的课后作业

'<!DOCTYPE html>\n<html lang="zh">\n<head>\n\t<script type="text/javascript" src="https://ip.ws.126.net/ipquery"></script>\n <script src="/newpage/js/all.js"></script>\n <meta charset="UTF-8">\n <title>《史记》全集在线阅读_史书典籍_诗词名句网</title>\n <meta name="viewport" content="width=device-width,user-scalable=no" />\n <meta name="description" content="" />\n <link rel="stylesheet" href="/public/css/www_v3.css">\n <link rel="stylesheet" href="/public/css/font_576092_m2icqcebp7evzpvi.css">\n <script src="/public/js/device.js"></script>\n <script src="/public/js/jq.js"></script>\n <script src="/public/js/www_v3.js"></script>\n</head>\n<body>\n<div id="top_bar">\n <div id="top_bar_main">\n <div id="top_left">\n <a href="/"><img src="/public/image/logo.png" id="logo"/></a>\n <div id="top_left_menu">\n <ul>\n <li><a href="/">首页</a></li>\n <li><a href="/shicimark">分类</a></li>\n <li><a href="/category/all">作者</a></li>\n <li><a href="/paiming">排行榜</a></li>\n <li><a href="/cate?cate_id=4">课本古诗</a></li>\n <li><a href="/cipai/index.html">词牌名</a></li>\n <li><a href="/cangtoushi/index.html">藏头诗</a></li>\n <li><a href="/hecheng/index.html">合称</a></li>\n <li><a href="/book/">古籍</a></li>\n\t\t <li><a href="/app/">安卓下载</a></li>\n </ul>\n </div>\n <div style="clear: both"></div>\n </div>\n <div id="top_right">\n <div id="top_right_nav">\n <ul>\n <li><a href="javascript:void(0)" data-cate="all">综合</a> </li>\n <li><a href="javascript:void(0)" data-cate="zuozhe">作者</a> </li>\n <li><a href="javascript:void(0)" data-cate="title">标题</a> </li>\n <li><a href="javascript:void(0)" data-cate="shiju">诗句</a> </li>\n <li><a href="javascript:void(0)" data-cate="first">句首</a> </li>\n <li><a href="javascript:void(0)" data-cate="end">句尾</a> </li>\n <li><a href="javascript:void(0)" data-cate="book">古籍</a> </li>\n <li><a href="javascript:void(0)" data-cate="chengyu">成语</a> </li>\n </ul>\n </div>\n <div id="input_div">\n <input type="text" name="kw" id="search_input" placeholder="" autocomplete="off" value="">\n <a href="javascript:doSearch();"><img src="/public/image/web_search.png" class="www-icon"></a>\n <div style="clear: both"></div>\n </div>\n <div id="tip_result" class="card">\n <ul>\n </ul>\n <div id="tip_content">\n </div>\n </div>\n </div>\n <div style="clear: both;"></div>\n </div>\n</div>\n\n<div id="nav-top"><a href="/">主页</a><span class="nav-arrow"></span><a href="/book/index.html">史书典籍</a><span class="nav-arrow"></span><a href="/book/ershisishi.html">二十四史</a></div>\n\n<div id="main">\n <div id="main_left">\n <div class="card bookmark-list">\n <h1>《史记》</h1>\n <div>\n <img src="/public/image/book/shiji.jpg" class="book-img">\n <p>年代：西汉</p>\n <p>作者：司马迁</p>\n <p class="des"> 《史记》：中国历史上第一部纪传体通史。最初称为《太史公》，或《太史公记》、《太史记》。作者是西汉时期的司马迁。司马迁，字子长，父亲司马谈任太史令，写古今通史的愿望没有实现，临终要司马迁完成其夙愿。后来，司马迁继任父亲太史令之职，开始写《史记》，历十余年。因替李陵败降之事辩解而受宫刑，可司马迁仍旧坚持完成所著史籍。\r\n《史记》是中国古代第一部通史，不但规模巨大，体系完备，而且对此后的纪传体史书影响很深，历朝正史皆采用这种体裁撰写。同时，书中的文字生动性，叙事的形象性也是成就最高的。\r\n鲁迅先生在他的《汉文学史纲要》一书中称赞《史记》是“史家之绝唱，无韵之离骚”。</p>\n <div style="clear: both"></div>\n </div>\n <div>\n <form method="get" action="/book/chaxun/" style="float: right;">\n <input type="hidden" name="book_id" value="23">\n <input type="text" name="kw" placeholder="全文检索" autocomplete="off">\n </form>\n <div style="clear: both"></div>\n </div>\n <div class="line"></div>\n <div class="book-mulu">\n <ul>\n <li><h5>十二本纪</h5></li><li><a href="/book/shiji/1.html">五帝本纪</a></li><li><a href="/book/shiji/2.html">夏本纪</a></li><li><a href="/book/shiji/3.html">殷本纪</a></li><li><a href="/book/shiji/4.html">周本纪</a></li><li><a href="/book/shiji/5.html">秦本纪</a></li><li><a href="/book/shiji/6.html">秦始皇本纪</a></li><li><a href="/book/shiji/7.html">项羽本纪</a></li><li><a href="/book/shiji/8.html">高祖本纪</a></li><li><a href="/book/shiji/9.html">吕太后本纪</a></li><li><a href="/book/shiji/10.html">孝文本纪</a></li><li><a href="/book/shiji/11.html">孝景本纪</a></li><li><a href="/book/shiji/12.html">孝武本纪</a></li><li><h5>十表</h5></li><li><a href="/book/shiji/13.html">三代世表</a></li><li><a href="/book/shiji/14.html">十二诸侯年表</a></li><li><a href="/book/shiji/15.html">六国年表</a></li><li><a href="/book/shiji/16.html">秦楚之际月表</a></li><li><a href="/book/shiji/17.html">汉兴以来诸侯王年表</a></li><li><a href="/book/shiji/18.html">高祖功臣侯者年表</a></li><li><a href="/book/shiji/19.html">惠景间侯者年表</a></li><li><a href="/book/shiji/20.html">建元以来侯者年表</a></li><li><a href="/book/shiji/21.html">建元已来王子侯者年表</a></li><li><a href="/book/shiji/22.html">汉兴以来将相名臣年表</a></li><li><h5>八书</h5></li><li><a href="/book/shiji/23.html">礼书</a></li><li><a href="/book/shiji/24.html">乐书</a></li><li><a href="/book/shiji/25.html">律书</a></li><li><a href="/book/shiji/26.html">历书</a></li><li><a href="/book/shiji/27.html">天官书</a></li><li><a href="/book/shiji/28.html">封禅书</a></li><li><a href="/book/shiji/29.html">河渠书</a></li><li><a href="/book/shiji/30.html">平准书</a></li><li><h5>三十世家</h5></li><li><a href="/book/shiji/31.html">吴太伯世家</a></li><li><a href="/book/shiji/32.html">齐太公世家</a></li><li><a href="/book/shiji/33.html">鲁周公世家</a></li><li><a href="/book/shiji/34.html">燕召公世家</a></li><li><a href="/book/shiji/35.html">管蔡世家</a></li><li><a href="/book/shiji/36.html">陈杞世家</a></li><li><a href="/book/shiji/37.html">卫康叔世家</a></li><li><a href="/book/shiji/38.html">宋微子世家</a></li><li><a href="/book/shiji/39.html">晋世家</a></li><li><a href="/book/shiji/40.html">楚世家</a></li><li><a href="/book/shiji/41.html">越王句践世家</a></li><li><a href="/book/shiji/42.html">郑世家</a></li><li><a href="/book/shiji/43.html">赵世家</a></li><li><a href="/book/shiji/44.html">魏世家</a></li><li><a href="/book/shiji/45.html">韩世家</a></li><li><a href="/book/shiji/46.html">田敬仲完世家</a></li><li><a href="/book/shiji/47.html">孔子世家</a></li><li><a href="/book/shiji/48.html">陈涉世家</a></li><li><a href="/book/shiji/49.html">外戚世家</a></li><li><a href="/book/shiji/50.html">楚元王世家</a></li><li><a href="/book/shiji/51.html">荆燕世家</a></li><li><a href="/book/shiji/52.html">齐悼惠王世家</a></li><li><a href="/book/shiji/53.html">萧相国世家</a></li><li><a href="/book/shiji/54.html">曹相国世家</a></li><li><a href="/book/shiji/55.html">留侯世家</a></li><li><a href="/book/shiji/56.html">陈丞相世家</a></li><li><a href="/book/shiji/57.html">绛侯周勃世家</a></li><li><a href="/book/shiji/58.html">梁孝王世家</a></li><li><a href="/book/shiji/59.html">五宗世家</a></li><li><a href="/book/shiji/60.html">三王世家</a></li><li><h5>七十列传</h5></li><li><a href="/book/shiji/61.html">伯夷列传</a></li><li><a href="/book/shiji/62.html">管晏列传</a></li><li><a href="/book/shiji/63.html">老子韩非列传</a></li><li><a href="/book/shiji/64.html">司马穰苴列传</a></li><li><a href="/book/shiji/65.html">孙子吴起列传</a></li><li><a href="/book/shiji/66.html">伍子胥列传</a></li><li><a href="/book/shiji/67.html">仲尼弟子列传</a></li><li><a href="/book/shiji/68.html">商君列传</a></li><li><a href="/book/shiji/69.html">苏秦列传</a></li><li><a href="/book/shiji/70.html">张仪列传</a></li><li><a href="/book/shiji/71.html">樗里子甘茂列传</a></li><li><a href="/book/shiji/72.html">穰侯列传</a></li><li><a href="/book/shiji/73.html">白起王翦列传</a></li><li><a href="/book/shiji/74.html">孟子荀卿列传</a></li><li><a href="/book/shiji/75.html">孟尝君列传</a></li><li><a href="/book/shiji/76.html">平原君虞卿列传</a></li><li><a href="/book/shiji/77.html">魏公子列传</a></li><li><a href="/book/shiji/78.html">春申君列传</a></li><li><a href="/book/shiji/79.html">范睢蔡泽列传</a></li><li><a href="/book/shiji/80.html">乐毅列传</a></li><li><a href="/book/shiji/81.html">廉颇蔺相如列传</a></li><li><a href="/book/shiji/82.html">田单列传</a></li><li><a href="/book/shiji/83.html">鲁仲连邹阳列传</a></li><li><a href="/book/shiji/84.html">屈原贾生列传</a></li><li><a href="/book/shiji/85.html">吕不韦列传</a></li><li><a href="/book/shiji/86.html">刺客列传</a></li><li><a href="/book/shiji/87.html">李斯列传</a></li><li><a href="/book/shiji/88.html">蒙恬列传</a></li><li><a href="/book/shiji/89.html">张耳陈馀列传</a></li><li><a href="/book/shiji/90.html">魏豹彭越列传</a></li><li><a href="/book/shiji/91.html">黥布列传</a></li><li><a href="/book/shiji/92.html">淮阴侯列传</a></li><li><a href="/book/shiji/93.html">韩信卢绾列传</a></li><li><a href="/book/shiji/94.html">田儋列传</a></li><li><a href="/book/shiji/95.html">樊郦滕灌列传</a></li><li><a href="/book/shiji/96.html">张丞相列传</a></li><li><a href="/book/shiji/97.html">郦生陆贾列传</a></li><li><a href="/book/shiji/98.html">傅靳蒯成列传</a></li><li><a href="/book/shiji/99.html">刘敬叔孙通列传</a></li><li><a href="/book/shiji/100.html">季布栾布列传</a></li><li><a href="/book/shiji/101.html">袁盎晁错列传</a></li><li><a href="/book/shiji/102.html">张释之冯唐列传</a></li><li><a href="/book/shiji/103.html">万石张叔列传</a></li><li><a href="/book/shiji/104.html">田叔列传</a></li><li><a href="/book/shiji/105.html">扁鹊仓公列传</a></li><li><a href="/book/shiji/106.html">吴王濞列传</a></li><li><a href="/book/shiji/107.html">魏其武安侯列传</a></li><li><a href="/book/shiji/108.html">韩长孺列传</a></li><li><a href="/book/shiji/109.html">李将军列传</a></li><li><a href="/book/shiji/110.html">匈奴列传</a></li><li><a href="/book/shiji/111.html">卫将军骠骑列传</a></li><li><a href="/book/shiji/112.html">平津侯主父列传</a></li><li><a href="/book/shiji/113.html">南越列传</a></li><li><a href="/book/shiji/114.html">东越列传</a></li><li><a href="/book/shiji/115.html">朝鲜列传</a></li><li><a href="/book/shiji/116.html">西南夷列传</a></li><li><a href="/book/shiji/117.html">司马相如列传</a></li><li><a href="/book/shiji/118.html">淮南衡山列传</a></li><li><a href="/book/shiji/119.html">循吏列传</a></li><li><a href="/book/shiji/120.html">汲郑列传</a></li><li><a href="/book/shiji/121.html">儒林列传</a></li><li><a href="/book/shiji/122.html">酷吏列传</a></li><li><a href="/book/shiji/123.html">大宛列传</a></li><li><a href="/book/shiji/124.html">游侠列传</a></li><li><a href="/book/shiji/125.html">佞幸列传</a></li><li><a href="/book/shiji/126.html">滑稽列传</a></li><li><a href="/book/shiji/127.html">日者列传</a></li><li><a href="/book/shiji/128.html">龟策列传</a></li><li><a href="/book/shiji/129.html">货殖列传</a></li><li><a href="/book/shiji/130.html">太史公自序</a></li> </ul>\n </div>\n </div>\n </div>\n <div id="main_right">\n \n \t<div class="card right-book-card">\n\t\t<div class="aside_title">推荐阅读</div>\n\t\t\t<h6>四大名著</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/sanguoyanyi.html">《三国演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/shuihuzhuan.html">《水浒传》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/xiyouji.html">《西游记》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/hongloumeng.html">《红楼梦》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t\t<h6>二十四史</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/shiji.html">《史记》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/hanshu.html">《汉书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/houhanshu.html">《后汉书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/sanguozhi.html">《三国志》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/jinshu.html">《晋书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/songshu.html">《宋书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/nanqishu.html">《南齐书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/liangshu.html">《梁书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/chenshu.html">《陈书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/weishu.html">《魏书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/beiqishu.html">《北齐书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/zhoushu.html">《周书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/suishu.html">《隋书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/nanshi.html">《南史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/beishi.html">《北史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/jiutangshu.html">《旧唐书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/xintangshu.html">《新唐书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/jiuwudaishi.html">《旧五代史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/xinwudaishi.html">《新五代史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/songshi.html">《宋史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/liaoshi.html">《辽史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/jinshi.html">《金史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/yuanshi.html">《元史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/mingshi.html">《明史》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t\t<h6>四书</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/lunyu.html">《论语》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/mengzi.html">《孟子》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/daxue.html">《大学》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/zhongyong.html">《中庸》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t\t<h6>五经</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/shangshu.html">《尚书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/liji.html">《礼记》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/zhouyi.html">《周易》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/chunqiuzuozhuan.html">《春秋左氏传》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/shijing.html">《诗经》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t\t<h6>演义小说</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/sanguoyanyi.html">《三国演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/suitangyanyi.html">《隋唐演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/dongzhoulieguozhi.html">《东周列国志》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/yangjiajiang.html">《杨家将》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/qianhanyanyi.html">《前汉演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/houhanyanyi.html">《后汉演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/liangjinyanyi.html">《两晋演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/nanbeishiyanyi.html">《南北史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/cantangwudaishiyanyi.html">《残唐五代史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/suitangliangchaozhizhuan.html">《隋唐两朝志传》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/yuanshiyanyi.html">《元史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/mingshiyanyi.html">《明史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/qingshiyanyi.html">《清史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/minguoyanyi.html">《民国演义》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t</div>\n\n \n </div>\n <div style="clear: both;"></div>\n</div><a href="javascript:toTop();"><div id="toTop">顶部</div></a>\n<div id="footer">\n <div id="footer_main">\n <div id="bottom_nav">\n <a href="/category/story">诗人故事</a>\n <a href="/category/xiehouyu">歇后语</a>\n <a href="/category/miyu">谜语</a>\n <a href="/category/duilian">对联</a>\n Copyright © 2010-2020 <a href="https://beian.miit.gov.cn/#/Integrated/index" target="_blank">琼ICP备2021001997号-1</a> \n </div>\n </div>\n</div>\n\n<script>\nvar _hmt = _hmt || [];\n(function() {\n var hm = document.createElement("script");\n hm.src = "https://hm.baidu.com/hm.js?649f268280b553df1f778477ee743752";\n var s = document.getElementsByTagName("script")[0];\n s.parentNode.insertBefore(hm, s);\n})();\n</script>\n\n</body>\n</html>'

复制代码

跳转到最佳答案楼层

wp231957 · 发表于 2021-6-20 07:07:14

这个网站还是太easy了根本就不设防

import requests
from lxml import etree
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.77 Safari/537.36",
}
loginurl='https://www.shicimingju.com/book/shiji.html'
r = requests.get(url=loginurl,headers=headers)
r.encoding="UTF-8"
tree=etree.HTML(r.text)
data=tree.xpath("//div[@id='main_left']/div/div[1]/p[3]")
print(data[0].text)
'''
PS E:\wp> py test6.py
《史记》：中国历史上第一部纪传体通史。最初称为《太史公》，或《太史公记》、《太史记》。作者是西汉时期的司马迁。司马迁，字子长，父亲司马谈任太史令，写古今通史的愿望没有实现，临终要
司马迁完成其夙愿。后来，司马迁继任父亲太史令之职，开始写《史记》，历十余年。因替李陵败降之事辩解而受宫刑，可司马迁仍旧坚持完成所著史籍。
《史记》是中国古代第一部通史，不但规模巨大，体系完备，而且对此后的纪传体史书影响很深，历朝正史皆采用这种体裁撰写。同时，书中的文字生动性，叙事的形象性也是成就最高的。
鲁迅先生在他的《汉文学史纲要》一书中称赞《史记》是“史家之绝唱，无韵之离骚”。
PS E:\wp>
'''

复制代码

nahongyan1997 · 发表于 2021-6-20 20:16:48

wp231957 发表于 2021-6-20 07:07
这个网站还是太easy了根本就不设防

大佬能解决设防的网站么，就是那种 <a> 标签不带链接全靠 js 控制的那种

wp231957 · 发表于 2021-6-20 20:18:31

nahongyan1997 发表于 2021-6-20 20:16
大佬能解决设防的网站么，就是那种标签不带链接全靠 js 控制的那种

不好说，得拿具体网站看看再说

nahongyan1997 · 发表于 2021-6-20 20:33:02

pornhub

wp231957 · 发表于 2021-6-20 21:06:26

nahongyan1997 发表于 2021-6-20 20:33
pornhub

想下载吗？？？

nahongyan1997 · 发表于 2021-6-20 21:15:01

本帖最后由 nahongyan1997 于 2021-6-20 21:16 编辑

wp231957 发表于 2021-6-20 21:06
想下载吗？？？

我开玩笑的

阳崽崽 · 发表于 2021-6-20 23:08:21

wp231957 发表于 2021-6-20 07:07
这个网站还是太easy了根本就不设防

这是什么原因导致的？

fc5igm · 发表于 2021-6-21 00:01:37

这个最佳答案由 fc5igm 给出，感谢 fc5igm 的回答。

单击隐藏图章

这网址没有任何问题啊
你一定没看小甲鱼的课后作业

'<!DOCTYPE html>\n<html lang="zh">\n<head>\n\t<script type="text/javascript" src="https://ip.ws.126.net/ipquery"></script>\n <script src="/newpage/js/all.js"></script>\n <meta charset="UTF-8">\n <title>《史记》全集在线阅读_史书典籍_诗词名句网</title>\n <meta name="viewport" content="width=device-width,user-scalable=no" />\n <meta name="description" content="" />\n <link rel="stylesheet" href="/public/css/www_v3.css">\n <link rel="stylesheet" href="/public/css/font_576092_m2icqcebp7evzpvi.css">\n <script src="/public/js/device.js"></script>\n <script src="/public/js/jq.js"></script>\n <script src="/public/js/www_v3.js"></script>\n</head>\n<body>\n<div id="top_bar">\n <div id="top_bar_main">\n <div id="top_left">\n <a href="/"><img src="/public/image/logo.png" id="logo"/></a>\n <div id="top_left_menu">\n <ul>\n <li><a href="/">首页</a></li>\n <li><a href="/shicimark">分类</a></li>\n <li><a href="/category/all">作者</a></li>\n <li><a href="/paiming">排行榜</a></li>\n <li><a href="/cate?cate_id=4">课本古诗</a></li>\n <li><a href="/cipai/index.html">词牌名</a></li>\n <li><a href="/cangtoushi/index.html">藏头诗</a></li>\n <li><a href="/hecheng/index.html">合称</a></li>\n <li><a href="/book/">古籍</a></li>\n\t\t <li><a href="/app/">安卓下载</a></li>\n </ul>\n </div>\n <div style="clear: both"></div>\n </div>\n <div id="top_right">\n <div id="top_right_nav">\n <ul>\n <li><a href="javascript:void(0)" data-cate="all">综合</a> </li>\n <li><a href="javascript:void(0)" data-cate="zuozhe">作者</a> </li>\n <li><a href="javascript:void(0)" data-cate="title">标题</a> </li>\n <li><a href="javascript:void(0)" data-cate="shiju">诗句</a> </li>\n <li><a href="javascript:void(0)" data-cate="first">句首</a> </li>\n <li><a href="javascript:void(0)" data-cate="end">句尾</a> </li>\n <li><a href="javascript:void(0)" data-cate="book">古籍</a> </li>\n <li><a href="javascript:void(0)" data-cate="chengyu">成语</a> </li>\n </ul>\n </div>\n <div id="input_div">\n <input type="text" name="kw" id="search_input" placeholder="" autocomplete="off" value="">\n <a href="javascript:doSearch();"><img src="/public/image/web_search.png" class="www-icon"></a>\n <div style="clear: both"></div>\n </div>\n <div id="tip_result" class="card">\n <ul>\n </ul>\n <div id="tip_content">\n </div>\n </div>\n </div>\n <div style="clear: both;"></div>\n </div>\n</div>\n\n<div id="nav-top"><a href="/">主页</a><span class="nav-arrow"></span><a href="/book/index.html">史书典籍</a><span class="nav-arrow"></span><a href="/book/ershisishi.html">二十四史</a></div>\n\n<div id="main">\n <div id="main_left">\n <div class="card bookmark-list">\n <h1>《史记》</h1>\n <div>\n <img src="/public/image/book/shiji.jpg" class="book-img">\n <p>年代：西汉</p>\n <p>作者：司马迁</p>\n <p class="des"> 《史记》：中国历史上第一部纪传体通史。最初称为《太史公》，或《太史公记》、《太史记》。作者是西汉时期的司马迁。司马迁，字子长，父亲司马谈任太史令，写古今通史的愿望没有实现，临终要司马迁完成其夙愿。后来，司马迁继任父亲太史令之职，开始写《史记》，历十余年。因替李陵败降之事辩解而受宫刑，可司马迁仍旧坚持完成所著史籍。\r\n《史记》是中国古代第一部通史，不但规模巨大，体系完备，而且对此后的纪传体史书影响很深，历朝正史皆采用这种体裁撰写。同时，书中的文字生动性，叙事的形象性也是成就最高的。\r\n鲁迅先生在他的《汉文学史纲要》一书中称赞《史记》是“史家之绝唱，无韵之离骚”。</p>\n <div style="clear: both"></div>\n </div>\n <div>\n <form method="get" action="/book/chaxun/" style="float: right;">\n <input type="hidden" name="book_id" value="23">\n <input type="text" name="kw" placeholder="全文检索" autocomplete="off">\n </form>\n <div style="clear: both"></div>\n </div>\n <div class="line"></div>\n <div class="book-mulu">\n <ul>\n <li><h5>十二本纪</h5></li><li><a href="/book/shiji/1.html">五帝本纪</a></li><li><a href="/book/shiji/2.html">夏本纪</a></li><li><a href="/book/shiji/3.html">殷本纪</a></li><li><a href="/book/shiji/4.html">周本纪</a></li><li><a href="/book/shiji/5.html">秦本纪</a></li><li><a href="/book/shiji/6.html">秦始皇本纪</a></li><li><a href="/book/shiji/7.html">项羽本纪</a></li><li><a href="/book/shiji/8.html">高祖本纪</a></li><li><a href="/book/shiji/9.html">吕太后本纪</a></li><li><a href="/book/shiji/10.html">孝文本纪</a></li><li><a href="/book/shiji/11.html">孝景本纪</a></li><li><a href="/book/shiji/12.html">孝武本纪</a></li><li><h5>十表</h5></li><li><a href="/book/shiji/13.html">三代世表</a></li><li><a href="/book/shiji/14.html">十二诸侯年表</a></li><li><a href="/book/shiji/15.html">六国年表</a></li><li><a href="/book/shiji/16.html">秦楚之际月表</a></li><li><a href="/book/shiji/17.html">汉兴以来诸侯王年表</a></li><li><a href="/book/shiji/18.html">高祖功臣侯者年表</a></li><li><a href="/book/shiji/19.html">惠景间侯者年表</a></li><li><a href="/book/shiji/20.html">建元以来侯者年表</a></li><li><a href="/book/shiji/21.html">建元已来王子侯者年表</a></li><li><a href="/book/shiji/22.html">汉兴以来将相名臣年表</a></li><li><h5>八书</h5></li><li><a href="/book/shiji/23.html">礼书</a></li><li><a href="/book/shiji/24.html">乐书</a></li><li><a href="/book/shiji/25.html">律书</a></li><li><a href="/book/shiji/26.html">历书</a></li><li><a href="/book/shiji/27.html">天官书</a></li><li><a href="/book/shiji/28.html">封禅书</a></li><li><a href="/book/shiji/29.html">河渠书</a></li><li><a href="/book/shiji/30.html">平准书</a></li><li><h5>三十世家</h5></li><li><a href="/book/shiji/31.html">吴太伯世家</a></li><li><a href="/book/shiji/32.html">齐太公世家</a></li><li><a href="/book/shiji/33.html">鲁周公世家</a></li><li><a href="/book/shiji/34.html">燕召公世家</a></li><li><a href="/book/shiji/35.html">管蔡世家</a></li><li><a href="/book/shiji/36.html">陈杞世家</a></li><li><a href="/book/shiji/37.html">卫康叔世家</a></li><li><a href="/book/shiji/38.html">宋微子世家</a></li><li><a href="/book/shiji/39.html">晋世家</a></li><li><a href="/book/shiji/40.html">楚世家</a></li><li><a href="/book/shiji/41.html">越王句践世家</a></li><li><a href="/book/shiji/42.html">郑世家</a></li><li><a href="/book/shiji/43.html">赵世家</a></li><li><a href="/book/shiji/44.html">魏世家</a></li><li><a href="/book/shiji/45.html">韩世家</a></li><li><a href="/book/shiji/46.html">田敬仲完世家</a></li><li><a href="/book/shiji/47.html">孔子世家</a></li><li><a href="/book/shiji/48.html">陈涉世家</a></li><li><a href="/book/shiji/49.html">外戚世家</a></li><li><a href="/book/shiji/50.html">楚元王世家</a></li><li><a href="/book/shiji/51.html">荆燕世家</a></li><li><a href="/book/shiji/52.html">齐悼惠王世家</a></li><li><a href="/book/shiji/53.html">萧相国世家</a></li><li><a href="/book/shiji/54.html">曹相国世家</a></li><li><a href="/book/shiji/55.html">留侯世家</a></li><li><a href="/book/shiji/56.html">陈丞相世家</a></li><li><a href="/book/shiji/57.html">绛侯周勃世家</a></li><li><a href="/book/shiji/58.html">梁孝王世家</a></li><li><a href="/book/shiji/59.html">五宗世家</a></li><li><a href="/book/shiji/60.html">三王世家</a></li><li><h5>七十列传</h5></li><li><a href="/book/shiji/61.html">伯夷列传</a></li><li><a href="/book/shiji/62.html">管晏列传</a></li><li><a href="/book/shiji/63.html">老子韩非列传</a></li><li><a href="/book/shiji/64.html">司马穰苴列传</a></li><li><a href="/book/shiji/65.html">孙子吴起列传</a></li><li><a href="/book/shiji/66.html">伍子胥列传</a></li><li><a href="/book/shiji/67.html">仲尼弟子列传</a></li><li><a href="/book/shiji/68.html">商君列传</a></li><li><a href="/book/shiji/69.html">苏秦列传</a></li><li><a href="/book/shiji/70.html">张仪列传</a></li><li><a href="/book/shiji/71.html">樗里子甘茂列传</a></li><li><a href="/book/shiji/72.html">穰侯列传</a></li><li><a href="/book/shiji/73.html">白起王翦列传</a></li><li><a href="/book/shiji/74.html">孟子荀卿列传</a></li><li><a href="/book/shiji/75.html">孟尝君列传</a></li><li><a href="/book/shiji/76.html">平原君虞卿列传</a></li><li><a href="/book/shiji/77.html">魏公子列传</a></li><li><a href="/book/shiji/78.html">春申君列传</a></li><li><a href="/book/shiji/79.html">范睢蔡泽列传</a></li><li><a href="/book/shiji/80.html">乐毅列传</a></li><li><a href="/book/shiji/81.html">廉颇蔺相如列传</a></li><li><a href="/book/shiji/82.html">田单列传</a></li><li><a href="/book/shiji/83.html">鲁仲连邹阳列传</a></li><li><a href="/book/shiji/84.html">屈原贾生列传</a></li><li><a href="/book/shiji/85.html">吕不韦列传</a></li><li><a href="/book/shiji/86.html">刺客列传</a></li><li><a href="/book/shiji/87.html">李斯列传</a></li><li><a href="/book/shiji/88.html">蒙恬列传</a></li><li><a href="/book/shiji/89.html">张耳陈馀列传</a></li><li><a href="/book/shiji/90.html">魏豹彭越列传</a></li><li><a href="/book/shiji/91.html">黥布列传</a></li><li><a href="/book/shiji/92.html">淮阴侯列传</a></li><li><a href="/book/shiji/93.html">韩信卢绾列传</a></li><li><a href="/book/shiji/94.html">田儋列传</a></li><li><a href="/book/shiji/95.html">樊郦滕灌列传</a></li><li><a href="/book/shiji/96.html">张丞相列传</a></li><li><a href="/book/shiji/97.html">郦生陆贾列传</a></li><li><a href="/book/shiji/98.html">傅靳蒯成列传</a></li><li><a href="/book/shiji/99.html">刘敬叔孙通列传</a></li><li><a href="/book/shiji/100.html">季布栾布列传</a></li><li><a href="/book/shiji/101.html">袁盎晁错列传</a></li><li><a href="/book/shiji/102.html">张释之冯唐列传</a></li><li><a href="/book/shiji/103.html">万石张叔列传</a></li><li><a href="/book/shiji/104.html">田叔列传</a></li><li><a href="/book/shiji/105.html">扁鹊仓公列传</a></li><li><a href="/book/shiji/106.html">吴王濞列传</a></li><li><a href="/book/shiji/107.html">魏其武安侯列传</a></li><li><a href="/book/shiji/108.html">韩长孺列传</a></li><li><a href="/book/shiji/109.html">李将军列传</a></li><li><a href="/book/shiji/110.html">匈奴列传</a></li><li><a href="/book/shiji/111.html">卫将军骠骑列传</a></li><li><a href="/book/shiji/112.html">平津侯主父列传</a></li><li><a href="/book/shiji/113.html">南越列传</a></li><li><a href="/book/shiji/114.html">东越列传</a></li><li><a href="/book/shiji/115.html">朝鲜列传</a></li><li><a href="/book/shiji/116.html">西南夷列传</a></li><li><a href="/book/shiji/117.html">司马相如列传</a></li><li><a href="/book/shiji/118.html">淮南衡山列传</a></li><li><a href="/book/shiji/119.html">循吏列传</a></li><li><a href="/book/shiji/120.html">汲郑列传</a></li><li><a href="/book/shiji/121.html">儒林列传</a></li><li><a href="/book/shiji/122.html">酷吏列传</a></li><li><a href="/book/shiji/123.html">大宛列传</a></li><li><a href="/book/shiji/124.html">游侠列传</a></li><li><a href="/book/shiji/125.html">佞幸列传</a></li><li><a href="/book/shiji/126.html">滑稽列传</a></li><li><a href="/book/shiji/127.html">日者列传</a></li><li><a href="/book/shiji/128.html">龟策列传</a></li><li><a href="/book/shiji/129.html">货殖列传</a></li><li><a href="/book/shiji/130.html">太史公自序</a></li> </ul>\n </div>\n </div>\n </div>\n <div id="main_right">\n \n \t<div class="card right-book-card">\n\t\t<div class="aside_title">推荐阅读</div>\n\t\t\t<h6>四大名著</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/sanguoyanyi.html">《三国演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/shuihuzhuan.html">《水浒传》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/xiyouji.html">《西游记》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/hongloumeng.html">《红楼梦》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t\t<h6>二十四史</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/shiji.html">《史记》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/hanshu.html">《汉书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/houhanshu.html">《后汉书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/sanguozhi.html">《三国志》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/jinshu.html">《晋书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/songshu.html">《宋书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/nanqishu.html">《南齐书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/liangshu.html">《梁书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/chenshu.html">《陈书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/weishu.html">《魏书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/beiqishu.html">《北齐书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/zhoushu.html">《周书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/suishu.html">《隋书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/nanshi.html">《南史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/beishi.html">《北史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/jiutangshu.html">《旧唐书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/xintangshu.html">《新唐书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/jiuwudaishi.html">《旧五代史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/xinwudaishi.html">《新五代史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/songshi.html">《宋史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/liaoshi.html">《辽史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/jinshi.html">《金史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/yuanshi.html">《元史》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/mingshi.html">《明史》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t\t<h6>四书</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/lunyu.html">《论语》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/mengzi.html">《孟子》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/daxue.html">《大学》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/zhongyong.html">《中庸》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t\t<h6>五经</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/shangshu.html">《尚书》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/liji.html">《礼记》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/zhouyi.html">《周易》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/chunqiuzuozhuan.html">《春秋左氏传》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/shijing.html">《诗经》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t\t<h6>演义小说</h6>\n\t\t\t<div>\n\t\t\t<ul>\n\t\t\t\t\t\t\t\t<li><a href="/book/sanguoyanyi.html">《三国演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/suitangyanyi.html">《隋唐演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/dongzhoulieguozhi.html">《东周列国志》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/yangjiajiang.html">《杨家将》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/qianhanyanyi.html">《前汉演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/houhanyanyi.html">《后汉演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/liangjinyanyi.html">《两晋演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/nanbeishiyanyi.html">《南北史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/cantangwudaishiyanyi.html">《残唐五代史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/suitangliangchaozhizhuan.html">《隋唐两朝志传》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/yuanshiyanyi.html">《元史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/mingshiyanyi.html">《明史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/qingshiyanyi.html">《清史演义》</a></li>\n\t\t\t\t\t\t\t\t<li><a href="/book/minguoyanyi.html">《民国演义》</a></li>\n\t\t\t\t\t\t\t\t<div style="clear: both"></div>\n\t\t\t</ul>\n\t\t</div>\n\t\t\t</div>\n\n \n </div>\n <div style="clear: both;"></div>\n</div><a href="javascript:toTop();"><div id="toTop">顶部</div></a>\n<div id="footer">\n <div id="footer_main">\n <div id="bottom_nav">\n <a href="/category/story">诗人故事</a>\n <a href="/category/xiehouyu">歇后语</a>\n <a href="/category/miyu">谜语</a>\n <a href="/category/duilian">对联</a>\n Copyright © 2010-2020 <a href="https://beian.miit.gov.cn/#/Integrated/index" target="_blank">琼ICP备2021001997号-1</a> \n </div>\n </div>\n</div>\n\n<script>\nvar _hmt = _hmt || [];\n(function() {\n var hm = document.createElement("script");\n hm.src = "https://hm.baidu.com/hm.js?649f268280b553df1f778477ee743752";\n var s = document.getElementsByTagName("script")[0];\n s.parentNode.insertBefore(hm, s);\n})();\n</script>\n\n</body>\n</html>'

复制代码

fc5igm · 发表于 2021-6-21 00:02:38

本帖最后由 fc5igm 于 2021-6-21 00:16 编辑

import urllib.request as urr
import chardet as chd
def get_code(url,headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36 Edg/91.0.864.48'}):
request=urr.Request(url=url,headers=headers)
response=urr.urlopen(request).read()
coding=chd.detect(response)['encoding']
if coding=='GB2312':
coding='GBK'
return coding

复制代码

你可以拿这个函数来获取网页编码

这样更好看一些
需要安装bs4和lxml

import urllib.request as ulr
from bs4 import BeautifulSoup as BS
import chardet as chd
def get_code(url,headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36 Edg/91.0.864.48'}):
request=urr.Request(url=url,headers=headers)
response=urr.urlopen(request).read()
coding=chd.detect(response)['encoding']
if coding=='GB2312':
coding='GBK'
return coding
header={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.101 Safari/537.36 Edg/91.0.864.48'}
url='https://www.shicimingju.com/book/shiji.html'
opener=ulr.build_opener()
request=ulr.Request(url,headers=header)
response=opener.open(request).read().decode(get_code(url),'ignore')
soup=BS(response,'lxml')
print(soup.prettify())

复制代码

wp231957 · 发表于 2021-6-21 06:22:57

阳崽崽发表于 2021-6-20 23:08
这是什么原因导致的？

但凡有乱码出现，都是编码格式不一致所导致

账号		自动登录	找回密码
密码			立即注册

[已解决]抓取下来的网页内容为乱码（见正文），怎么转换成正常的可读模式？

马上注册，结交更多好友，享用更多功能^_^

浏览过的版块