谁有兴趣，把这个英语断句改吧改吧，看是否能完善一下呢?

blackantt

1. # -*- coding: utf-8 -*-
   
2. import re
   
3. alphabets= "([A-Za-z])"
   
4. prefixes = "(Mr|St|Mrs|Ms|Dr)[.]"
   
5. suffixes = "(Inc|Ltd|Jr|Sr|Co)"
   
6. starters = "(Mr|Mrs|Ms|Dr|Prof|Capt|Cpt|Lt|He\s|She\s|It\s|They\s|Their\s|Our\s|We\s|But\s|However\s|That\s|This\s|Wherever)"
   
7. acronyms = "([A-Z][.][A-Z][.](?:[A-Z][.])?)"
   
8. websites = "[.](com|net|org|io|gov|edu|me)"
   
9. digits = "([0-9])"
   
10. multiple_dots = r'\.{2,}'
    
11. 
    
12. def split_into_sentences(text: str) -> list[str]:
    
13.     """
    
14.     Split the text into sentences.
    
15. 
    
16.     If the text contains substrings "<prd>" or "<stop>", they would lead
    
17.     to incorrect splitting because they are used as markers for splitting.
    
18. 
    
19.     :param text: text to be split into sentences
    
20.     :type text: str
    
21. 
    
22.     :return: list of sentences
    
23.     :rtype: list[str]
    
24.     """
    
25.     text = " " + text + "  "
    
26.     text = text.replace("\n"," ")
    
27.     text = re.sub(prefixes,"\\1<prd>",text)
    
28.     text = re.sub(websites,"<prd>\\1",text)
    
29.     text = re.sub(digits + "[.]" + digits,"\\1<prd>\\2",text)
    
30.     text = re.sub(multiple_dots, lambda match: "<prd>" * len(match.group(0)) + "<stop>", text)
    
31.     if "Ph.D" in text: text = text.replace("Ph.D.","Ph<prd>D<prd>")
    
32.     text = re.sub("\s" + alphabets + "[.] "," \\1<prd> ",text)
    
33.     text = re.sub(acronyms+" "+starters,"\\1<stop> \\2",text)
    
34.     text = re.sub(alphabets + "[.]" + alphabets + "[.]" + alphabets + "[.]","\\1<prd>\\2<prd>\\3<prd>",text)
    
35.     text = re.sub(alphabets + "[.]" + alphabets + "[.]","\\1<prd>\\2<prd>",text)
    
36.     text = re.sub(" "+suffixes+"[.] "+starters," \\1<stop> \\2",text)
    
37.     text = re.sub(" "+suffixes+"[.]"," \\1<prd>",text)
    
38.     text = re.sub(" " + alphabets + "[.]"," \\1<prd>",text)
    
39.     if "”" in text: text = text.replace(".”","”.")
    
40.     if """ in text: text = text.replace("."","".")
    
41.     if "!" in text: text = text.replace("!"",""!")
    
42.     if "?" in text: text = text.replace("?"",""?")
    
43.     text = text.replace(".",".<stop>")
    
44.     text = text.replace("?","?<stop>")
    
45.     text = text.replace("!","!<stop>")
    
46.     text = text.replace("<prd>",".")
    
47. 
    
48. 
    
49. 
    
50. 
    
51.     # text = re.sub(r'([,.?!][\'"&#187;’]*)(\w)', r'\1 \2', text)
    
52.     text = re.sub(r'([,][\'"&#187;’]*)(\w)', r'\1 \2', text)
    
53. 
    
54.     # --- 保留并优化原函数中一些有用的规则 ---
    
55. 
    
56.     # 规则2: 去除标点符号（.?!,;:)之前的多余空格
    
57.     # 例如："Hello ." -> "Hello."
    
58.     text = re.sub(r'\s+([.?!,;:])', r'\1', text)
    
59.     text = re.sub(r' +', ' ', text)
    
60. 
    
61. 
    
62. 
    
63. 
    
64.     sentences = text.split("<stop>")
    
65.     sentences = [s.strip() for s in sentences]
    
66.     if sentences and not sentences[-1]: sentences = sentences[:-1]
    
67.     return sentences
    
68. 
    
69. 
    
70. text = """
    
71. I love you.I am Tom.      Hello,world!How are you ?
    
72.     Dr. Smith and Mr. Jones met at 3:30 p.m. The meeting was held at example.com.
    
73.     "I love this project!" exclaimed Prof. Brown. "It's amazing."
    
74.     The U.S. Department of Energy (DOE) reported a 2.5% increase in 3.15 at Beijing.
    
75.     I love you. Tom said.
    
76.     I love you.Tom said.
    
77.     “苏超”联赛火爆出圈，场均观众8798人远超中甲，门票一票难求！全民参与+城市荣誉模式激活经济内循环，地域热梗成看点，文旅消费激增，政府主导打造“移动的城市广告”。
    
78.     网上热度也是相当狂飙。  虎扑App紧急新增“江苏联”频道，上线首日访问量破百万；抖音话题#江苏城市联赛#播放量破亿，素人拍摄的赛事短视频占比超70%……第三轮的门票已经是花钱都抢不到了，甚至有球迷在二手物品交易平台表示，愿意花100元求购一张门票。
    
79.     But then Wei poured a shower of gold coins into her lap."Never mind where I got them", he whispered. "Let's just say... I made a brilliant business deal!"Mei said nothing — she was too busy polishing the gold.Now news travels fast.Their neighbor,Jin ,soon, heard,    that Wei had returned from a big business trip and was now rich. His wife heard too?"Brilliant deal, eh?" she said to him. "If that fool Wei can make all that money,why can't you?"
    
80. 
    
81. 
    
82. 
    
83. """
    
84. 
    
85. print(split_into_sentences(text))

复制代码

某一个“天”

必须手搓吗？能用自然语言处理库吗

某一个“天”

手搓，我最近没时间

blackantt

某一个“天” 发表于 2025-6-12 13:05
必须手搓吗？能用自然语言处理库吗

NLTK ， spaCy ， blingfire 这些估计都够呛。  不但要考虑正常句子。还要考虑 I am Tom.I love you. It's 3.15.  这种缺少空格的情况。

某一个“天”

那你搞个这种特殊的字符串，我测试下