我想做一个嘈杂的解决方案,以便给定一个人称代词,该代词被前一个(最近的)人代替。
例如:
Alex is looking at buying a U.K. startup for $1 billion. He is very confident that this is going to happen. Sussan is also in the same situation. However, she has lost hope.
输出为:
Alex is looking at buying a U.K. startup for $1 billion. Alex is very confident that this is going to happen. Sussan is also in the same situation. However, Susan has lost hope.
另一个例子,
Peter is a friend of Gates. But Gates does not like him.
在这种情况下,输出将是:
Peter is a friend of Gates. But Gates does not like Gates.
是的!这太吵了。
使用spacy: 我已经使用 NER 提取了
Person
,但是如何正确替换代词?
代码:
import spacy
nlp = spacy.load("en_core_web_sm")
for ent in doc.ents:
if ent.label_ == 'PERSON':
print(ent.text, ent.label_)
有专门的 neuralcoref 库来解决共指问题。请参阅下面的最小可重现示例:
import spacy
import neuralcoref
nlp = spacy.load('en_core_web_sm')
neuralcoref.add_to_pipe(nlp)
doc = nlp(
'''Alex is looking at buying a U.K. startup for $1 billion.
He is very confident that this is going to happen.
Sussan is also in the same situation.
However, she has lost hope.
Peter is a friend of Gates. But Gates does not like him.
''')
print(doc._.coref_resolved)
Alex is looking at buying a U.K. startup for $1 billion.
Alex is very confident that this is going to happen.
Sussan is also in the same situation.
However, Sussan has lost hope.
Peter is a friend of Gates. But Gates does not like Peter.
注意,如果您使用 pip 安装它,您可能会遇到一些问题,因此最好从源代码构建它,正如我在此处概述的那样
考虑使用更大的模型,例如
neuralcoref
以获得更准确的标记。
en_core_web_lg
import spacy
from string import punctuation
nlp = spacy.load("en_core_web_lg")
def pronoun_coref(text):
doc = nlp(text)
pronouns = [(tok, tok.i) for tok in doc if (tok.tag_ == "PRP")]
names = [(ent.text, ent[0].i) for ent in doc.ents if ent.label_ == 'PERSON']
doc = [tok.text_with_ws for tok in doc]
for p in pronouns:
replace = max(filter(lambda x: x[1] < p[1], names),
key=lambda x: x[1], default=False)
if replace:
replace = replace[0]
if doc[p[1] - 1] in punctuation:
replace = ' ' + replace
if doc[p[1] + 1] not in punctuation:
replace = replace + ' '
doc[p[1]] = replace
doc = ''.join(doc)
return doc