有没有什么办法可以使用Wikipedia2Vec从文本中提取所有维基百科实体?或者还有其他方法可以做同样的事情。
例:
Text : "Scarlett Johansson is an American actress."
Entities : [ 'Scarlett Johansson' , 'American' ]
我想用Python做
谢谢
你可以使用spacy:
import spacy
import en_core_web_sm
nlp = en_core_web_sm.load()
doc = doc = nlp('Scarlett Johansson is an American actress.')
print([(X.text, X.label_) for X in doc.ents])
你得到:
[('Scarlett Johansson', 'PERSON'), ('American', 'NORP')]
在spacy documentation找到更多。
这是一个NLTK版本(可能不如SpaCy):
from nltk import Tree
from nltk import ne_chunk, pos_tag, word_tokenize
def get_continuous_chunks(text, chunk_func=ne_chunk):
chunked = chunk_func(pos_tag(word_tokenize(text)))
continuous_chunk = []
current_chunk = []
for subtree in chunked:
if type(subtree) == Tree:
current_chunk.append(" ".join([token for token, pos in subtree.leaves()]))
elif current_chunk:
named_entity = " ".join(current_chunk)
if named_entity not in continuous_chunk:
continuous_chunk.append(named_entity)
current_chunk = []
else:
continue
return continuous_chunk
text = 'Scarlett Johansson is an American actress.'
get_continuous_chunks(text)