从文本中提取维基百科实体

问题描述 投票:0回答:2

有没有什么办法可以使用Wikipedia2Vec从文本中提取所有维基百科实体?或者还有其他方法可以做同样的事情。

例:

Text : "Scarlett Johansson is an American actress."  
Entities : [ 'Scarlett Johansson' , 'American' ]

我想用Python做

谢谢

python nlp nltk wikipedia entity-linking
2个回答
1
投票

你可以使用spacy

import spacy
import en_core_web_sm
nlp = en_core_web_sm.load()
doc = doc = nlp('Scarlett Johansson is an American actress.')
print([(X.text, X.label_) for X in doc.ents])

你得到:

[('Scarlett Johansson', 'PERSON'), ('American', 'NORP')]

spacy documentation找到更多。


0
投票

这是一个NLTK版本(可能不如SpaCy):

from nltk import Tree
from nltk import ne_chunk, pos_tag, word_tokenize

def get_continuous_chunks(text, chunk_func=ne_chunk):
    chunked = chunk_func(pos_tag(word_tokenize(text)))
    continuous_chunk = []
    current_chunk = []

    for subtree in chunked:
        if type(subtree) == Tree:
            current_chunk.append(" ".join([token for token, pos in subtree.leaves()]))
        elif current_chunk:
            named_entity = " ".join(current_chunk)
            if named_entity not in continuous_chunk:
                continuous_chunk.append(named_entity)
                current_chunk = []
        else:
            continue

    return continuous_chunk


text = 'Scarlett Johansson is an American actress.'
get_continuous_chunks(text)
© www.soinside.com 2019 - 2024. All rights reserved.