没有标点符号的句子分词

问题描述 投票:0回答:0

我想对没有标点符号的句子进行分词,代码如下:

import nltk

def segment_sentences(text):
    # Download the Punkt tokenizer if necessary
    nltk.download('punkt')
    
    # Tokenize the text into sentences
    sentences = nltk.sent_tokenize(text)
    
    return sentences

input_text = "hello how are you today i hope you're doing well have a great day"

sentences = segment_sentences(input_text)

# Print the segmented sentences
for sentence in sentences:
    print(sentence)

期望的输出

hello how are you today
i hope you're doing well
have a great day

但是电流输出

hello how are you today i hope you're doing well have a great day

我该如何解决?

python python-3.x nlp nltk spacy
© www.soinside.com 2019 - 2024. All rights reserved.