令牌生成器并打印它

Question

在分词器之后，我的字符串列表会试图获取单词的值及其编号的关联。 f.e：= 3我该怎么做？？ （蟒蛇）这是代码

sentences_train, sentences_test, y_train, y_test = train_test_split(X,y, test_size=0.2, random_state=42)


from keras.preprocessing.text import Tokenizer
tokenizer = Tokenizer(num_words=5000)
tokenizer.fit_on_texts(sentences_train)

X_train = tokenizer.texts_to_sequences(sentences_train)
X_test = tokenizer.texts_to_sequences(sentences_test)

vocab_size = len(tokenizer.word_index) + 1

Answer 1

尝试tokenizer.texts_to_sequences（['the']）