训练用于词嵌入的神经网络

Question

Attached是实体的链接文件。我想训练一个神经网络，将每个实体表示为一个向量。附件是我的训练代码

import pandas as pd
import numpy as np

from numpy import array
from keras.preprocessing.text import one_hot

from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.models import Model
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers import Input


from keras.layers.embeddings import Embedding
from sklearn.model_selection import train_test_split 

file_path = '/content/drive/My Drive/Colab Notebooks/Deep Learning/NLP/Data/entities.txt'
df = pd.read_csv(file_path, delimiter = '\t', engine='python', quoting = 3, header = None)
df.columns = ['Entity']
Entity = df['Entity']

X_train, X_test = train_test_split(Entity, test_size = 0.10)
print('Total Entities: {}'.format(len(Entity)))
print('Training Entities: {}'.format(len(X_train)))
print('Test Entities: {}'.format(len(X_test)))
vocab_size = len(Entity)
X_train_encode = [one_hot(d, vocab_size,lower=True, split=' ') for d in X_train]
X_test_encode = [one_hot(d, vocab_size,lower=True, split=' ') for d in X_test]
model = Sequential()
model.add(Embedding(input_length=1,input_dim=vocab_size, output_dim=100))
model.add(Flatten())
model.add(Dense(vocab_size, activation='softmax'))

model.compile(optimizer='adam', loss='mse', metrics=['acc'])
print(model.summary())

model.fit(X_train_encode, X_train_encode, epochs=20, batch_size=1000, verbose=1)

我尝试执行代码时遇到以下错误。

检查模型输入时发生错误：传递给模型的Numpy数组列表不是模型预期的大小。预计会看到1个阵列，但获得了以下34826个阵列的列表：

Answer 1

您正在传递numpy数组的model.fit列表。以下代码生成x_train_encode和X_test_encode的数组列表。

X_train_encode = [one_hot(d, vocab_size,lower=True, split=' ') for d in X_train]
X_test_encode = [one_hot(d, vocab_size,lower=True, split=' ') for d in X_test]

传递给model.fit方法时，将这些列表更改为numpy数组。

X_train_encode = np.array(X_train_encode)
X_test_encode = np.array(X_test_encode)

而且我不认为需要对X_train和X_test进行one_hot编码，嵌入层期望整数（在您的情况下为单词索引）不是单词索引的一个热编码值。因此，如果X_train和X_test是单词索引的数组，则可以直接将其输入到model.fit方法中。

编辑：

当前正在使用'mse'损失。由于最后一层是softmax层，因此交叉熵损失在此处更适用。并且输出是一类（单词）稀疏分类的整数值，应该用于损失。

model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['acc'])