使用Naive Bayes训练模型预测“用户输入”评论

Question

我正在使用带有文本Yelp餐厅评论的数据集以及他们的“明星”评级。我的数据是df，看起来像这样：

Textual Review           Numeric rating
"super cool restaurant"  5
"horrible experience"    1

我已经构建了MultinomialNB模型，该模型预测“星形”（1代表负面，5代表正面;仅使用这两个类别）进行评审。

import pandas as pd
import numpy as np
from textblob import TextBlob
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import confusion_matrix, classification_report
from nltk.corpus import stopwords
import string
import numpy

df = pd.read_csv('YELP_rev.csv')
#subsetting only the reviews on the extreme sides of the rating
df_class = df[(df['Numeric rating'] ==1) | (df['Numeric rating'] == 5)]

X = df_class['Textual review']
y = df_class['Numeric rating']
vectorizer=CountVectorizer()
X = vectorizer.fit_transform(X)

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=101)

nb = MultinomialNB()
#fiting the model with X_train, y_train
nb.fit(X_train, y_train)
#doing preditions
pred = nb.predict(X_test)
print(confusion_matrix(y_test, pred))



precision    recall  f1-score   support

           1       0.43      0.33      0.38         9
           5       0.90      0.93      0.92        61

   micro avg       0.86      0.86      0.86        70
   macro avg       0.67      0.63      0.65        70
weighted avg       0.84      0.86      0.85        70

我想要做的是预测用户提供的餐厅评论的“星级”评级。以下是我的尝试：

test_review = input("Enter a review:")  

def input_process(text):
    nopunc = [char for char in text if char not in string.punctuation]
    nopunc = ''.join(nopunc)
    return [word for word in nopunc.split() if word.lower() not in stopwords.words('english')]

new_x=vectorizer.transform(input_process(test_review))
test_review_rate = nb.predict(new_x)
print(test_review_rate)

我不确定我得到的输出是否正确，因为我获得了一系列分数。有人可以帮我解释这些分数吗？我只是取平均值，这将是我评价的“明星”评级吗？

>>Enter a review:We had dinner here for my birthday in Stockholm. The restaurant was very popular, so I would advise you book in advance.Blahblah
#my output
>>[5 5 5 5 5 5 5 5 5 1 5 1 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
 5 5 5 5]

ps我意识到样本数据很差，我的模型偏向正面评级！先谢谢！

Answer 1

你需要将你的单词join重新组成一个字符串。现在你的input_process函数的输出是一个单词列表，所以你的模型将每个单词解释为一个单独的输入样本，这就是为什么你得到评论中每个单词的分数，而不是整个文本的一个分数。 。

您的代码中的一些更改：

def input_process(text):
    # Something you can try for removing punctuations
    translator = str.maketrans('', '', string.punctuation)
    nopunc = text.translate(translator)
    words = [word for word in nopunc.split() if word.lower() not in stopwords.words('english')]
    # Join the words back and return as a string
    return ' '.join(words)

# vectorizer.transform takes a list as input
# You will have to pass your single string input as a list
new_x=vectorizer.transform([input_process(test_review)])