如何在Pandas和sklearn中把预测值合并回原始DataFrame?

问题描述 投票:0回答:3

第一次尝试使用sklearn与pandas,所以如果这可能是一个基本的问题,抱歉。这是我的代码。

import pandas as pd
from sklearn.linear_model import LogisticRegression

X = df[predictors]
y = df['Plc']

X_train = X[:int(X.shape[0]*0.7)]
X_test = X[int(X.shape[0]*0.7):]
y_train = y[:int(X.shape[0]*0.7)]
y_test = y[int(X.shape[0]*0.7):]


model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
result = model.score(X_test, y_test)
print("Accuracy: %.3f%%" % (result*100.0))

现在,我希望做的是把预测值放回原处 df 所以我可以看一下实际情况的差异。df['Plc'] 栏的预测值,以及 y_test.

我已经试过了,但感觉它的a)可能不是最好的方法,b)指数数字没有像预期的那样排列。

y_pred = pd.DataFrame()
y_pred['preds'] = model.predict(X_test)
y_test = pd.DataFrame(y_test)
y_test['index1'] = y_test.index
y_test = y_test.reset_index()
y_test = pd.concat([y_test,y_pred],axis=1)
y_test.set_index('index1')
df = df.reset_index()
df_out = pd.merge(df,y_test,how = 'inner',left_index = True, right_index = True)

有什么想法,我应该怎么做呢?谢谢

python pandas scikit-learn
3个回答
2
投票

你可以定义 preds 栏目 df "在飞行中",而不需要创建其他数据框。

import pandas as pd
import numpy as np
from sklearn.linear_model import LogisticRegression

# Generate fake data
df = pd.DataFrame(np.random.rand(1000, 4),
                  columns = list('abcd'))
df['Plc'] = np.random.randint(0,2,1000)

# Split X and y
predictors = list('abcd')
X = df[predictors]
y = df['Plc']

# Split train and test
train_size = int(X.shape[0]*0.7)
X_train = X[:train_size]
X_test = X[train_size:]
y_train = y[:train_size]
y_test = y[train_size:]

# Train the model
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)

# Predict train and test
y_pred_train = model.predict(X_train)
y_pred_test = model.predict(X_test)

现在你至少有两个选择:

  • 叠加预测和根据叠加的数组创建列。
df['preds'] = np.hstack([y_pred_train, y_pred_test])
  • 初始化列,然后分配预测。
df['preds'] = np.nan
df.loc[:train_size-1, 'pred'] = y_pred_train
df.loc[train_size:, 'pred'] = y_pred_test

它们产生同样的结果。


1
投票

成为你的 X_test 相当于 X_test = X[int(X.shape[0]*0.7):]你可以将预测结果添加到原始数据框架的较低的30%部分,也就是你的样本的最后30%。

Z=model.predict(X_test)
df.loc[int(X.shape[0]*0.7):,'predictions']=Z

这里,我们有一个新的列,叫做 "预测结果"。df. 一个例子,如果你的数据框架是。

df=pd.DataFrame({'predictor1':[0.1,0.3,0.3,0.3,0.5,0.9,0.02,0.8,0.8,0.75],
             'predictor2':[0.1,0.4,0.4,0.5,0.5,0.9,0.02,0.8,0.8,0.75],
        'Plc':np.array([0,1,1,1,1,1,1,0,1,1])})
predictor=['predictor1','predictor2']

它给你的结果。

   predictor1  predictor2  Plc  predictions
0        0.10        0.10    0          NaN
1        0.30        0.40    1          NaN
2        0.30        0.40    1          NaN
3        0.30        0.50    1          NaN
4        0.50        0.50    1          NaN
5        0.90        0.90    1          NaN
6        0.02        0.02    1          NaN
7        0.80        0.80    0          1.0
8        0.80        0.80    1          1.0
9        0.75        0.75    1          1.0

哪儿 Z=[1,1,1] 被添加到最后3个样本中。


1
投票

我相信你想要的是把X_test、y_test和y_pred合并到同一个数据框中(因为有X_train没有用)。我想用Pandas的training_test_split来保留指数是很容易的(虽然也有办法用numpy来保留指数)。Scikit-learn train_test_split with indices.). 我打算在这里用iris作为玩具数据,但你明白我的意思。

from sklearn.datasets import load_iris
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
X = pd.DataFrame(X)
y = pd.Series(y)
### you can use shuffle = False instead of random if it's needed
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.3,random_state=42)

model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
df = X_test.copy()
df['Plc']= y_test
df.reset_index(inplace=True)
df['pred'] = model.predict(X_test)

## then print df, you can remove the index of the original df if you like

如果你真的想合并X_train,y_train,并且在pred列中有NaN,你可以用同样的方式合并X_train和y_train,然后使用pd.concat来制作一个单一的数据框。

df2 = X_train.copy()
df2['Plc'] = y_train
df2.reset_index(inplace=True)
pd.concat([df,df2])
index   0   1   2   3   Plc pred
0   73  6.1 2.8 4.7 1.2 1   1.0
1   18  5.7 3.8 1.7 0.3 0   0.0
2   118 7.7 2.6 6.9 2.3 2   2.0
3   78  6.0 2.9 4.5 1.5 1   1.0
4   76  6.8 2.8 4.8 1.4 1   1.0
... ... ... ... ... ... ... ...
100 71  6.1 2.8 4.0 1.3 1   NaN
101 106 4.9 2.5 4.5 1.7 2   NaN
102 14  5.8 4.0 1.2 0.2 0   NaN
103 92  5.8 2.6 4.0 1.2 1   NaN
104 102 7.1 3.0 5.9 2.1 2   NaN
150 rows × 7 columns

© www.soinside.com 2019 - 2024. All rights reserved.