Python pandas：为什么 concat 不起作用？

Question

我有一个要连接的 pd.DataFrames

all_predictions

列表。当我收到错误时，我开始调查：

从列表中的两个 DataFrame 中，我获取了行和列的子集；并重置索引：

df1 = all_predictions[1]
df2 = all_predictions[2]
df1 = df1.head(3)
df2 = df2.head(3)
df1.reset_index(drop=True, inplace=True)
df2.reset_index(drop=True, inplace=True)
df1 = df1[["key_id", "prediction", "yrmon"]]
df2 = df2[["key_id", "prediction", "yrmon"]]

但我无法连接它们：

ValueError                                Traceback (most recent call last)
Cell In[78], line 1
----> 1 pd.concat([df1, df2], ignore_index=True)

File ~/projects/bcs-modeling/.venv/lib/python3.9/site-packages/pandas/core/reshape/concat.py:393, in concat(objs, axis, join, ignore_index, keys, levels, names, verify_integrity, sort, copy)
    378     copy = False
    380 op = _Concatenator(
    381     objs,
    382     axis=axis,
   (...)
    390     sort=sort,
    391 )
--> 393 return op.get_result()

File ~/projects/bcs-modeling/.venv/lib/python3.9/site-packages/pandas/core/reshape/concat.py:680, in _Concatenator.get_result(self)
    676             indexers[ax] = obj_labels.get_indexer(new_labels)
    678     mgrs_indexers.append((obj._mgr, indexers))
--> 680 new_data = concatenate_managers(
    681     mgrs_indexers, self.new_axes, concat_axis=self.bm_axis, copy=self.copy
    682 )
    683 if not self.copy and not using_copy_on_write():
    684     new_data._consolidate_inplace()

File ~/projects/bcs-modeling/.venv/lib/python3.9/site-packages/pandas/core/internals/concat.py:199, in concatenate_managers(mgrs_indexers, axes, concat_axis, copy)
...
   2116 if block_shape[0] == 0:
   2117     raise ValueError("Empty data passed with indices specified.")
-> 2118 raise ValueError(f"Shape of passed values is {passed}, indices imply {implied}")

ValueError: Shape of passed values is (3, 3), indices imply (6, 3)

然后我重新创建了 DataFrame（我从例如

df1.to_dict()

中获得了准确的值）：

d1 = pd.DataFrame({
    "key_id": ["99c5a5fef58b0e89d22f6e2d99a7cdf5", "b2074747c5bbcaddfc588339e75542f4", "bd7798b548f115440054473557ec90f7"],
    "prediction": [57.9340063165089, -82.29114989923285, -141.58455971583805],
    "yrmon": ["2021-09-30","2021-09-30","2021-09-30"],
})
d2 = pd.DataFrame({
    "key_id": ["0873065925bca4e1cd2b5ef42ca979fa", "8c55ac2774db7b6c20a4f1b0cf80b2b5", "cdaa56f8ff1863040a1593086c8e61c6"],
    "prediction": [-298.70691, -24907.70776706384, 1.2290192287788002],
    "yrmon": ["2021-09-30","2021-09-30","2021-09-30"],
})

连接进行得很顺利：

为什么我无法连接 df1 和 df2？怎么解决这个问题？

非常感谢！

---

编辑： @Corralien：

ValueError                                Traceback (most recent call last)
Cell In[84], line 3
      1 df1 = all_predictions[1].head(3).reset_index(drop=True)
      2 df2 = all_predictions[2].head(3).reset_index(drop=True)
----> 3 pd.concat([df1, df2], ignore_index=True)

File ~/projects/bcs-modeling/.venv/lib/python3.9/site-packages/pandas/core/reshape/concat.py:393, in concat(objs, axis, join, ignore_index, keys, levels, names, verify_integrity, sort, copy)
    378     copy = False
    380 op = _Concatenator(
    381     objs,
    382     axis=axis,
   (...)
    390     sort=sort,
    391 )
--> 393 return op.get_result()

File ~/projects/bcs-modeling/.venv/lib/python3.9/site-packages/pandas/core/reshape/concat.py:680, in _Concatenator.get_result(self)
    676             indexers[ax] = obj_labels.get_indexer(new_labels)
    678     mgrs_indexers.append((obj._mgr, indexers))
--> 680 new_data = concatenate_managers(
    681     mgrs_indexers, self.new_axes, concat_axis=self.bm_axis, copy=self.copy
    682 )
    683 if not self.copy and not using_copy_on_write():
    684     new_data._consolidate_inplace()
...
   2116 if block_shape[0] == 0:
   2117     raise ValueError("Empty data passed with indices specified.")
-> 2118 raise ValueError(f"Shape of passed values is {passed}, indices imply {implied}")

ValueError: Shape of passed values is (3, 195), indices imply (6, 195)

Answer 1

根据您的示例，该错误是不可重现的。但是，假设您正在使用

pandas==2.1.0

并基于此未决问题 (GH53640)，它似乎与 dtypes 不匹配有关。

所以，你可以试试这个：

out = pd.concat([df1, df2.astype(df1.dtypes)], ignore_index=True)