在应该存在的索引上索引数据帧时出现KeyError

Question

这基本上只是我试图标准化我的数据，以便我的模型表现更好。它没有工作，但我正在尝试提高性能。它是 LSTM 中的一个时间序列，因此必须进行排序，它工作得很好，下面的第一段代码将我的数据根据年份分成训练测试部分。

test = {}
tracks = incident_data["geoid10"].unique().tolist()
train_x, train_y = creatsequence(
    incident_data[incident_data["geoid10"] == tracks[0]], 4
)
tracks.pop(0)
for track in tracks:
    test_data = incident_data[incident_data["geoid10"] == track]
    if test_data[test_data["Year"] < 2016].shape[0] > 4:
        trainx, trainy = creatsequence(test_data[test_data["Year"] < 2016], 4)
        train_x = np.concatenate((train_x, trainx))
        train_y = np.concatenate((train_y, trainy))
        if test_data[test_data["Year"] >= 2015].shape[0] > 4:
            test_x, test_y = creatsequence(test_data[test_data["Year"] >= 2015], 4)
            test[track] = {"X": test_x, "y": test_y}

def creatsequence(data, length):
    x = []
    y = []
    for column in ["All Other Thefts_y", "Simple Assault_y", "Theft From Motor Vehicle_y"]:
        data[column] = normalizeSeries(data[column], 4)
    for i in range(len(data) - length):
        x.append(data.drop(columns=['geoid10',
        'Year','Quarter'])[i:i+length])
        y.append(np.array(data[["All Other Thefts_y", "Simple Assault_y", "Theft From Motor Vehicle_y"]])[i+length])
        
    return(np.array(x), np.array(y))

def normalizeSeries(x, priorDays):

    new_x = []
    
    start = len(x) - 1
    while start > 0:
        print(x)
        if start - priorDays > 4:
            subarray = x[start - 2 * priorDays : start - priorDays]
        else:
            subarray = x[0:4]
            
        max_value = max([max(subarray),0.001])
        for i in range(4):
            value = x[start]
            new_x.append(value / max_value)
            start -= 1

    return new_x

错误：

KeyError: 11

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
/tmp/ipykernel_12/1047915744.py in <cell line: 11>()
     12     test_data = incident_data[incident_data["geoid10"] == track]
     13     if test_data[test_data["Year"] < 2016].shape[0] > 4:
---> 14         trainx, trainy = creatsequence(test_data[test_data["Year"] < 2016], 4)
     15         train_x = np.concatenate((train_x, trainx))
     16         train_y = np.concatenate((train_y, trainy))

/tmp/ipykernel_12/2018859376.py in creatsequence(data, length)
      8     y = []
      9     for column in ["All Other Thefts_y", "Simple Assault_y", "Theft From Motor Vehicle_y"]:
---> 10         data[column] = normalizeSeries(data[column], 4)
     11     for i in range(len(data) - length):
     12         x.append(data.drop(columns=['geoid10',

/tmp/ipykernel_12/2642226622.py in normalizeSeries(x, priorDays)
     17         max_value = max([max(subarray),0.001])
     18         for i in range(4):
---> 19             value = x[start]
     20             new_x.append(value / max_value)
     21             start -= 1

~/.cache/pypoetry/virtualenvs/python-kernel-OtKFaj5M-py3.9/lib/python3.9/site-packages/pandas/core/series.py in __getitem__(self, key)
    979 
    980         elif key_is_scalar:
--> 981             return self._get_value(key)
    982 
    983         if is_hashable(key):

~/.cache/pypoetry/virtualenvs/python-kernel-OtKFaj5M-py3.9/lib/python3.9/site-packages/pandas/core/series.py in _get_value(self, label, takeable)
   1087 
   1088         # Similar to Index.get_value, but we do not fall back to positional
-> 1089         loc = self.index.get_loc(label)
   1090         return self.index._get_values_for_loc(self, loc, label)
   1091 

~/.cache/pypoetry/virtualenvs/python-kernel-OtKFaj5M-py3.9/lib/python3.9/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   3802                 return self._engine.get_loc(casted_key)
   3803             except KeyError as err:
-> 3804                 raise KeyError(key) from err
   3805             except TypeError:
   3806                 # If we have a listlike key, _check_indexing_error will raise

我正在尝试对我的数据进行标准化排序，并将其放入火车测试拆分中。前几个表在第一个轨道上工作，但一旦到达第二个轨道，就会抛出索引错误。我检查了轨道，它有数据并且确实有那个长度。

Answer 1

您正在使用索引 11 对系列进行索引。 在这一行中：

---> 19             value = x[start]

我可以解释这一点的唯一方法是该索引超出了 pd.Series 的长度。

事实上，这在第一次迭代中有效，但在第二次迭代中无效，这告诉我您正在以有意义的方式覆盖/更改一些数据。而实际上循环外还有变量，例如：

train_x, train_y = creatsequence(
    incident_data[incident_data["geoid10"] == tracks[0]], 4
)

然后在循环中更改这些变量：

train_x = np.concatenate((train_x, trainx))
train_y = np.concatenate((train_y, trainy))

请注意，当您在循环外初始化这些变量时，它们可能不会复制

incident_data

，但您可能会返回一个切片。因此，覆盖 train_x 和 train_y 可能会改变 event_data，从而可能会破坏这个：

if test_data[test_data["Year"] < 2016].shape[0] > 4:
    trainx, trainy = creatsequence(test_data[test_data["Year"] < 2016], 4)

这当然只是基于我能读懂的程序逻辑，但是如果没有可运行的示例，很难调试。但我认为你的问题出在这些方面。