Pandas中的随机数据块

Question

我需要从我的数据框架df获取随机数据块。我尝试过使用df.sample(10)，但它只生成单个样本，而不是连续的块。有没有办法对随机块进行采样（例如，6个连续数据点的块）？

这是数据框的示例

Year_DoY_Hour
2015-11-20 12:00:00         NaN
2015-11-20 12:30:00         NaN
2015-11-20 13:00:00         NaN
2015-11-20 13:30:00         NaN
2015-11-20 14:00:00         NaN
2015-11-20 14:30:00         NaN
2015-11-20 15:00:00    0.083298
  ...
2016-04-30 13:00:00    0.055639
2016-04-30 13:30:00    0.030809
2016-04-30 14:00:00    0.079277
2016-04-30 14:30:00    0.040736
2016-04-30 15:00:00    0.066980
2016-04-30 15:30:00    0.076448
2016-04-30 16:00:00    0.066822
2016-04-30 16:30:00    0.073143
2016-04-30 17:00:00         NaN
2016-04-30 17:30:00         NaN
2016-04-30 18:00:00         NaN
2016-04-30 18:30:00         NaN
2016-04-30 19:00:00         NaN
2016-04-30 19:30:00         NaN

所以从df我需要创建3个随机选择的6行线块。

例：

块1

2016-04-30 15:00:00    0.066980
2016-04-30 15:30:00    0.076448
2016-04-30 16:00:00    0.066822
2016-04-30 16:30:00    0.073143
2016-04-30 17:00:00         NaN
2016-04-30 17:30:00         NaN

块2

2016-04-30 09:30:00    0.036728
2016-04-30 10:00:00    0.036108
2016-04-30 10:30:00    0.031045
2016-04-30 11:00:00    0.031762
2016-04-30 11:30:00    0.033714
2016-04-30 12:00:00    0.042499

块3

2015-11-20 04:30:00         NaN
2015-11-20 05:00:00         NaN
2015-11-20 05:30:00         NaN
2015-11-20 06:00:00         NaN
2015-11-20 06:30:00         NaN
2015-11-20 07:00:00         NaN

块应按随机顺序排列，但块内的数据必须按顺序排列。我没有找到任何功能或类似的东西来做到这一点。

Answer 1

您可以生成从0到数据帧长度的随机数，然后在该索引处对数据帧进行切片。

import pandas as pd
import numpy as np

# create a fake data frame
index = pd.DatetimeIndex(start='2015-11-20', end='2016-04-30', freq='30min')
df = pd.DataFrame(np.random.normal(loc=10, size=len(index)), index=index, columns=['vals'])

# set the block size and the number of samples
block_size = 6
num_samples = 3
samples = [df.iloc[x:x+block_size] for x in np.random.randint(len(df), size=num_samples)]

# check results
samples[0]
                          vals
2016-01-06 00:30:00  10.313824
2016-01-06 01:00:00   9.445082
2016-01-06 01:30:00  11.952581
2016-01-06 02:00:00   9.496415
2016-01-06 02:30:00  10.404322
2016-01-06 03:00:00   8.506910

samples[1]
                          vals
2015-12-23 02:00:00  10.472048
2015-12-23 02:30:00  10.276933
2015-12-23 03:00:00  10.013481
2015-12-23 03:30:00  11.293218
2015-12-23 04:00:00  10.258379
2015-12-23 04:30:00   9.543600

samples[2]
                          vals
2016-01-10 06:00:00  10.809594
2016-01-10 06:30:00   8.953594
2016-01-10 07:00:00  10.254928
2016-01-10 07:30:00   9.911142
2016-01-10 08:00:00  10.377016
2016-01-10 08:30:00  11.907871

Answer 2

如果没有找到连续的条目，则返回6个连续条目的块或更小的块：

df = pd.read_csv(data, sep='\s+', header=None, parse_dates=[[0,1]], index_col=0)

# define delta t
delta = pd.Timedelta('30min')

# sampling only 1 values
sample = df.sample(1)

# add 6 timesteps
istart = sample.index
iend = istart + 6*delta

# Loc it
df.loc[istart.values[0]:iend.values[0]]