我可以使用神经网络来完成这个分类任务吗？

Question

我有一些时间序列（以 0.5s 和 3kHz 采样，这意味着每个时间序列都有 1,500 个点），想要通过神经网络进行二元分类。序列被视为照片photo1，photo3。你可以很容易地发现照片1中的曲线比照片3中的曲线变化更剧烈。所以我想设计一个神经网络来做到这一点。

我尝试的：

原始数据看起来有点“嘈杂”，所以我首先对序列应用低通滤波器，使它们“清晰”。

b, a = signal.butter(8,80,'lowpass',fs=3000)

，急剧变化，温和变化（蓝色：原始数据，橙色：过滤后接下来我发现只有大约10个序列具有剧烈变化，而300个序列具有平缓变化。所以我使用SMOTE方法对少数序列进行过采样。然后我将尖锐的标记为“1”，温和的标记为“0”，并设计一个 LSTM 网络来处理数据集。 我分享我的数据集（带有“之前”的csv文件是原始数据，带有“之后”的数据是过滤后的数据，也是我的训练集）和云中的colab笔记本。

我的期望：

我想要的只是高准确率，但我不知道下一步该做什么。如果您能帮助我，我将不胜感激！

Tips：如果只有LSTM不能得到好的结果，也许CNN-LSTM会做得更好？也许我可以将时间序列转换为灰度图像来进行 CNN-LSTM（我从 matlab 示例得到这个灵感）？

Answer 1

我下面的尝试使用了卷积神经网络，包含大约 1400 个参数。经过 17 个 epoch 训练后，我获得了 98-100% 的验证 AUC 和准确率。

Split sizes are: 464 train, 193 val, 117 test
Model size is 1405 parameters

[epoch   1] trn loss: 0.5230 (auc 0.928, acc  88.58%) | val_loss: 0.5328 (auc 0.923, acc  85.49%)
...
[epoch  16] trn loss: 0.0268 (auc 1.000, acc 100.00%) | val_loss: 0.0859 (auc 0.990, acc  98.96%)
[epoch  17] trn loss: 0.0321 (auc 1.000, acc  99.35%) | val_loss: 0.1146 (auc 0.989, acc  98.45%)

总体架构如下：

• 序列很长，因此使用堆叠扩张卷积可以在输入端有效地实现宽感受野
• 然后使用卷积层和 maxpooling 对序列进行下采样
• 最后，全局 maxpool 接管生成的压缩序列（约 350 个步骤），并且密集层渲染输出 logit。

训练验证比例为 60%/25%。我在输入处使用了batchnorm来帮助初始缩放（尽管原始数据已经处于合理的范围内），并且使用tanh激活，因为它们与输入范围相似。批量大小为 8 效果很好，权重衰减的

Adam

也是如此。

原始数据也可能适用于非神经网络方法，因为类别看起来相当遥远（我认为部分原因是 SMOTE）：

---

加载并可视化数据。

import pandas as pd
import numpy as np
from matplotlib import pyplot as plt

#Load data
features = pd.read_csv('dataBeforeFilter.csv', dtype=np.float32).values
labels = pd.read_csv('labelBeforeFilter.csv', dtype=int).values.ravel()

n_samples = len(features)
sequence_len = features.shape[1]
n_features = 1
output_size = 1

#Visualise (ideally do this after splitting off the train set)
f, axs = plt.subplots(nrows=2, figsize=(9, 6), sharex=True, sharey=True)
common_params = dict(linewidth=0.1, color='black', alpha=0.5)

ax = axs[0]
features0 = features[labels == 0]
ax.plot(features0.T, **common_params)
ax.plot(features0.mean(axis=0), color='tab:red', linewidth=2)
ax.set(ylabel='signal', title='y=0')
ax.spines[['top', 'right']].set_visible(False)

ax = axs[1]
features1 = features[labels == 1]
ax.plot(features1.T, **common_params);
ax.plot(features1.mean(axis=0), 'tab:red', linewidth=2)
ax.set(xlabel='sample index', ylabel='signal', title='y=1')
ax.spines[['right', 'top']].set_visible(False)

f.suptitle('Raw data')
[ax.set_xlim(0, 500) for ax in axs]

#Legend
ax.plot([], [], color='black', label='raw data')
ax.plot([], [], 'tab:red', label='averaged')
ax.legend(loc='upper right', fancybox=True, shadow=True)

拆分：

train_size = int(0.6 * n_samples)
val_size = int(0.25 * n_samples)
test_size = n_samples - (train_size + val_size)

print('Split sizes are:', train_size, 'train,', val_size, 'val,', test_size, 'test')

shuffle_ixs = np.random.permutation(n_samples)
train_ixs = shuffle_ixs[:train_size]
val_ixs = shuffle_ixs[train_size:train_size + val_size]
test_ixs = shuffle_ixs[-test_size:]

#Split X and reshape to (n_samples, n_features, sequence_len)
X_trn, X_val, X_test = [
    features[ixs].reshape(-1, n_features, sequence_len)
    for ixs in [train_ixs, val_ixs, test_ixs]
]
y_trn, y_val, y_test = [labels[ixs] for ixs in [train_ixs, val_ixs, test_ixs]]

#To tensors
import torch

X_trn_t, X_val_t, X_test_t = [
    torch.from_numpy(X).float() for X in [X_trn, X_val, X_test]
]

y_trn_t, y_val_t, y_test_t = [
    torch.from_numpy(y).float() for y in [y_trn, y_val, y_test]
]

定义模型：

from torch import nn

#X_trn.shape: (464, 1, 1500)

#Lambda layer useful for simple manipulations
class LambdaLayer(nn.Module):
    def __init__(self, func):
        super().__init__()
        self.func = func

    def forward(self, x):
        return self.func(x)

#Define the model
def create_model():
    torch.manual_seed(0)
    activation_class = nn.Tanh
    
    model = nn.Sequential(
        #> (batch, features, seq len)

        nn.BatchNorm1d(n_features),
        nn.Conv1d(n_features, 2, 2),
        activation_class(),
    
        nn.Conv1d(2, 4, 2, dilation=2),
        activation_class(),
        nn.BatchNorm1d(4),
    
        nn.Conv1d(4, 8, 2, dilation=4),
        activation_class(),
        nn.BatchNorm1d(8),
    
        nn.Conv1d(8, 8, 2, dilation=8),
        activation_class(),
        nn.BatchNorm1d(8),
    
        nn.Conv1d(8, 8, 2, dilation=16),
        activation_class(),
        nn.BatchNorm1d(8),
    
        nn.Conv1d(8, 8, 2, dilation=32),
        activation_class(),
        nn.BatchNorm1d(8),
    
        #Downsample: using conv, then maxpool
        nn.Conv1d(8, 8, 9, stride=2),
        activation_class(),
        nn.BatchNorm1d(8),
        
        #MaxPool downsample
        nn.MaxPool1d(2),
    
        #Final features gen
        nn.Conv1d(8, 8, 3),
        activation_class(),
        nn.BatchNorm1d(8),
        nn.MaxPool1d(2),
    
        #Output layer: global max pool, then dense
        LambdaLayer(lambda x: nn.MaxPool1d(x.shape[2])(x)),
        nn.Flatten(),
        nn.Linear(8, 1),
    )
    return model

model = create_model()
print(
    'Model size is',
    sum([p.numel() for p in model.parameters() if p.requires_grad]),
    'parameters'
)

model(torch.rand(8, 1, 1500)).shape

训练并查看结果：

model = create_model()

optimiser = torch.optim.AdamW(model.parameters(), weight_decay=0.01)
batch_size = 4

#Batchify training data
from torch.utils.data import TensorDataset, DataLoader
train_loader = DataLoader(TensorDataset(X_trn_t, y_trn_t), batch_size, shuffle=True)

#Metrics tracking
from torcheval.metrics.functional import binary_auroc, binary_accuracy
from collections import defaultdict
metrics_dict = defaultdict(list)

for epoch in range(n_epochs := 17):
    model.train()

    for X_minibatch, y_minibatch in train_loader:
        logits = model(X_minibatch).ravel()
        loss = nn.BCEWithLogitsLoss()(logits, y_minibatch)

        optimiser.zero_grad()
        loss.backward()
        optimiser.step()
    #/end of epoch

    time_to_print = (epoch == 0) or (epoch + 1) % 1 == 0
    if not time_to_print:
        continue

    #Get the trn and val metrics at this epoch
    model.eval()
    with torch.no_grad():
        trn_logits, val_logits = [model(X).ravel() for X in [X_trn_t, X_val_t]]

    logits_y_trn_val = [[trn_logits, y_trn_t], [val_logits, y_val_t]]
    trn_loss, val_loss = [
        nn.BCEWithLogitsLoss()(logits, labels)
        for logits, labels in logits_y_trn_val
    ]

    trn_auc, val_auc = [
        binary_auroc(logits, labels) for logits, labels in logits_y_trn_val
    ]

    trn_acc, val_acc = [
        binary_accuracy(logits, labels) for logits, labels in logits_y_trn_val
    ]

    #Print results
    print(
        f'[epoch {epoch + 1:>3d}]',
        f'trn loss: {trn_loss:>6.4f} (auc {trn_auc:>5.3f}, acc {trn_acc:>7.2%})',
        f'| val_loss: {val_loss:>6.4f} (auc {val_auc:5.3f}, acc {val_acc:>7.2%})',
    )
    
    #Record metrics in dict
    for metric_name, metric_value in [
        ['epoch', epoch + 1],
        ['trn_loss', trn_loss], ['val_loss', val_loss],
        ['trn_auc', trn_auc], ['val_auc', val_auc],
        ['trn_acc', trn_acc], ['val_acc', val_acc]
    ]:
        metrics_dict[metric_name].append(metric_value)

#Plot training curve and selected metric
metric = 'auc'

trn_met, val_met = [
    metrics_dict[name] for name in [f'trn_{metric}', f'val_{metric}']
]

f, ax = plt.subplots(figsize=(9, 4))

loss_clr, met_clr = 'darkgreen', 'crimson'
epochs = metrics_dict['epoch']

ax.plot(epochs, metrics_dict['trn_loss'], loss_clr, marker='o', lw=2.8, label='train loss')
ax.plot(epochs, metrics_dict['val_loss'], loss_clr, marker='o', lw=2.8, ls='--', label='val loss')
ax.set(xlabel='epoch', ylabel='BCE loss')
#Colour loss labels and ticks:
ax.tick_params(axis='y', labelcolor=loss_clr, color=loss_clr)
ax.yaxis.label.set_color(loss_clr)
ax.spines.left.set_color(loss_clr)

ax2 = ax.twinx()
ax2.plot(epochs, trn_met, 'crimson', marker='o', lw=1, label=f'train {metric}')
ax2.plot(epochs, val_met, 'crimson', marker='o', lw=1, ls='--', label=f'val {metric}')
ax2.set_ylabel(metric)
#Colour metric labels and ticks:
ax2.tick_params(axis='y', labelcolor=met_clr, color=met_clr)
ax2.yaxis.label.set_color(met_clr)
ax2.spines.right.set_color(met_clr)
ax2.spines.left.set_visible(False) #overlaps with first ax

[ax_i.yaxis.label.set_weight('bold') for ax_i in [ax, ax2]]
[ax_i.spines.top.set_visible(False) for ax_i in [ax, ax2]]
ax.set_xticks(epochs)
f.legend(ncols=2, fancybox=True, shadow=True, loc='upper center')