Snakemake temp()

Question

我已将 Snakemake 连接到 S3 帐户，并且我想在处理管道后删除某些 temp() 文件。

我有一条规则，将某些文件指定为 temp()。以下是其中一个示例：

 #Split rep element mapped bam file into subfiles
rule split_rep_bam:
  input:
    'rep_element_pipeline/{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam'
  output:
    temp('rep_element_pipeline/AA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/AC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/AG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/AN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/AT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/CT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/GT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/NT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TA.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TC.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TG.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TN.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp'),
    temp('rep_element_pipeline/TT.{sample}.fastq.gz.mapped_vs_' + config["ref"]["bt2_index"] + '.sam.tmp')
  conda:
    '../envs/rep_element.yaml'
  params:
    fp=full_path
  shell:
    'perl ../scripts/split_bam_to_subfiles.pl {input[0]} '
    '{params.fp}/rep_element_pipeline/'

我一直在运行 Snakemake：

snakemake --default-remote-provider S3 --default-remote-prefix '$s3' --use-conda --cores 32 --rerun-incomplete --printshellcmds --delete-temp-output

我指定了

--delete-temp-output

选项。

运行此规则时，snakemake 似乎正在删除这些文件。但是，这些文件仍然保留在 S3 中。有谁知道为什么这些文件在 S3 中没有被删除？

Building DAG of jobs...
Deleting 20211222-rp030/rep_element_pipeline/AA.APP-01_rep_1_Ribo_R1.fastq.gz.mapped_vs_bt2_hg38.sam.tmp
Deleting 20211222-rp030/rep_element_pipeline/AC.APP-01_rep_1_Ribo_R1.fastq.gz.mapped_vs_bt2_hg38.sam.tmp

Answer 1

这个问题与另一个问题非常相似，核心问题是：

remote() 包装器与 temp() 和 protected() 包装器是互斥的。

来源：文档。

一种解决方案是使用

local

函数显式标记文件，因此它看起来像这样：

rule some_rule:
     output: temp(local('test.txt'))
     run: # ...