我正在尝试将Docker上现有管道中的脚本绑定到我的snakemake管道中。我使用奇异性设置了docker管道,并且可以正常工作。例如,
singularity exec docker://mypipeline some_command.sh file.bam out_file.bam
当我在命令行上交互式运行时,效果很好。同样,当我将完全相同的命令合并到我的Snakefile中时,它也可以工作:
rule myrule:
input:
"file.bam"
output:
"out_file.bam"
shell:
"singularity exec docker://mypipeline some_command.sh {input} {output}"
但是,当我尝试按照本教程https://reproducibility.sschmeier.com/container/index.html#using-a-container-in-our-workflow将容器合并到我的工作流中时,如下所示
singularity: "docker://mypipeline"
rule myrule:
input:
"file.bam"
output:
"out_file.bam"
shell:
"some_command.sh {input} {output}"
然后运行snakemake -p --use-singularity --cores 1
,我得到以下输出
Building DAG of jobs...
Using shell: /bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job counts:
count jobs
1 myrule
1
[Sun May 17 15:28:11 2020]
rule myrule:
input: file.bam
output: out_file.bam
jobid: 0
some_command.sh file.bam out_file.bam
Activating singularity image myImage.simg
然后,我收到一份很长的报告,不确定该如何处理,然后出现此错误消息
Waiting at most 5 seconds for missing files.
MissingOutputException in line 3 of Snakefile:
Job completed successfully, but some output files are missing. Missing files after 5 seconds:
out_file.bam
This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2020-05-17T152810.484310.snakemake.log
我的问题:
singularity: "docker://...
是好的做法还是没关系?错误消息表明奇异性命令已成功执行,但snakemake没有看到输出文件。代码中显示的输出文件out_file.bam
是否与实际使用的文件相同,或者您删除了一些文件路径?我建议将--verbose
标志添加到snakemake并检查snakemake执行的实际奇点命令。