snakemake 总是在第 44 行报告“MissingOutputException，5 秒后丢失文件：

Question

在我的 RNAs-seq 管道中，我总是收到 snakemake 的相同错误报告：

MissingOutputException in line 44 of /root/s/r/snakemake/my_rnaseq_data/Snakefile:
Missing files after 5 seconds:
03_align/wt2.bam
This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.

这是我的 Snakefile：

SBT=["wt1","wt2","epcr1","epcr2"]

rule all:
    input:
        expand("02_clean/{nico}_1.paired.fq", nico=SBT),
        expand("02_clean/{nico}_2.paired.fq", nico=SBT),
        expand("03_align/{nico}.bam", nico=SBT)

rule trim:
    input:
        "01_raw/{nico}_1.fastq",
        "01_raw/{nico}_2.fastq"
    output:
        "02_clean/{nico}_1.paired.fq.gz",
        "02_clean/{nico}_1.unpaired.fq.gz",
        "02_clean/{nico}_2.paired.fq.gz",
        "02_clean/{nico}_2.unpaired.fq.gz",
    shell:
        "java -jar /software/Trimmomatic-0.36/trimmomatic-0.36.jar PE -threads 16 {input[0]} {input[1]} {output[0]} {output[1]} {output[2]} {output[3]} ILLUMINACLIP:/software/Trimmomatic-0.36/adapters/TruSeq3-PE-2.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 MINLEN:36 &"

rule gzip:
    input:
        "02_clean/{nico}_1.paired.fq.gz",
        "02_clean/{nico}_2.paired.fq.gz"
    output:
        "02_clean/{nico}_1.paired.fq",
        "02_clean/{nico}_2.paired.fq"
    run:
        shell("gzip -d {input[0]} > {output[0]}")
        shell("gzip -d {input[1]} > {output[1]}")

rule map:
    input:
        "02_clean/{nico}_1.paired.fq",
        "02_clean/{nico}_2.paired.fq"
    output:
        "03_align/{nico}.sam"
    log:
        "logs/map/{nico}.log"
    threads: 40
    shell:
        "hisat2 -p 20 --dta -x /root/s/r/p/A_th/WT-Al_VS_WT-CK/index/tair10 -1 {input[0]} -2 {input[1]} -S {output} >{log} 2>&1 &"

rule sort2bam:
    input:
        "03_align/{nico}.sam"
    output:
        "03_align/{nico}.bam"
    threads:30
    shell:
        "samtools sort -@ 20 -m 20G -o {output} {input} &"

一切正常，直到我添加 "rule sort2bam" 部分。

当我干燥时-运行，一切正常。但是当我执行它时，它会报告问题所描述的错误。令人惊讶的是，它运行它报告的任务卡在 background.But 它总是运行一个 task.like 这些：

rule sort2bam:
    input: 03_align/epcr1.sam
    output: 03_align/epcr1.bam
    jobid: 11
    wildcards: nico=epcr1

Waiting at most 5 seconds for missing files.
MissingOutputException in line 45 of /root/s/r/snakemake/my_rnaseq_data/Snakefile:
Missing files after 5 seconds:
03_align/epcr1.bam
This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

[Sat Apr 27 06:10:22 2019]
rule sort2bam:
    input: 03_align/wt1.sam
    output: 03_align/wt1.bam
    jobid: 9
    wildcards: nico=wt1

Waiting at most 5 seconds for missing files.
MissingOutputException in line 45 of /root/s/r/snakemake/my_rnaseq_data/Snakefile:
Missing files after 5 seconds:
03_align/wt1.bam
This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

[Sat Apr 27 06:23:13 2019]
rule sort2bam:
    input: 03_align/wt2.sam
    output: 03_align/wt2.bam
    jobid: 6
    wildcards: nico=wt2

Waiting at most 5 seconds for missing files.
MissingOutputException in line 44 of /root/s/r/snakemake/my_rnaseq_data/Snakefile:
Missing files after 5 seconds:
03_align/wt2.bam
This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

我不知道我的代码有什么问题？有什么理想吗？提前致谢！

Answer 1

我知道如何解决，但我不知道它为什么有效！只需删除

中的“&”

samtools sort -@ 20 -m 20G -o {output} {input} &

Answer 2

如您所想，& 是问题所在。控制运算符 & 使您的命令运行在子 shell 的后台运行，这导致 snakemake 认为工作已完成，而实际上并未完成。在您的情况下，似乎不需要使用它。

来自 man bash 关于 & 的用法（从 this answer 窃取）：

If a command is terminated by the control operator &, the shell executes the command in the background in a subshell. The shell does not wait for the command to finish, and the return status is 0.

snakemake 总是在第 44 行报告“MissingOutputException，5 秒后丢失文件：

snakemake always report " MissingOutputException in line 44, Missing files after 5 seconds:

pipeline

snakemake

rna-seq