snakemake

Question

我正在尝试创建一个管道，它将 config.yml 中的用户配置目录（他们从 BaseSpace 下载了 .fastq.gz 文件的项目目录）到运行序列文件上的 fastqc。我已经有了按车道合并 fastqs 的下游步骤，并在合并的文件上运行ning fastqc。

但是，通配符给我带来了问题运行在原始基础空间文件上使用 fastqc。以下是我尝试运行ning snakemake.

时的错误

Missing input files for rule all:
qc/fastqc_premerge/DEX-13_S9_L001_ngc1838-10_L001_ds.9fd1f6dff0df47ab821125aab07be69b_r1_fastqc.zip
qc/fastqc_premerge/BOMB-3-2-19D_S8_L002_ngc1838-8_L002_ds.b81c308d62ba447b8caf074ffb27917e_r1_fastqc.zip
qc/fastqc_premerge/DEX-13_S9_L002_ngc1838-10_L002_ds.6369bc71fac44f00931eecb9b0a45d59_r1_fastqc.zip

如有任何建议，我们将不胜感激。下面是重现此问题的最少代码。

import glob

configfile: "config.yaml"

wildcard_constraints:
   bsdir = '\w+_L\d+_ds.\w+',
   lanenum = '\d+'

inputdirectory=config["directory"]
DIRECTORY, SAMPLES, LANENUMS = glob_wildcards(inputdirectory+"/{bsdir}/{sample}_L{lanenum}_R1_001.fastq.gz")
DIRECTORY, SAMPLES, LANENUMS = glob_wildcards(inputdirectory+"/{bsdir}/{sample}_L{lanenum}_R2_001.fastq.gz")


##### target rules #####
rule all:
    input:
       #expand('qc/fastqc_premerge/{sample}_L{lanenum}_{bsdir}_r1_fastqc.zip', sample=SAMPLES, bsdir=DIRECTORY, lanenum=LANENUMS)
        expand('qc/fastqc_premerge/{sample}_L{lanenum}_{bsdir}_r1_fastqc.zip', zip, sample=SAMPLES, bsdir=DIRECTORY, lanenum=LANENUMS)  ##Changed to this from commenters suggestion, however, snakemake still wont run


rule fastqc_premerge_r1:
    input:
        f"{config['directory']}/{{bsdir}}/{{sample}}_L{{lanenum}}_R1_001.fastq.gz"
    output:
        html="qc/fastqc_premerge/{sample}_L{lanenum}_{bsdir}_r1.html",
        zip="qc/fastqc_premerge/{sample}_L{lanenum}_{bsdir}_r1_fastqc.zip" # the suffix _fastqc.zip is necessary for multiqc to find the file. If not using multiqc, you are free to choose an arbitrary filename
    params: ""
    log:
        "logs/fastqc_premerge/{sample}_L{lanenum}_{bsdir}_r1.log"
    threads: 1
    wrapper:
        "v0.69.0/bio/fastqc"

目录结构：

ngc1838-10_L001_ds.9fd1f6dff0df47ab821125aab07be69b/DEX-13_S9_L001_R1_001.fastq.gz
ngc1838-10_L001_ds.9fd1f6dff0df47ab821125aab07be69b/DEX-13_S9_L001_R2_001.fastq.gz
ngc1838-10_L002_ds.6369bc71fac44f00931eecb9b0a45d59/DEX-13_S9_L002_R1_001.fastq.gz
ngc1838-10_L002_ds.6369bc71fac44f00931eecb9b0a45d59/DEX-13_S9_L002_R2_001.fastq.gz
ngc1838-8_L002_ds.b81c308d62ba447b8caf074ffb27917e/BOMB-3-2-19D_S8_L002_R1_001.fastq.gz
ngc1838-8_L002_ds.b81c308d62ba447b8caf074ffb27917e/BOMB-3-2-19D_S8_L002_R2_001.fastq.gz

在上述情况下，我想对所有 6 个输入 R1/R2 文件进行运行 fastqc，然后在下游为 DEX_13_S9 创建一个合并文件（对于两个输入merge) 和 BOMB-3_2_19D （这将是 1 输入的副本）。然后在这些生成的 R1 和 R2 文件上创建 4 个 fastqc 报告。

编辑：我必须更改以下内容才能让 snakemake 变为运行

inputdirectory=config["directory"]
PROJECTDIR, RANDOMINT, LANENUM1, BSSTRINGS, SAMPLES, LANENUMS = glob_wildcards(inputdirectory+"/{proj}-{randint}_L{lanenum1}_ds.{bsstring}/{sample}_L{lanenum}_R1_001.fastq.gz", followlinks=True)
PROJECTDIR, RANDOMINT, LANENUM1, BSSTRINGS, SAMPLES, LANENUMS = glob_wildcards(inputdirectory+"/{proj}-{randint}_L{lanenum1}_ds.{bsstring}/{sample}_L{lanenum}_R2_001.fastq.gz", followlinks=True)


##### target rules #####
rule all:
    input:
       "qc/multiqc_report_premerge.html"




rule fastqc_premerge_r1:
    input:
        f"{config['directory']}/{{proj}}-{{randint}}_L{{lanenum1}}_ds.{{bsstring}}/{{sample}}_L{{lanenum}}_R1_001.fastq.gz"
    output:
        html="qc/fastqc_premerge/{sample}_L{lanenum}_{proj}-{randint}_L{lanenum1}_ds.{bsstring}_r1.html",
        zip="qc/fastqc_premerge/{sample}_L{lanenum}_{proj}-{randint}_L{lanenum1}_ds.{bsstring}_r1_fastqc.zip" # the suffix _fastqc.zip is necessary for multiqc
    params: ""
    log:
        "logs/fastqc_premerge/{sample}_L{lanenum}_{proj}-{randint}_L{lanenum1}_ds.{bsstring}_r1.log"
    threads: 1
    wrapper:
        "v0.69.0/bio/fastqc"

rule fastqc_premerge_r2:
    input:
        f"{config['directory']}/{{proj}}-{{randint}}_L{{lanenum1}}_ds.{{bsstring}}/{{sample}}_L{{lanenum}}_R2_001.fastq.gz"
    output:
        html="qc/fastqc_premerge/{sample}_L{lanenum}_{proj}-{randint}_L{lanenum1}_ds.{bsstring}_r2.html",
        zip="qc/fastqc_premerge/{sample}_L{lanenum}_{proj}-{randint}_L{lanenum1}_ds.{bsstring}_r2_fastqc.zip" # the suffix _fastqc.zip is necessary for multiqc
    params: ""
    log:
        "logs/fastqc_premerge/{sample}_L{lanenum}_{proj}-{randint}_L{lanenum1}_ds.{bsstring}_r2.log"
    threads: 1
    wrapper:
        "v0.69.0/bio/fastqc"

rule multiqc_pre:
    input:
        expand("qc/fastqc_premerge/{sample}_L{lanenum}_{proj}-{randint}_L{lanenum1}_ds.{bsstring}_r1_fastqc.zip", zip, sample=SAMPLES, lanenum=LANENUMS, proj=PROJECTDIR, randint=RANDOMINT, lanenum1=LANENUM1, bsstring=BSSTRINGS),
        expand("qc/fastqc_premerge/{sample}_L{lanenum}_{proj}-{randint}_L{lanenum1}_ds.{bsstring}_r2_fastqc.zip", zip, sample=SAMPLES, lanenum=LANENUMS, proj=PROJECTDIR, randint=RANDOMINT, lanenum1=LANENUM1, bsstring=BSSTRINGS)
    output:
        "qc/multiqc_report_premerge.html"
    log:
        "logs/multiqc_premerge.log"
    wrapper:
        "0.62.0/bio/multiqc"

Answer 1

在你的 rule all 你有：

expand('qc/fastqc_premerge/{sample}_L{lanenum}_{bsdir}_r1_fastqc.zip', sample=SAMPLES, bsdir=DIRECTORY, lanenum=LANENUMS)

这应该会生成 SAMPLES、DIRECTORY 和 LANENUMS 的所有组合。这是你想要的吗？我怀疑不是，因为这意味着所有样本都在所有目录中，并且它们在所有通道上。也许您希望 zip 函数扩展列表：

expand('qc/fastqc_premerge/{sample}_L{lanenum}_{bsdir}_r1_fastqc.zip', zip, sample=SAMPLES, bsdir=DIRECTORY, lanenum=LANENUMS)

Answer 2

它告诉你丢失了什么文件；这就是“缺少所有规则的输入文件”下的行。

话虽这么说，但要回答您最初的问题，如果您做一个简单的运行，那应该会告诉您 input/output 文件是您想要 [=16= 的每个计划规则的]（在你的运行命令中使用标志 -n -r）。

snakemake - 规则全部缺少输入文件

snakemake - Missing input files for rule all