使用 Slurm 和 Singularity 时缺少 'stdout' 文件

Question

我想使用 Nextflow 创建将由 Slurm 作业调度程序执行的奇点组件管道。我创建了一个简单的工作运行只是一个任务：

#!/usr/bin/env nextflow

process createGroups {

  executor = 'slurm'
  queue = 'upgrade'
  memory = '10 GB'
  time = '30 min'
  cpus = 8
  clusterOptions = '--workdir /path/to/the/singularity/image'
    """
    singularity run ...
    """
}

作业成功执行并给出退出状态 0，但 Nextflow 出现“缺少 'stdout' 文件”错误。有完整的输出：

Error executing process > 'createGroups'

Caused by:
  Missing 'stdout' file: .../work/89/449f849ffc77a967be91ed994da860/.command.out for process > createGroups

Command executed:
executor >  slurm (1)
[89/449f84] process > createGroups [100%] 1 of 1, failed: 1 ✘
Error executing process > 'createGroups'

Caused by:
  Missing 'stdout' file: .../work/89/449f849ffc77a967be91ed994da860/.command.out for process > createGroups

Command executed:

  singularity run ...

Command exit status:
  0

Command output:
  (empty)

Work dir:
  .../work/89/449f849ffc77a967be91ed994da860

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

我找不到解决此错误的任何解决方案:(

更新：使用 slurm 的 --workdir 参数会导致该错误。我只是使用“cd”命令更改路径，它解决了我的问题。

Answer 1

最好避免手动调用 Singularity，就像在您的脚本块中一样。假设 Singularity 将在队列中每个 execution/compute 节点上的 $PATH 中，那么，例如，您需要的是：

在你的nextflow.config中：

process {

  executor = 'slurm'
}

singularity {

  enabled = true
  cacheDir = '/path/to/containers'
}

然后，您的 nextflow 脚本将如下所示：

process bwa_mem {

    container = 'quay.io/biocontainers/bwa:0.7.17--h5bf99c6_8'

    queue = 'upgrade'

    cpus = 8
    memory = '10 GB'
    time = '30.m'

    """
    bwa mem
    """
}

另请参阅：https://www.nextflow.io/docs/latest/singularity.html

使用 Slurm 和 Singularity 时缺少 'stdout' 文件

Missing 'stdout' file when using Slurm and Singularity

slurm

nextflow

singularity-container