Ubuntu 升级后运行奇点时出现 squashfs 错误

Question

概览:

我正在运行ning Ubuntu 18.04 机器上处理奇点图像中的代码。截至周二晚上，它在机器 A 上运行（好吧，我正在调试代码）。我运行更新了，从第二天开始，任何对运行 any 奇点容器的尝试都给我以下错误：

ERROR  : Failed to mount squashfs image in (read only): Invalid argument
ABORT  : Retval = 255

确认这不是我的代码造成的（通过运行ning singularity exec <my_image_file>.sif 并得到同样的错误，我记得我在另一台机器上有几乎相同的设置（B)。我能够运行那里的图像没有任何问题。

不幸的是，我忘记了我在那台机器上选择了“无人值守更新”并重新启动。现在 returns 同样的错误。

我的尝试:

我这里有点空虚。我所做的只是说服自己，它与我的代码没有直接关系，而且几乎可以肯定与更新的 something 相关。我真的对 squashfs 知之甚少，而且对 singularity 还比较陌生。我还没有在网上找到任何有这个确切错误的东西（“无效参数”部分）。作为万福玛丽，我在机器 A 上重新安装了奇点，但它没有改变。

我不认为这是奇点本身的问题，而且我使用的是旧版本，所以我认为这不是一个有效的 github 问题。我使用的是 2.5.2 版，因为这是此代码适用于集群的版本。

详情

-vv 来自奇点的日志（注意：无论我传递什么参数，反馈都是相同的）：

Increasing verbosity level (4)
Singularity version: 2.5.2-dist
Exec'ing: /usr/local/libexec/singularity/cli/exec.exec
Evaluating args: '<my_image_file>.sif <my_program> <program_arg>'
VERBOSE: Set messagelevel to: 4
VERBOSE: Initialize configuration file: /usr/local/etc/singularity/singularity.conf
VERBOSE: Got config key allow setuid = 'yes'
VERBOSE: Got config key max loop devices = '256'
VERBOSE: Got config key allow pid ns = 'yes'
VERBOSE: Got config key config passwd = 'yes'
VERBOSE: Got config key config group = 'yes'
VERBOSE: Got config key config resolv_conf = 'yes'
VERBOSE: Got config key mount proc = 'yes'
VERBOSE: Got config key mount sys = 'yes'
VERBOSE: Got config key mount dev = 'yes'
VERBOSE: Got config key mount devpts = 'yes'
VERBOSE: Got config key mount home = 'yes'
VERBOSE: Got config key mount tmp = 'yes'
VERBOSE: Got config key mount hostfs = 'no'
VERBOSE: Got config key bind path = '/etc/localtime'
VERBOSE: Got config key bind path = '/etc/hosts'
VERBOSE: Got config key user bind control = 'yes'
VERBOSE: Got config key enable overlay = 'try'
VERBOSE: Got config key mount slave = 'yes'
VERBOSE: Got config key sessiondir max size = '16'
VERBOSE: Got config key allow container squashfs = 'yes'
VERBOSE: Got config key allow container extfs = 'yes'
VERBOSE: Got config key allow container dir = 'yes'
VERBOSE: Initializing Singularity Registry
VERBOSE: Adding value to registry: 'COMMAND' = 'exec'
VERBOSE: Adding value to registry: 'SYSCONFDIR' = '/usr/local/etc'
VERBOSE: Adding value to registry: 'VERSION' = '2.5.2-dist'
VERBOSE: Adding value to registry: 'IMAGE' = '<my_image_file>.sif'
VERBOSE: Adding value to registry: 'MESSAGELEVEL' = '4'
VERBOSE: Adding value to registry: 'BINDIR' = '/usr/local/bin'
VERBOSE: Adding value to registry: 'LOCALSTATEDIR' = '/usr/local/var'
VERBOSE: Adding value to registry: 'LIBEXECDIR' = '/usr/local/libexec'
VERBOSE: Set home (via getpwuid()) to: <my_home_directory>
VERBOSE: Running SUID program workflow
VERBOSE: Checking program has appropriate permissions
VERBOSE: Checking configuration file is properly owned by root
VERBOSE: Checking if singularity.conf allows us to run as suid
VERBOSE: Invoking the user namespace
VERBOSE: Not virtualizing USER namespace: running as SUID
VERBOSE: No autofs bug path in configuration, skipping
VERBOSE: Instantiating read only container image object
VERBOSE: Checking that file pointer is a Singularity image
VERBOSE: File is a valid SquashFS image
VERBOSE: Adding value to registry: 'CLEANUPD_FD' = '-1'
VERBOSE: Not virtualizing IPC namespace on user request
VERBOSE: Not virtualizing PID namespace on user request
VERBOSE: Not virtualizing network namespace on user request
VERBOSE: Using session directory: /usr/local/var/singularity/mnt/session
VERBOSE: Adding value to registry: 'SESSIONDIR' = '/usr/local/var/singularity/mnt/session'
VERBOSE: Found available loop device: /dev/loop18
VERBOSE: Using loop device: /dev/loop18
VERBOSE: Mounting squashfs image: /dev/loop18 -> /usr/local/var/singularity/mnt/container
ERROR  : Failed to mount squashfs image in (read only): Invalid argument
ABORT  : Retval = 255

grep for 'upgrade' in dpkg.log:

2020-07-23 06:10:36 upgrade evolution-data-server-common:all 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:10:41 upgrade linux-libc-dev:amd64 4.15.0-111.112 4.15.0-112.113
2020-07-23 06:10:46 upgrade python-pil:amd64 5.1.0-1ubuntu0.2 5.1.0-1ubuntu0.3
2020-07-23 06:10:51 upgrade python3-pil:amd64 5.1.0-1ubuntu0.2 5.1.0-1ubuntu0.3
2020-07-23 06:10:56 upgrade python2.7:amd64 2.7.17-1~18.04ubuntu1 2.7.17-1~18.04ubuntu1.1
2020-07-23 06:10:57 upgrade libpython2.7:amd64 2.7.17-1~18.04ubuntu1 2.7.17-1~18.04ubuntu1.1
2020-07-23 06:10:57 upgrade libpython2.7-stdlib:amd64 2.7.17-1~18.04ubuntu1 2.7.17-1~18.04ubuntu1.1
2020-07-23 06:10:58 upgrade python2.7-minimal:amd64 2.7.17-1~18.04ubuntu1 2.7.17-1~18.04ubuntu1.1
2020-07-23 06:10:58 upgrade libpython2.7-minimal:amd64 2.7.17-1~18.04ubuntu1 2.7.17-1~18.04ubuntu1.1
2020-07-23 06:11:06 upgrade python3.6-venv:amd64 3.6.9-1~18.04ubuntu1 3.6.9-1~18.04ubuntu1.1
2020-07-23 06:11:06 upgrade python3.6-dev:amd64 3.6.9-1~18.04ubuntu1 3.6.9-1~18.04ubuntu1.1
2020-07-23 06:11:06 upgrade libpython3.6-dev:amd64 3.6.9-1~18.04ubuntu1 3.6.9-1~18.04ubuntu1.1
2020-07-23 06:11:11 upgrade libpython3.6:amd64 3.6.9-1~18.04ubuntu1 3.6.9-1~18.04ubuntu1.1
2020-07-23 06:11:11 upgrade python3.6:amd64 3.6.9-1~18.04ubuntu1 3.6.9-1~18.04ubuntu1.1
2020-07-23 06:11:11 upgrade libpython3.6-stdlib:amd64 3.6.9-1~18.04ubuntu1 3.6.9-1~18.04ubuntu1.1
2020-07-23 06:11:12 upgrade python3.6-minimal:amd64 3.6.9-1~18.04ubuntu1 3.6.9-1~18.04ubuntu1.1
2020-07-23 06:11:12 upgrade libpython3.6-minimal:amd64 3.6.9-1~18.04ubuntu1 3.6.9-1~18.04ubuntu1.1
2020-07-23 06:11:29 upgrade linux-generic-hwe-18.04:amd64 5.3.0.62.115 5.4.0.42.46~18.04.35
2020-07-23 06:11:29 upgrade linux-image-generic-hwe-18.04:amd64 5.3.0.62.115 5.4.0.42.46~18.04.35
2020-07-23 06:11:41 upgrade linux-headers-generic-hwe-18.04:amd64 5.3.0.62.115 5.4.0.42.46~18.04.35
2020-07-23 06:12:37 upgrade evolution-data-server:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:37 upgrade libebook-1.2-19:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:38 upgrade libedata-book-1.2-25:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:38 upgrade libedata-cal-1.2-28:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:38 upgrade libebook-contacts-1.2-2:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:38 upgrade libebackend-1.2-10:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:38 upgrade libecal-1.2-19:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:38 upgrade libedataserver-1.2-23:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:39 upgrade libcamel-1.2-61:amd64 3.28.5-0ubuntu0.18.04.2 3.28.5-0ubuntu0.18.04.3
2020-07-23 06:12:44 upgrade libavformat-dev:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:44 upgrade libavcodec-dev:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:45 upgrade libswresample-dev:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:45 upgrade libswresample2:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:45 upgrade libswscale-dev:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:45 upgrade libswscale4:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:46 upgrade libavresample-dev:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:46 upgrade libavresample3:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:46 upgrade libavutil-dev:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:46 upgrade libavutil55:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:46 upgrade libavcodec57:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:47 upgrade libavformat57:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:52 upgrade libpostproc54:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:12:57 upgrade libavfilter6:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:13:02 upgrade libavdevice57:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2
2020-07-23 06:13:06 upgrade ffmpeg:amd64 7:3.4.6-0ubuntu0.18.04.1 7:3.4.8-0ubuntu0.2

该图像是根据 Docker 图像构建的，并经过一些额外的步骤。我没有提供任何相关信息，因为行为似乎并不取决于我何时构建图像。

Answer 1

我遇到了这个确切的问题，我设法在 Ubuntu 18.04 中使用奇点版本 2.6.1 修复了它（不过我认为在 2.5.2 中解决方案对你来说不会有太大不同）。

要采取的步骤

如果需要请卸载早期版本的 Singularity。
通过打开终端和运行以下命令下载正确的 Singularity 版本的 tarball：

# Go to the place you want to work in
cd ~
VERSION=2.6.1  # Set an environment variable just for convenience, could replace with another version.
# Download
wget https://github.com/singularityware/singularity/releases/download/$VERSION/singularity-$VERSION.tar.gz
# Extract
tar xvf singularity-$VERSION.tar.gz

现在，应该有一个文件夹叫singularity-2.6.1（我是放在首页（~）。然后打开文件mount.c放在singularity-2.6.1/src/lib/image/squashfs/mount.c（我不确定 v2.5.2 是否相同，但我认为变化不大。不过我们可能运气不好）
在此文件中找到代码行

if ( singularity_mount(loop_dev, mount_point, "squashfs", MS_NOSUID|MS_RDONLY|MS_NODEV, "errors=remount-ro") < 0 )

改为

if ( singularity_mount(loop_dev, mount_point, "squashfs", MS_NOSUID|MS_RDONLY|MS_NODEV, "") < 0 )

别忘了保存！

Return 到终端，通过运行命令

cd singularity-$VERSION
./configure --prefix=/usr/local
make
sudo make install

对我来说，问题现在已经解决了！

编辑

正如 aguadopd 所指出的，如果您没有安装 libarchive-dev，make 命令可能无法工作。如果是这种情况，那么首先在终端运行中修复它

sudo apt-get update -y
sudo apt-get install -y libarchive-dev

然后在一个新终端

singularity shell -p $HOME/Downloads/hrwros-09.simg

问题原理

显然，squashfs 中曾经有一个错误，如果它不知道，它就会简单地忽略参数。当今年早些时候修复此问题时，它会在遇到无效参数时开始出错，就像我们的案例一样。（这里也有解释https://www.mail-archive.com/debian-kernel@lists.debian.org/msg117435.html）因此，删除未知参数 "errors=remount-ro" 解决了问题。

老实说，由于我是 100% 的新手，所以我不确定它是否会在其他地方引起任何新的错误。然而，这也是在后来的 Singularity (https://github.com/hpcng/singularity/pull/4802/files/3f3136be150d9a8f63e58a0b4e39dc8487f1c0dc) 版本中基本修复此错误的方式。

希望对您有所帮助！

Answer 2

谢谢。我按照你的步骤做了，成功了。但是，我在“make”步骤中遇到了问题。它没有用，因为我没有安装“libarchive-dev”。为了修复它，我做了

sudo apt-get update -y
sudo apt-get install -y libarchive-dev

然后在一个新终端

奇点shell -p $HOME/Downloads/hrwros-09.simg

现在可以使用了。

Ubuntu 升级后运行奇点时出现 squashfs 错误

squashfs error when running singularity after Ubuntu upgrade

squashfs

singularity-container

ubuntu-18.04

Ubuntu 升级后 运行 奇点时出现 squashfs 错误

squashfs error when running singularity after Ubuntu upgrade

squashfs

singularity-container

ubuntu-18.04

Ubuntu 升级后运行奇点时出现 squashfs 错误