docker compos up 挂载卷失败
docker compose up Failed to mount volumes
我在 运行 docker-compose up
时遇到这些错误
ERROR: for airflow_webserver_1 Cannot start service webserver: OCI
runtime create failed: container_linux.go:346: starting container
process caused "process_linux.go:449: container init caused
\"rootfs_linux.go:58: mounting \"/srv/airflow/dags\" to rootfs
\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a/merged\"
at
\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a/merged/usr/local/airflow/dags\"
caused \"not a directory\"\"": unknown: Are you trying to mount a
directory onto a file (or vice-versa)? Check if the specified host
path exists and is the expected type
ERROR: for webserver Cannot start service webserver: OCI runtime
create failed: container_linux.go:346: starting container process
caused "process_linux.go:449: container init caused
\"rootfs_linux.go:58: mounting \"/srv/airflow/dags\" to rootfs
\"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a
我看过类似的帖子说我应该检查挂载的目录是否存在并且是目录（不是文件）等...
我检查过在我的主机上路径 /src/airflow/dags 存在并且包含几个文件（我的 dags 的代码来自以前的容器 运行s）。
我怎样才能摆脱这个错误，为什么会发生这种情况？。我不介意从头开始重新启动容器，丢失之前的卷内容。
这里是docker-撰写内容：
版本：'2.1'
服务：
邮递员：
图片：postgres:9.6
重新启动：除非停止
环境：
- POSTGRES_USER=气流
- POSTGRES_PASSWORD=气流
- POSTGRES_DB=气流
webserver:
build: airflow-server
restart: unless-stopped
depends_on:
- postgres
environment:
- LOAD_EX=n
- EXECUTOR=Local
- VIRTUAL_HOST=airflow.agatha.com
- VIRTUAL_PORT=8080
volumes:
- /srv/airflow/dags:/usr/local/airflow/dags
# Uncomment to include custom plugins
# - /srv/airflow/plugins:/usr/local/airflow/plugins
ports:
- "8075:8080"
command: webserver
healthcheck:
test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
interval: 30s
timeout: 30s
retries: 3
logging:
driver: json-file
options:
max-size: 50m
网络：
默认：
外部的：
名称：nginx-proxy
（airflow-server 文件夹包含自定义 Dockerfile 以复制一些配置文件）
这是 Dockerfile
FROM puckel/docker-airflow
USER root
COPY proxy.conf /etc/apt/apt.conf.d/proxy.conf
RUN apt-get update && apt-get install -y gnupg apt-transport-https procps
RUN mkdir /usr/local/airflow/ssh
COPY id_rsa_airflow_user /usr/local/airflow/ssh/id_rsa_airflow_user
RUN mkdir /usr/local/airflow/sparkScripts
COPY FileUnion.py /usr/local/airflow/sparkScripts
COPY ETLDag.py /usr/local/airflow/dags
USER airflow
完整的命令行输出：
> :~/docker/airflow$ sudo docker-compose up WARNING: The Docker Engine
> you're using is running in swarm mode.
>
> Compose does not use swarm mode to deploy services to multiple nodes
> in a swarm. All containers will be scheduled on the current node.
>
> To deploy your application across the swarm, use `docker stack
> deploy`.
>
> Starting airflow_postgres_1 ... done Recreating airflow_webserver_1
> ... error
>
> ERROR: for airflow_webserver_1 Cannot start service webserver: OCI
> runtime create failed: container_linux.go:348: starting container
> process caused "process_linux.go:402: container init caused
> \"rootfs_linux.go:58: mounting \\"/srv/airflow/dags\\" to rootfs
> \\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac\\"
> at
> \\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac/usr/local/airflow/dags\\"
> caused \\"not a directory\\"\"": unknown: Are you trying to mount a
> directory onto a file (or vice-versa)? Check if the specified host
> path exists and is the expected type
>
> ERROR: for webserver Cannot start service webserver: OCI runtime
> create failed: container_linux.go:348: starting container process
> caused "process_linux.go:402: container init caused
> \"rootfs_linux.go:58: mounting \\"/srv/airflow/dags\\" to rootfs
> \\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac\\"
> at
> \\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac/usr/local/airflow/dags\\"
> caused \\"not a directory\\"\"": unknown: Are you trying to mount a
> directory onto a file (or vice-versa)? Check if the specified host
> path exists and is the expected type ERROR: Encountered errors while
> bringing up the project.
以及airflow-server目录内容（部分文件未被使用）
~/docker/airflow$ ls -l airflow-server/
total 326788
-rw-rw-r-- 1 user user 5008 mar 25 12:16 Dockerfile
-rw-rw-r-- 1 user user 7302 mar 24 15:37 ETLDagCreator.py
-rw-rw-r-- 1 user user 5900 mar 24 16:16 ETLDag.py
-rw-rw-r-- 1 user user 2564 mar 24 15:19 FileUnion.py
-rw-r--r-- 1 user user 334559382 mar 18 11:08 hadoop-3.1.1.tar.gz
drwxrwxr-x 2 user user 4096 mar 19 10:42 hadoop-client-conf
-rwxrwxr-x 1 user user 2026 mar 18 10:40 hbase.service.keytab
-rwxrwxr-x 1 user user 132 mar 18 10:39 hdfs.headless.keytab
-rwxrwxr-x 1 user user 2002 mar 18 10:38 hive.service.keytab
-rw-rw-r-- 1 user user 625 mar 19 16:02 hosts
-rwxrwxr-x 1 user user 1679 mar 20 10:17 id_rsa_airflow_user
-rw-rw-r-- 1 user user 529 mar 18 10:33 krb5.conf
-rw-rw-r-- 1 user user 101 mar 18 10:45 proxy.conf
-rw-rw-r-- 1 user user 58 mar 18 16:47 resolv.conf
谢谢
如错误所示，您正在尝试将目录装载到文件中。这不是 Linux 中的有效绑定挂载，因此 docker 无法创建容器。该文件是在您的图像中创建的：
COPY ETLDag.py /usr/local/airflow/dags
如果您想将其复制到名为 dags 的目录中，您需要一个尾部斜杠：
COPY ETLDag.py /usr/local/airflow/dags/
您尝试在文件上挂载的目录来自您的卷挂载：
volumes:
- /srv/airflow/dags:/usr/local/airflow/dags
您在评论中显示的是一个目录：
$ ls -ld /srv/airflow/dags
drwxr-xr-x 2 root root 4096 mar 25 12:45 /srv/airflow/dags
这些必须都是文件或都是目录。不能混用。
我在 运行 docker-compose up
ERROR: for airflow_webserver_1 Cannot start service webserver: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \"/srv/airflow/dags\" to rootfs \"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a/merged\" at \"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a/merged/usr/local/airflow/dags\" caused \"not a directory\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: for webserver Cannot start service webserver: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \"/srv/airflow/dags\" to rootfs \"/var/lib/docker/overlay2/ccf82b65760c1dc8714db8d6b105ab865183c422e8d7d015988dc157104cce0a
我看过类似的帖子说我应该检查挂载的目录是否存在并且是目录（不是文件）等...
我检查过在我的主机上路径 /src/airflow/dags 存在并且包含几个文件（我的 dags 的代码来自以前的容器 运行s）。
我怎样才能摆脱这个错误，为什么会发生这种情况？。我不介意从头开始重新启动容器，丢失之前的卷内容。
这里是docker-撰写内容：
版本：'2.1' 服务： 邮递员： 图片：postgres:9.6 重新启动：除非停止 环境： - POSTGRES_USER=气流 - POSTGRES_PASSWORD=气流 - POSTGRES_DB=气流
webserver:
build: airflow-server
restart: unless-stopped
depends_on:
- postgres
environment:
- LOAD_EX=n
- EXECUTOR=Local
- VIRTUAL_HOST=airflow.agatha.com
- VIRTUAL_PORT=8080
volumes:
- /srv/airflow/dags:/usr/local/airflow/dags
# Uncomment to include custom plugins
# - /srv/airflow/plugins:/usr/local/airflow/plugins
ports:
- "8075:8080"
command: webserver
healthcheck:
test: ["CMD-SHELL", "[ -f /usr/local/airflow/airflow-webserver.pid ]"]
interval: 30s
timeout: 30s
retries: 3
logging:
driver: json-file
options:
max-size: 50m
网络： 默认： 外部的： 名称：nginx-proxy
（airflow-server 文件夹包含自定义 Dockerfile 以复制一些配置文件）
这是 Dockerfile
FROM puckel/docker-airflow
USER root
COPY proxy.conf /etc/apt/apt.conf.d/proxy.conf
RUN apt-get update && apt-get install -y gnupg apt-transport-https procps
RUN mkdir /usr/local/airflow/ssh
COPY id_rsa_airflow_user /usr/local/airflow/ssh/id_rsa_airflow_user
RUN mkdir /usr/local/airflow/sparkScripts
COPY FileUnion.py /usr/local/airflow/sparkScripts
COPY ETLDag.py /usr/local/airflow/dags
USER airflow
完整的命令行输出：
> :~/docker/airflow$ sudo docker-compose up WARNING: The Docker Engine > you're using is running in swarm mode. > > Compose does not use swarm mode to deploy services to multiple nodes > in a swarm. All containers will be scheduled on the current node. > > To deploy your application across the swarm, use `docker stack > deploy`. > > Starting airflow_postgres_1 ... done Recreating airflow_webserver_1 > ... error > > ERROR: for airflow_webserver_1 Cannot start service webserver: OCI > runtime create failed: container_linux.go:348: starting container > process caused "process_linux.go:402: container init caused > \"rootfs_linux.go:58: mounting \\"/srv/airflow/dags\\" to rootfs > \\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac\\" > at > \\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac/usr/local/airflow/dags\\" > caused \\"not a directory\\"\"": unknown: Are you trying to mount a > directory onto a file (or vice-versa)? Check if the specified host > path exists and is the expected type > > ERROR: for webserver Cannot start service webserver: OCI runtime > create failed: container_linux.go:348: starting container process > caused "process_linux.go:402: container init caused > \"rootfs_linux.go:58: mounting \\"/srv/airflow/dags\\" to rootfs > \\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac\\" > at > \\"/var/lib/docker/aufs/mnt/8ce629cdb64b63552688c4843fff357530fb5a938ac4219ffc0ef753d3b222ac/usr/local/airflow/dags\\" > caused \\"not a directory\\"\"": unknown: Are you trying to mount a > directory onto a file (or vice-versa)? Check if the specified host > path exists and is the expected type ERROR: Encountered errors while > bringing up the project.
以及airflow-server目录内容（部分文件未被使用）
~/docker/airflow$ ls -l airflow-server/
total 326788
-rw-rw-r-- 1 user user 5008 mar 25 12:16 Dockerfile
-rw-rw-r-- 1 user user 7302 mar 24 15:37 ETLDagCreator.py
-rw-rw-r-- 1 user user 5900 mar 24 16:16 ETLDag.py
-rw-rw-r-- 1 user user 2564 mar 24 15:19 FileUnion.py
-rw-r--r-- 1 user user 334559382 mar 18 11:08 hadoop-3.1.1.tar.gz
drwxrwxr-x 2 user user 4096 mar 19 10:42 hadoop-client-conf
-rwxrwxr-x 1 user user 2026 mar 18 10:40 hbase.service.keytab
-rwxrwxr-x 1 user user 132 mar 18 10:39 hdfs.headless.keytab
-rwxrwxr-x 1 user user 2002 mar 18 10:38 hive.service.keytab
-rw-rw-r-- 1 user user 625 mar 19 16:02 hosts
-rwxrwxr-x 1 user user 1679 mar 20 10:17 id_rsa_airflow_user
-rw-rw-r-- 1 user user 529 mar 18 10:33 krb5.conf
-rw-rw-r-- 1 user user 101 mar 18 10:45 proxy.conf
-rw-rw-r-- 1 user user 58 mar 18 16:47 resolv.conf
谢谢
如错误所示，您正在尝试将目录装载到文件中。这不是 Linux 中的有效绑定挂载，因此 docker 无法创建容器。该文件是在您的图像中创建的：
COPY ETLDag.py /usr/local/airflow/dags
如果您想将其复制到名为 dags 的目录中，您需要一个尾部斜杠：
COPY ETLDag.py /usr/local/airflow/dags/
您尝试在文件上挂载的目录来自您的卷挂载：
volumes:
- /srv/airflow/dags:/usr/local/airflow/dags
您在评论中显示的是一个目录：
$ ls -ld /srv/airflow/dags
drwxr-xr-x 2 root root 4096 mar 25 12:45 /srv/airflow/dags
这些必须都是文件或都是目录。不能混用。