将 EBS 卷（不是快照）挂载到 Elastic Beanstalk EC2

Question

我正在将遗留应用程序迁移到 Elastic Beanstalk。它需要持久存储（暂时）。我想挂载 EBS 卷。

我希望以下内容在 .ebextensions/ebs.config 中有效：

commands:
  01mkdir:
    command: "mkdir /data"
  02mount:
    command: "mount /dev/sdh /data"

option_settings:
  - namespace: aws:autoscaling:launchconfiguration
    option_name: BlockDeviceMappings
    value: /dev/sdh=vol-XXXXX

https://blogs.aws.amazon.com/application-management/post/Tx224DU59IG3OR9/Customize-Ephemeral-and-EBS-Volumes-in-Elastic-Beanstalk-Environments

但不幸的是，我收到以下错误消息“(vol-XXXX) for parameter snapshotId is invalid. Expected: 'snap-...'.”

显然这种方法只允许快照。任何人都可以建议修复或替代方法。

Answer 1

我找到了解决办法。它可以通过删除 "sleep 10" 来改进，但不幸的是，这是必需的，因为 aws ec2 attach-volume 是异步的，并且 returns 在附件发生之前立即。

container_commands:
  01mount:
    command: "aws ec2 attach-volume --volume-id vol-XXXXXX --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --device /dev/sdh"
    ignoreErrors: true
  02wait:
    command: "sleep 10"
  03mkdir:
    command: "mkdir /data"
    test: "[ ! -d /data ]"
  04mount:
    command: "mount /dev/sdh /data"
    test: "! mountpoint -q /dev/sdh"

注意。理想情况下，commands 部分中的运行而不是 container_commands 但环境变量未及时设置。

Answer 2

添加到@Simon 的回答（以避免粗心者落入陷阱）：

如果挂载的持久存储最终将在 Docker 容器内使用（例如，如果您是运行 Jenkins 并且想要持久存储 jenkins_home），则需要重新启动运行安装后的 docker 容器。
您需要允许对 EC2 实例（或实例/* ARN）和要附加的卷（或卷/* ARN）执行 'ec2:AttachVolumes' 操作EB 承担了角色策略。没有这个，aws ec2 attach-volume 命令失败。
您还需要将 --region 传递给 aws ec2 ... 命令（至少，在撰写本文时）

Answer 3

这是缺少配置：

commands:
  01mount:
    command: "export AWS_ACCESS_KEY_ID=<replace by your AWS key> && export AWS_SECRET_ACCESS_KEY=<replace by your AWS secret> && aws ec2 attach-volume --volume-id <replace by you volume id> --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --device /dev/xvdf --region <replace with your region>"
    ignoreErrors: true
  02wait:
    command: "sleep 10"
  03mkdir:
    command: "mkdir /home/lucene"
    test: "[ ! -d /home/lucene ]"
  04mount:
    command: "mount /dev/xvdf /home/lucene"
    test: "! mountpoint -q /dev/xvdf"

Answer 4

或者，您可以考虑使用弹性文件系统 (EFS) 存储，而不是使用 EBS 卷。 AWS 发布了如何将 EFS 卷挂载到 Elastic Beanstalk EC2 实例的脚本，它还可以同时附加到多个 EC2 实例（这对于 EBS 是不可能的）。

http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/services-efs.html

Answer 5

这是一个配置文件，您可以将其放入 .ebextensions。您需要提供要附加的 VOLUME_ID。测试命令使得附加和安装仅在需要时发生，因此您可以 eb deploy 重复而不会出错。

container_commands:
  00attach:
    command: |
      export REGION=$(/opt/aws/bin/ec2-metadata -z | awk '{print substr(, 0, length()-1)}')
      export INSTANCE_ID=$(/opt/aws/bin/ec2-metadata -i | awk '{print }')
      export VOLUME_ID=$(aws ec2 describe-volumes --region ${REGION} --output text --filters Name=tag:Name,Values=tf-trading-prod --query 'Volumes[*].VolumeId')

      aws ec2 attach-volume --region ${REGION} --device /dev/sdh --instance-id ${INSTANCE_ID} --volume-id ${VOLUME_ID}
      aws ec2 wait volume-in-use --region ${REGION} --volume-ids ${VOLUME_ID}
      sleep 1
    test: "! file -E /dev/xvdh"
  01mkfs:
    command: "mkfs -t ext3 /dev/xvdh"
    test: "file -s /dev/xvdh | awk '{print }' | grep -q data"
  02mkdir:
    command: "mkdir -p /data"
  03mount:
    command: "mount /dev/xvdh /data"
    test: "! mountpoint /data"

Answer 6

必须使用 container_commands 因为当 commands 是运行时源包还没有完全解压。

.ebextensions/whatever.config

container_commands:
  chmod:
    command: chmod +x .platform/hooks/predeploy/mount-volume.sh

预部署挂钩运行在容器命令之后但在部署之前。无需重新启动您的 docker 容器，即使它在附加的 ebs 卷上安装了一个目录，因为 beantalk 在预部署挂钩完成后将其旋转起来。您可以在日志中看到它。

.platform/hooks/predeploy/mount-volume.sh

#!/bin/sh

# Make sure LF line endings are used in the file, otherwise there would be an error saying "file not found".

# All platform hooks run as root user, no need for sudo.

# Before attaching the volume find out the root volume's name, so that we can later use it for filtering purposes.
# -d – to filter out partitions.
# -P – to display the result as key-value pairs.
# -o – to output only the matching part.
# lsblk strips the "/dev/" part
ROOT_VOLUME_NAME=$(lsblk -d -P | grep -o 'NAME="[a-z0-9]*"' | grep -o '[a-z0-9]*')

aws ec2 attach-volume --volume-id vol-xxx --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --device /dev/sdf --region us-east-1
# The above command is async, so we need to wait.
aws ec2 wait volume-in-use --volume-ids vol-xxx --region us-east-1

# Now lsblk should show two devices. We figure out which one is non-root by filtering out the stored root volume name.
NON_ROOT_VOLUME_NAME=$(lsblk -d -P | grep -o 'NAME="[a-z0-9]*"' | grep -o '[a-z0-9]*' | awk -v name="$ROOT_VOLUME_NAME" '[=11=] !~ name')

FILE_COMMAND_OUTPUT=$(file -s /dev/$NON_ROOT_VOLUME_NAME)

# Create a file system on the non-root device only if there isn't one already, so that we don't accidentally override it.
if test "$FILE_COMMAND_OUTPUT" = "/dev/$NON_ROOT_VOLUME_NAME: data"; then
  mkfs -t xfs /dev/$NON_ROOT_VOLUME_NAME
fi

mkdir /data

mount /dev/$NON_ROOT_VOLUME_NAME /data

# Need to make sure that the volume gets mounted after every reboot, because by default only root volume is automatically mounted.

cp /etc/fstab /etc/fstab.orig

NON_ROOT_VOLUME_UUID=$(lsblk -d -P -o +UUID | awk -v name="$NON_ROOT_VOLUME_NAME" '[=11=] ~ name' | grep -o 'UUID="[-0-9a-z]*"' | grep -o '[-0-9a-z]*')

# We specify 0 to prevent the file system from being dumped, and 2 to indicate that it is a non-root device.
# If you ever boot your instance without this volume attached, the nofail mount option enables the instance to boot
# even if there are errors mounting the volume.
# Debian derivatives, including Ubuntu versions earlier than 16.04, must also add the nobootwait mount option.
echo "UUID=$NON_ROOT_VOLUME_UUID /data xfs defaults,nofail 0 2" | tee -a /etc/fstab

很确定我用 grep 和 awk 做的事情可以用更简洁的方式完成。我不擅长 Linux。

实例配置文件应包含以下权限：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AttachVolume",
                "ec2:DetachVolume",
                "ec2:DescribeVolumes"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:volume/*",
                "arn:aws:ec2:*:*:instance/*"
            ]
        }
    ]
}

您必须确保将 ebs 卷部署在与 beantalk 相同的 AZ 中，并且您使用单实例部署。然后，如果您的实例崩溃，ASG 将终止它，创建另一个实例，并将卷附加到保留所有数据的新实例。

将 EBS 卷（不是快照）挂载到 Elastic Beanstalk EC2

Mount a EBS volume (not snapshot) to Elastic Beanstalk EC2

amazon-ec2

amazon-ebs

amazon-elastic-beanstalk