Collectstatic failing - botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

Question

我正在尝试运行在部署到 AWS 的阶段收集静态信息。我收到以下错误，并且没有文件被放入存储桶中：

Traceback (most recent call last):
  File "/home/hcc/.local/lib/python3.6/site-packages/storages/backends/s3boto3.py", line 523, in _open
    f = S3Boto3StorageFile(name, mode, self)
  File "/home/hcc/.local/lib/python3.6/site-packages/storages/backends/s3boto3.py", line 74, in __init__
    self.obj.load()
  File "/home/hcc/.local/lib/python3.6/site-packages/boto3/resources/factory.py", line 505, in do_action
    response = action(self, *args, **kwargs)
  File "/home/hcc/.local/lib/python3.6/site-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(*args, **params)
  File "/home/hcc/.local/lib/python3.6/site-packages/botocore/client.py", line 316, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/hcc/.local/lib/python3.6/site-packages/botocore/client.py", line 635, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/home/hcc/.local/lib/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/home/hcc/.local/lib/python3.6/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/hcc/.local/lib/python3.6/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/hcc/.local/lib/python3.6/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/home/hcc/.local/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 161, in handle
    if self.is_local_storage() and self.storage.location:
  File "/home/hcc/.local/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 215, in is_local_storage
    return isinstance(self.storage, FileSystemStorage)
  File "/home/hcc/.local/lib/python3.6/site-packages/django/utils/functional.py", line 224, in inner
    self._setup()
  File "/home/hcc/.local/lib/python3.6/site-packages/django/contrib/staticfiles/storage.py", line 501, in _setup
    self._wrapped = get_storage_class(settings.STATICFILES_STORAGE)()
  File "/home/hcc/.local/lib/python3.6/site-packages/django/contrib/staticfiles/storage.py", line 379, in __init__
    self.hashed_files = self.load_manifest()
  File "/home/hcc/.local/lib/python3.6/site-packages/django/contrib/staticfiles/storage.py", line 389, in load_manifest
    content = self.read_manifest()
  File "/home/hcc/.local/lib/python3.6/site-packages/django/contrib/staticfiles/storage.py", line 383, in read_manifest
    with self.open(self.manifest_name) as manifest:
  File "/home/hcc/.local/lib/python3.6/site-packages/django/core/files/storage.py", line 36, in open
    return self._open(name, mode)
  File "/home/hcc/.local/lib/python3.6/site-packages/storages/backends/s3boto3.py", line 526, in _open
    raise IOError('File does not exist: %s' % name)
OSError: File does not exist: static/staticfiles.json

我的静态文件设置如下settings.py：

STATICFILES_FINDERS = [
    "django.contrib.staticfiles.finders.FileSystemFinder",
    "django.contrib.staticfiles.finders.AppDirectoriesFinder",
]

STATICFILES_DIRS = [join(BASE_DIR, "assets", "build")]

DEFAULT_FILE_STORAGE = "staticfiles.storage.MediaStorage"
STATICFILES_STORAGE = "staticfiles.storage.ManifestStaticFilesStorage"

STATIC_FILES_BUCKET = "stage-bucket"
STATIC_FILES_LOCATION = "static"

STATIC_ROOT = join(BASE_DIR, "static")
STATIC_URL = f"https://{STATIC_FILES_BUCKET}.s3.amazonaws.com/{STATIC_FILES_LOCATION}/"

MEDIA_FILES_BUCKET = ""
MEDIA_FILES_LOCATION = ""

MEDIA_ROOT = join(BASE_DIR, "media")
MEDIA_URL = "/media/"

我使用的自定义存储class是：

from django.contrib.staticfiles.storage import ManifestFilesMixin
from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage


class ManifestStaticFilesStorage(ManifestFilesMixin, S3Boto3Storage):
    bucket_name = settings.STATIC_FILES_BUCKET
    location = settings.STATIC_FILES_LOCATION


class MediaStorage(S3Boto3Storage):
    bucket_name = settings.MEDIA_FILES_BUCKET
    location = settings.MEDIA_FILES_LOCATION
    file_overwrite = False

Requirements.txt 包括：

Django==3.0.3
gunicorn==20.0.4

django-model-utils==4.0.0
django-storages==1.9.1
boto3==1.13.15
htmlmin==0.1.12

sentry-sdk==0.14.3
slackclient==2.5.0
structlog==18.2.0
ddtrace==0.31.0
colorama==0.3.9
json-logging-py==0.2

当我从 ManifestStaticFilesStorage class 中删除 ManifestFilesMixin 时，collectstatic 有效，但显然不适用于散列文件或我需要的 staticfiles.json。

我花了好几天时间试图解决这个问题，但我对可能的原因感到困惑。如有任何信息，我们将不胜感激。

Answer 1

从遇到类似问题的人那里找到了关于 another site 的答案。似乎 S3Boto3Storage 抛出 IOError 但 ManifestFilesMixin 期待 FileNotFound。这解决了我的问题，我现在在我的 S3 存储桶中看到散列文件以及 staticfiles.json。

class ManifestStaticFilesStorage(ManifestFilesMixin, S3Boto3Storage):
    bucket_name = settings.STATIC_FILES_BUCKET
    location = settings.STATIC_FILES_LOCATION

    def read_manifest(self):
        try:
            return super(ManifestStaticFilesStorage,self).read_manifest()
        except IOError:
            return None

Collectstatic failing - botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

Collectstatic failing - botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

python

django

amazon-s3

django-storage

collectstatic