为所有现有 ActiveStorage 变体创建数据库记录的 Rake 任务

Question

在 Rails 6.1 中，ActiveStorage 在第一次加载时为所有变体创建数据库记录：https://github.com/rails/rails/pull/37901

我想启用它，但由于我的生产 Rails 应用程序中有数万个文件，让用户创建如此多的数据库记录会有问题（而且可能很慢）当他们浏览网站时。有没有办法编写一个 Rake 任务来遍历我数据库中的每个附件，并生成变体并将它们保存在数据库中？

我曾经运行启用新的 active_storage.track_variants 配置后，所有新上传的文件在第一次加载时都会被保存。

感谢您的帮助！

Answer 1

这是我最终为此创建的 Rake 任务。如果你有一个较小的数据集，可以删除并行的东西，但我发现在没有任何并行化的情况下，对于 70k+ 变体，它的速度慢得令人无法忍受。也可以忽略进度条相关的代码:)

基本上，我只获取所有有附件的模型（我手动执行此操作，如果你有大量附件，你可以以更动态的方式进行），然后过滤那些不可变的模型.然后我检查每个附件并为我定义的每个尺寸生成一个变体，然后对其调用 process 以强制将其保存到数据库中。

确保捕获任务中的 MiniMagick（或 vips，如果你愿意）错误，这样一个错误的图像文件就不会破坏一切。

# Rails 6.1 changes the way ActiveStorage works so that variants are
# tracked in the database. The intent of this task is to create the
# necessary variants for all game covers and user avatars in our database.
# This way, the user isn't creating dozens of variant records as they
# browse the site. We want to create them ahead-of-time, when we deploy
# the change to track variants.
namespace 'active_storage:vglist:variants' do
  require 'ruby-progressbar'
  require 'parallel'

  desc "Create all variants for covers and avatars in the database."
  task create: :environment do
    games = Game.joins(:cover_attachment)
    # Only attempt to create variants if the cover is able to have variants.
    games = games.filter { |game| game.cover.variable? }
    puts 'Creating game cover variants...'

    # Use the configured max number of threads, with 2 leftover for web requests.
    # Clamp it to 1 if the configured max threads is 2 or less for whatever reason.
    thread_count = [(ENV.fetch('RAILS_MAX_THREADS', 5).to_i - 2), 1].max

    games_progress_bar = ProgressBar.create(
      total: games.count,
      format: "\e[0;32m%c/%C |%b>%i| %e\e[0m"
    )

    # Disable logging in production to prevent log spam.
    Rails.logger.level = 2 if Rails.env.production?

    Parallel.each(games, in_threads: thread_count) do |game|
      ActiveRecord::Base.connection_pool.with_connection do
        begin
          [:small, :medium, :large].each do |size|
            game.sized_cover(size).process
          end
        # Rescue MiniMagick errors if they occur so that they don't block the
        # task from continuing.
        rescue MiniMagick::Error => e
          games_progress_bar.log "ERROR: #{e.message}"
          games_progress_bar.log "Failed on game ID: #{game.id}"
        end
        games_progress_bar.increment
      end
    end

    games_progress_bar.finish unless games_progress_bar.finished?

    users = User.joins(:avatar_attachment)
    # Only attempt to create variants if the avatar is able to have variants.
    users = users.filter { |user| user.avatar.variable? }
    puts 'Creating user avatar variants...'

    users_progress_bar = ProgressBar.create(
      total: users.count,
      format: "\e[0;32m%c/%C |%b>%i| %e\e[0m"
    )

    Parallel.each(users, in_threads: thread_count) do |user|
      ActiveRecord::Base.connection_pool.with_connection do
        begin
          [:small, :medium, :large].each do |size|
            user.sized_avatar(size).process
          end
        # Rescue MiniMagick errors if they occur so that they don't block the
        # task from continuing.
        rescue MiniMagick::Error => e
          users_progress_bar.log "ERROR: #{e.message}"
          users_progress_bar.log "Failed on user ID: #{user.id}"
        end
        users_progress_bar.increment
      end
    end

    users_progress_bar.finish unless users_progress_bar.finished?
  end
end

这就是 sized_cover 在 game.rb 中的样子：

def sized_cover(size)
  width, height = COVER_SIZES[size]
  cover&.variant(
    resize_to_limit: [width, height]
  )
end

sized_avatar 几乎是一回事。

为所有现有 ActiveStorage 变体创建数据库记录的 Rake 任务

Rake task for creating database records for all existing ActiveStorage variants

ruby-on-rails

rails-activestorage