站点地图生成器无法生成站点地图

Question

我有这个网站https://shopus.pk。我无法使用站点地图生成器工具生成站点地图。他们只是给出 "Error: 422 Unprocessable Entity" 之类的错误，或者只给我 1 URL 如下所示：-

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">
</urlset>

我知道有一些问题可能与网站或服务器的安全设置有关。但是请有人帮我找出问题所在。谢谢

顺便说一句，我的网站由 dreamhost 托管。但我不认为dreamhost与此有任何关系。

我已经尝试了 https://www.xml-sitemaps.com/ , http://www.web-site-map.com/, http://www.check-domains.com/sitemap/index.php , https://websiteseochecker.com/html-sitemap-generator/ 以及更多。

我还下载并试用了 A1 Sitemap Generator、gnucrawlandmap、SiteMapBuilder、HSEO Sitemap Generator 和一些免费的站点地图生成工具。

以上所有网站或工具都会出现访问错误或 return 只需 1 或 2 URLs。

由于我的网站是在 Ruby 上构建的 Rails 我的生产环境配置文件如下：-

Rails.application.configure do
  # Settings specified here will take precedence over those in config/application.rb.

  # Code is not reloaded between requests.
  config.cache_classes = true

  # Eager load code on boot. This eager loads most of Rails and
  # your application in memory, allowing both threaded web servers
  # and those relying on copy on write to perform better.
  # Rake tasks automatically ignore this option for performance.
  config.eager_load = true

  # Full error reports are disabled and caching is turned on.
   config.consider_all_requests_local       = false
   config.action_controller.perform_caching = true

  # Enable Rack::Cache to put a simple HTTP cache in front of your application
  # Add `rack-cache` to your Gemfile before enabling this.
  # For large-scale production use, consider using a caching reverse proxy like
  # NGINX, varnish or squid.
  # config.action_dispatch.rack_cache = true

  # Disable serving static files from the `/public` folder by default since
  # Apache or NGINX already handles this.
  config.serve_static_files = ENV['RAILS_SERVE_STATIC_FILES'].present?

  # Compress JavaScripts and CSS.
  config.assets.js_compressor = :uglifier
  # config.assets.css_compressor = :sass

  # Do not fallback to assets pipeline if a precompiled asset is missed.
  config.assets.compile = true

  # Asset digests allow you to set far-future HTTP expiration dates on all assets,
  # yet still be able to expire them through the digest params.
  config.assets.digest = true

  # `config.assets.precompile` and `config.assets.version` have moved to config/initializers/assets.rb

  # Specifies the header that your server uses for sending files.
  # config.action_dispatch.x_sendfile_header = 'X-Sendfile' # for Apache
  # config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for NGINX

  # Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
  config.force_ssl = true

  # Use the lowest log level to ensure availability of diagnostic information
  # when problems arise.
  config.log_level = :error

  # Prepend all log lines with the following tags.
  # config.log_tags = [ :subdomain, :uuid ]

  # Use a different logger for distributed setups.
  # config.logger = ActiveSupport::TaggedLogging.new(SyslogLogger.new)

  # Use a different cache store in production.
  # config.cache_store = :mem_cache_store

  # Enable serving of images, stylesheets, and JavaScripts from an asset server.
  # config.action_controller.asset_host = 'http://assets.example.com'

  # Ignore bad email addresses and do not raise email delivery errors.
  # Set this to true and configure the email server for immediate delivery to raise delivery errors.
  # config.action_mailer.raise_delivery_errors = false

  config.action_mailer.default_url_options = { host: "https://shopus.pk" }
  # configure action_mailer
   config.action_mailer.delivery_method = :smtp
   config.action_mailer.smtp_settings = {}
  config.action_mailer.raise_delivery_errors = true
  config.action_mailer.perform_deliveries = true
  config.action_mailer.asset_host = 'https://shopus.pk'
  # Enable locale fallbacks for I18n (makes lookups for any locale fall back to
  # the I18n.default_locale when a translation cannot be found).
  config.i18n.fallbacks = true

   # Send deprecation notices to registered listeners.
   config.active_support.deprecation = :notify

  # Use default logging formatter so that PID and timestamp are not suppressed.
   config.log_formatter = ::Logger::Formatter.new

   # Do not dump schema after migrations.
  config.active_record.dump_schema_after_migration = false
end

这就是我的应用程序控制器的样子：-

class ApplicationController < ActionController::Base
  # Prevent CSRF attacks by raising an exception.
  # For APIs, you may want to use :null_session instead.
  protect_from_forgery with: :exception
  include SessionsHelper
  include ApplicationHelper
  private
  # Confirms a logged-in user.
  def logged_in_customer
    unless logged_in?
      store_location
      redirect_to login_url
    end
  end
end

如果您需要任何其他方法来解决此问题，请告诉我。

Answer 1

好的，看来我找到问题了。但还是不太确定。

因此，在尝试几乎所有站点地图生成器都惨遭失败后，我决定使用 Ruby Gems Nokogiri 和 Mechanize 创建我自己的站点地图生成器。但令我惊讶的是，每当我尝试从我的网站提取 HTML 代码时，同样的错误会显示“422 Unprocessable Entity”。这是我从几个站点地图生成器那里得到的确切错误消息。

我从应用程序控制器中删除了“protect_from_forgery with: :exception”，站点地图生成器开始在我的网站上运行。

但这不对，因为“protect_from_forgery with: :exception”应该在那里。我还有 2 个其他网站，应用程序控制器中包含“protect_from_forgery with: :exception”。站点地图生成器在使用这两个网站时没有显示任何问题。

我的第一个网站和其他两个网站的唯一区别是我的第一个网站使用 ajax 而其他两个很简单。所以我终于想通了，当我从

中删除 format.js 行时

respond_to do |format|
   format.js
   format.html
end

我在主控制器中的索引操作的代码块，一切都会开始工作。后来我意识到我应该把format.js下面format.html的respond_to代码写成这样

respond_to do |format|
   format.html
   format.js
end

在此之后，我在 format.html 以上 format.js

所有控制器的每个动作中更改了所有 respond_to 代码

现在一切正常。

但是我还是一头雾水，不确定我对问题原因的判断是否正确？我还是一个新手程序员。我也不明白为什么 format.html 和 format.js 的顺序在这种情况下很重要。

我愿意接受所有建议，并对问题有更多的了解。

站点地图生成器无法生成站点地图

Sitemap generators unable to generate sitemap

ruby-on-rails

dreamhost

xml-sitemap