RETS gem GetObject 请求不一致地失败
RETS gem GetObject request failing inconsistently
我有一个很长的 运行ning rake 任务，每晚更新 MLS 数据。该任务首先更新listing数据，然后在所有listing更新后，通过Rails 5.2 activestorage加载图片。列表在 2-4 小时内更新得很好，但图片加载将 运行 持续 10 到 30 分钟，然后失败并出现相同的 Rets::InvalidIdentifier: Got error code 20402 (Invalid Identifier) 错误。
此错误来自 MLS Rets 服务器。
我正在异步使用 estately/rets gem and using Suckerpunch 到 运行 任务。导致错误的代码：
line 156 puts 'Now save the pictures'
157
158 ids.each do |id|
159 p = Listing.find(id)
160 mlsid = p.mlsid
161 photos = client.objects '*', {
162 resource: 'Property',
object_type: 'Photo',
resource_id: mlsid
}
if photos.count < 1
next
end
photos.each_with_index do |photo, index|
p.images.attach(io: StringIO.new(photo.body), filename: "#{p.mlsid}-#{index}.jpg")
end
puts "#{photos.count} images attached to listing_id = #{p.id}"
end # pictures ids.each do
client.logout
以及下面的完整错误跟踪：
I, [2018-06-07T06:26:53.943065 #4] INFO -- : S3 Storage (36.8ms) Uploaded file to key: XWoiiTfSh9LR25zKA8y4tpc8 (checksum: oybNBOCMwYTKqaTO5j5Nuw==)
D, [2018-06-07T06:26:53.944271 #4] DEBUG -- : (1.0ms) BEGIN
D, [2018-06-07T06:26:53.946751 #4] DEBUG -- : ActiveStorage::Blob Create (1.4ms) INSERT INTO "active_storage_blobs" ("key", "filename", "content_type", "metadata", "byte_size", "checksum", "created_at") VALUES (, , , , , , ) RETURNING "id" [["key", "XWoiiTfSh9LR25zKA8y4tpc8"], ["filename", "09973750-21.jpg"], ["content_type", "image/jpeg"], ["metadata", "{\"identified\":true}"], ["byte_size", 35097], ["checksum", "oybNBOCMwYTKqaTO5j5Nuw=="], ["created_at", "2018-06-07 06:26:53.944514"]]
D, [2018-06-07T06:26:53.950865 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] (1.7ms) BEGIN
D, [2018-06-07T06:26:53.952073 #4] DEBUG -- : (4.7ms) COMMIT
D, [2018-06-07T06:26:53.953379 #4] DEBUG -- : (1.0ms) BEGIN
D, [2018-06-07T06:26:53.953591 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] ActiveStorage::Blob Update (1.6ms) UPDATE "active_storage_blobs" SET "metadata" = WHERE "active_storage_blobs"."id" = [["metadata", "{\"identified\":true,\"width\":180,\"height\":225,\"analyzed\":true}"], ["id", 25654]]
D, [2018-06-07T06:26:53.956386 #4] DEBUG -- : ActiveStorage::Attachment Create (0.9ms) INSERT INTO "active_storage_attachments" ("name", "record_type", "record_id", "blob_id", "created_at") VALUES (, , , , ) RETURNING "id" [["name", "images"], ["record_type", "Listing"], ["record_id", 441544], ["blob_id", 25655], ["created_at", "2018-06-07 06:26:53.954865"]]
D, [2018-06-07T06:26:53.958520 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] (2.7ms) COMMIT
D, [2018-06-07T06:26:53.958787 #4] DEBUG -- : Listing Update (1.3ms) UPDATE "listings" SET "updated_at" = WHERE "listings"."id" = [["updated_at", "2018-06-07 06:26:53.956749"], ["id", 441544]]
I, [2018-06-07T06:26:53.959030 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] Performed ActiveStorage::AnalyzeJob (Job ID: e18de7a4-86bb-4075-844d-5466801613d0) from Async(default) in 47.95ms
D, [2018-06-07T06:26:53.960650 #4] DEBUG -- : (1.4ms) COMMIT
I, [2018-06-07T06:26:53.961439 #4] INFO -- : [ActiveJob] Enqueued ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) to Async(default) with arguments: #<GlobalID:0x0000000003645a20 @uri=#<URI::GID gid://schoolsparrow/ActiveStorage::Blob/25655>>
22 images attached to listing_id = 441544
D, [2018-06-07T06:26:53.964406 #4] DEBUG -- : Listing Load (1.4ms) SELECT "listings".* FROM "listings" WHERE "listings"."id" = LIMIT [["id", 441545], ["LIMIT", 1]]
D, [2018-06-07T06:26:53.965427 #4] DEBUG -- : ActiveStorage::Blob Load (1.1ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = LIMIT [["id", 25655], ["LIMIT", 1]]
I, [2018-06-07T06:26:53.966994 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] Performing ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) from Async(default) with arguments: #<GlobalID:0x00000000035480a0 @uri=#<URI::GID gid://schoolsparrow/ActiveStorage::Blob/25655>>
D, [2018-06-07T06:26:54.003064 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] (0.9ms) BEGIN
D, [2018-06-07T06:26:54.005782 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] ActiveStorage::Blob Update (1.3ms) UPDATE "active_storage_blobs" SET "metadata" = WHERE "active_storage_blobs"."id" = [["metadata", "{\"identified\":true,\"width\":202,\"height\":225,\"analyzed\":true}"], ["id", 25655]]
D, [2018-06-07T06:26:54.012433 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] (6.0ms) COMMIT
I, [2018-06-07T06:26:54.012766 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] Performed ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) from Async(default) in 45.53ms
rake aborted!
Rets::InvalidIdentifier: Got error code 20402 (Invalid Identifier)
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/error_checker.rb:33:in `check'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/multipart.rb:35:in `check_for_invalids_parts!'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/multipart.rb:29:in `parse'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/client.rb:216:in `create_parts_from_response'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/client.rb:203:in `objects'
/app/app/jobs/mred_job.rb:162:in `block in perform'
/app/app/jobs/mred_job.rb:159:in `each'
/app/app/jobs/mred_job.rb:159:in `perform'
/app/lib/tasks/update.rake:10:in `update_mred'
/app/lib/tasks/update.rake:6:in `block (2 levels) in <top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:74:in `load'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:74:in `kernel_load'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:27:in `run'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:360:in `exec'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor.rb:369:in `dispatch'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:20:in `dispatch'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/base.rb:444:in `start'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:10:in `start'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/exe/bundle:30:in `block in <top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/exe/bundle:22:in `<top (required)>'
/app/bin/bundle:3:in `load'
/app/bin/bundle:3:in `<main>'
Tasks: TOP => update:mred
有没有人知道为什么这可以工作一段时间，然后因同样的错误而崩溃？任何想法将不胜感激！
尝试更改请求 ID。 MLS 的列表 ID 和列表键字段都将可用。取决于 MLS，它可能需要任何一个来提取图像。
请查看此 document。它说，
If the server does not support UID for the requested type of objects and the client submits a UID instead of the ID, the server MUST respond with
an error. The preferred error code is 20403: No Object Found. Servers that do not implement the PostObject functionality (Section 13) MAY
respond with a 20402: Invalid Identifier. If the requested type of object has an ObjectData class linked in the metadata, the server MUST support
this argument
希望这会有所帮助。
我最终通过在 RETS 查询周围添加错误救援解决了这个问题，如下所示：
begin
photos = client.objects '*', {
resource: 'Property',
object_type: 'Photo',
resource_id: mlsid
}
rescue => e
puts 'Error: ' + e.message
next
end
我有一个很长的 运行ning rake 任务，每晚更新 MLS 数据。该任务首先更新listing数据，然后在所有listing更新后，通过Rails 5.2 activestorage加载图片。列表在 2-4 小时内更新得很好，但图片加载将 运行 持续 10 到 30 分钟，然后失败并出现相同的 Rets::InvalidIdentifier: Got error code 20402 (Invalid Identifier) 错误。
此错误来自 MLS Rets 服务器。
我正在异步使用 estately/rets gem and using Suckerpunch 到 运行 任务。导致错误的代码：
line 156 puts 'Now save the pictures'
157
158 ids.each do |id|
159 p = Listing.find(id)
160 mlsid = p.mlsid
161 photos = client.objects '*', {
162 resource: 'Property',
object_type: 'Photo',
resource_id: mlsid
}
if photos.count < 1
next
end
photos.each_with_index do |photo, index|
p.images.attach(io: StringIO.new(photo.body), filename: "#{p.mlsid}-#{index}.jpg")
end
puts "#{photos.count} images attached to listing_id = #{p.id}"
end # pictures ids.each do
client.logout
以及下面的完整错误跟踪：
I, [2018-06-07T06:26:53.943065 #4] INFO -- : S3 Storage (36.8ms) Uploaded file to key: XWoiiTfSh9LR25zKA8y4tpc8 (checksum: oybNBOCMwYTKqaTO5j5Nuw==)
D, [2018-06-07T06:26:53.944271 #4] DEBUG -- : (1.0ms) BEGIN
D, [2018-06-07T06:26:53.946751 #4] DEBUG -- : ActiveStorage::Blob Create (1.4ms) INSERT INTO "active_storage_blobs" ("key", "filename", "content_type", "metadata", "byte_size", "checksum", "created_at") VALUES (, , , , , , ) RETURNING "id" [["key", "XWoiiTfSh9LR25zKA8y4tpc8"], ["filename", "09973750-21.jpg"], ["content_type", "image/jpeg"], ["metadata", "{\"identified\":true}"], ["byte_size", 35097], ["checksum", "oybNBOCMwYTKqaTO5j5Nuw=="], ["created_at", "2018-06-07 06:26:53.944514"]]
D, [2018-06-07T06:26:53.950865 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] (1.7ms) BEGIN
D, [2018-06-07T06:26:53.952073 #4] DEBUG -- : (4.7ms) COMMIT
D, [2018-06-07T06:26:53.953379 #4] DEBUG -- : (1.0ms) BEGIN
D, [2018-06-07T06:26:53.953591 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] ActiveStorage::Blob Update (1.6ms) UPDATE "active_storage_blobs" SET "metadata" = WHERE "active_storage_blobs"."id" = [["metadata", "{\"identified\":true,\"width\":180,\"height\":225,\"analyzed\":true}"], ["id", 25654]]
D, [2018-06-07T06:26:53.956386 #4] DEBUG -- : ActiveStorage::Attachment Create (0.9ms) INSERT INTO "active_storage_attachments" ("name", "record_type", "record_id", "blob_id", "created_at") VALUES (, , , , ) RETURNING "id" [["name", "images"], ["record_type", "Listing"], ["record_id", 441544], ["blob_id", 25655], ["created_at", "2018-06-07 06:26:53.954865"]]
D, [2018-06-07T06:26:53.958520 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] (2.7ms) COMMIT
D, [2018-06-07T06:26:53.958787 #4] DEBUG -- : Listing Update (1.3ms) UPDATE "listings" SET "updated_at" = WHERE "listings"."id" = [["updated_at", "2018-06-07 06:26:53.956749"], ["id", 441544]]
I, [2018-06-07T06:26:53.959030 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [e18de7a4-86bb-4075-844d-5466801613d0] Performed ActiveStorage::AnalyzeJob (Job ID: e18de7a4-86bb-4075-844d-5466801613d0) from Async(default) in 47.95ms
D, [2018-06-07T06:26:53.960650 #4] DEBUG -- : (1.4ms) COMMIT
I, [2018-06-07T06:26:53.961439 #4] INFO -- : [ActiveJob] Enqueued ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) to Async(default) with arguments: #<GlobalID:0x0000000003645a20 @uri=#<URI::GID gid://schoolsparrow/ActiveStorage::Blob/25655>>
22 images attached to listing_id = 441544
D, [2018-06-07T06:26:53.964406 #4] DEBUG -- : Listing Load (1.4ms) SELECT "listings".* FROM "listings" WHERE "listings"."id" = LIMIT [["id", 441545], ["LIMIT", 1]]
D, [2018-06-07T06:26:53.965427 #4] DEBUG -- : ActiveStorage::Blob Load (1.1ms) SELECT "active_storage_blobs".* FROM "active_storage_blobs" WHERE "active_storage_blobs"."id" = LIMIT [["id", 25655], ["LIMIT", 1]]
I, [2018-06-07T06:26:53.966994 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] Performing ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) from Async(default) with arguments: #<GlobalID:0x00000000035480a0 @uri=#<URI::GID gid://schoolsparrow/ActiveStorage::Blob/25655>>
D, [2018-06-07T06:26:54.003064 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] (0.9ms) BEGIN
D, [2018-06-07T06:26:54.005782 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] ActiveStorage::Blob Update (1.3ms) UPDATE "active_storage_blobs" SET "metadata" = WHERE "active_storage_blobs"."id" = [["metadata", "{\"identified\":true,\"width\":202,\"height\":225,\"analyzed\":true}"], ["id", 25655]]
D, [2018-06-07T06:26:54.012433 #4] DEBUG -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] (6.0ms) COMMIT
I, [2018-06-07T06:26:54.012766 #4] INFO -- : [ActiveJob] [ActiveStorage::AnalyzeJob] [a9027f11-5ec0-4f0a-96f7-ae999a373395] Performed ActiveStorage::AnalyzeJob (Job ID: a9027f11-5ec0-4f0a-96f7-ae999a373395) from Async(default) in 45.53ms
rake aborted!
Rets::InvalidIdentifier: Got error code 20402 (Invalid Identifier)
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/error_checker.rb:33:in `check'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/multipart.rb:35:in `check_for_invalids_parts!'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/parser/multipart.rb:29:in `parse'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/client.rb:216:in `create_parts_from_response'
/app/vendor/bundle/ruby/2.3.0/gems/rets-0.11.0/lib/rets/client.rb:203:in `objects'
/app/app/jobs/mred_job.rb:162:in `block in perform'
/app/app/jobs/mred_job.rb:159:in `each'
/app/app/jobs/mred_job.rb:159:in `perform'
/app/lib/tasks/update.rake:10:in `update_mred'
/app/lib/tasks/update.rake:6:in `block (2 levels) in <top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:74:in `load'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:74:in `kernel_load'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli/exec.rb:27:in `run'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:360:in `exec'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor.rb:369:in `dispatch'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:20:in `dispatch'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/vendor/thor/lib/thor/base.rb:444:in `start'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/cli.rb:10:in `start'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/exe/bundle:30:in `block in <top (required)>'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/lib/bundler/friendly_errors.rb:121:in `with_friendly_errors'
/app/vendor/bundle/ruby/2.3.0/gems/bundler-1.15.2/exe/bundle:22:in `<top (required)>'
/app/bin/bundle:3:in `load'
/app/bin/bundle:3:in `<main>'
Tasks: TOP => update:mred
有没有人知道为什么这可以工作一段时间，然后因同样的错误而崩溃？任何想法将不胜感激！
尝试更改请求 ID。 MLS 的列表 ID 和列表键字段都将可用。取决于 MLS，它可能需要任何一个来提取图像。
请查看此 document。它说，
If the server does not support UID for the requested type of objects and the client submits a UID instead of the ID, the server MUST respond with an error. The preferred error code is 20403: No Object Found. Servers that do not implement the PostObject functionality (Section 13) MAY respond with a 20402: Invalid Identifier. If the requested type of object has an ObjectData class linked in the metadata, the server MUST support this argument
希望这会有所帮助。
我最终通过在 RETS 查询周围添加错误救援解决了这个问题，如下所示：
begin
photos = client.objects '*', {
resource: 'Property',
object_type: 'Photo',
resource_id: mlsid
}
rescue => e
puts 'Error: ' + e.message
next
end