Perl REST::Client - 响应中的垃圾数据

Question

我在从 Perl REST::Client 获得针对 Red Hat Satellite REST API 的有效响应时遇到问题。我收到以下编码响应：

$VAR1 = '���j�0��~
�g9��   ���#�9�`dIm�m�-���uJ
        �����f4U�@▒��
                     ���F��xګ X�;�\'r��/���3R�s�C�u�*�2_N��٧�������f\�������WA0����نp��T͖�l�▒Pȣ}�x��8�&�d�n��ߦ`��.���Tƙ�V�c�&����a���%�ZH·�aJ�0�yT��q� �Jz��ճMO�\�����'

当我通过 Encode::decode_utf8 解码时，它同样出现乱码：

$VAR1 = "\x{fffd\x{fffd}\x{fffd}\x{fffd}j\x{fffd}0\x{fffd}\x{fffd}~
\x{fffd}g9\x{fffd}\x{fffd}      \x{fffd}\x{fffd}\x{fffd}#\x{fffd}9\x{fffd}`dIm\x{fffd}m\x{fffd}-\x{fffd}\x{fffd}\x{fffd}uJ
        \x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{fffd}f4U\x{fffd}\@▒\x{fffd}\x{1a0220}
                                                                                \x{fffd}\x{fffd}\x{fffd}F\x{fffd}\x{fffd}x\x{6ab} X\x{fffd};\x{fffd}'r\x{fffd}\x{fffd}/\x{fffd}\x{fffd}\x{fffd}3R\x{fffd}s\x{fffd}C\x{fffd}u\x{fffd}*\x{fffd}2_N\x{fffd}\x{fffd}\x{667}\x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{fffd}f\\x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{fffd}WA0\x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{646}p\x{fffd}\x{fffd}T\x{356}\x{fffd}l\x{fffd}▒P\x{223}}\x{fffd}x\x{fffd}\x{fffd}8\x{fffd}&\x{fffd}d\x{fffd}n\x{fffd}\x{fffd}\x{7e6}`\x{fffd}\x{fffd}.\x{fffd}\x{fffd}\x{fffd}T\x{199}\x{fffd}V\x{fffd}c\x{fffd}&\x{fffd}\x{fffd}\x{fffd}\x{fffd}a\x{fffd}\x{fffd}\x{fffd}%\x{fffd}ZH\x{b7}\x{fffd}aJ\x{fffd}0\x{fffd}yT\x{fffd}\x{fffd}q\x{fffd} \x{fffd}Jz\x{fffd}\x{fffd}\x{573}MO\x{fffd}\\x{fffd}\x{fffd}\x{fffd}\x{fffd}\x{fffd}";

我用来测试的脚本：

#!/usr/bin/perl

use strict;
use warnings;

use 5.010;
use REST::Client;
use Data::Dumper;

require JSON;
require MIME::Base64;
require HTTP::Cookies;

my $host        = 'https://sat.example.com/api/v2/domains';
my $client      = REST::Client->new({useragent => LWP::UserAgent->new(cookie_jar => HTTP::Cookies->new)});
my $headers     = {
        'Authorization'         => 'Basic XXXXXXXXX',
        'Accept-Encoding'       => scalar HTTP::Message::decodable,
        #'Content-Type'         => 'application/json',
        'Content-Type'          => 'application/json;charset=utf8',
        'Connection'            => 'keep-alive',
        'Accept'                => 'application/json',
        'Host'                  => URI->new($host)->canonical->host_port,
        #'Charset'              => 'UTF-8'
};

$client->addHeader($_, $headers->{$_}) for keys %{$headers};
$client->setCa('/var/www/html/pub/katello-server-ca.crt');

print Dumper JSON::decode_json($client->GET($host)->responseContent);

#print Dumper $client->GET($host);
__END__

当我通过 Data::Dumper 查看 REST::Client object 时，所有 headers 似乎都设置正确且可读。唯一不可读的是实际内容。

我可以使用来自同一服务器的 curl 命令查询 API：

$ curl -\# -X GET -v -H 'Accept: application/json' -H 'Authorization: Basic XXXXXXXXX' --cacert katello-server-ca.crt https://sat.example.com/api/v2/domains -o /dev/null 
* About to connect() to sat.example.com port 443 (#0)
*   Trying 192.168.0.1...
* Connected to sat.example.com (192.168.0.1) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: katello-server-ca.crt
  CApath: none
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=sat.example.com,OU=SomeOrgUnit,O=Katello,ST=North Carolina,C=US
*       start date: Oct 11 20:24:12 2018 GMT
*       expire date: Jan 17 20:24:12 2038 GMT
*       common name: sat.example.com
*       issuer: CN=sat.example.com,OU=SomeOrgUnit,O=Katello,L=Raleigh,ST=North Carolina,C=US
> GET /api/v2/domains HTTP/1.1
> User-Agent: curl/7.29.0
> Host: sat.example.com
> Accept: application/json
> Authorization: Basic XXXXXXXXX
> 
< HTTP/1.1 200 OK
< Date: Wed, 24 Apr 2019 18:50:34 GMT
< Server: Apache/2.4.6 (Red Hat Enterprise Linux)
< Foreman_version: 1.15.6.48
< Foreman_api_version: 2
< Apipie-Checksum: 2a54cbc5a3f59fad6e7e697ec609cda8
< Cache-Control: max-age=0, private, must-revalidate
< X-Request-Id: 57672b22-6a4f-4b3a-83fb-6b1a54747d67
< X-Runtime: 0.036638
< Content-Security-Policy: default-src 'self'; child-src 'self'; connect-src 'self' ws: wss:; img-src 'self' data: *.gravatar.com; script-src 'unsafe-eval' 'unsafe-inline' 'self'; style-src 'unsafe-inline' 'self'
< Strict-Transport-Security: max-age=631152000; includeSubdomains
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Frame-Options: sameorigin
< X-Permitted-Cross-Domain-Policies: none
< X-XSS-Protection: 1; mode=block
< X-Powered-By: Phusion Passenger 4.0.18
< Set-Cookie: _session_id=d35c9df0bce80baeca85b0ad38298ee8; path=/; secure; HttpOnly
< ETag: W/"0eeb7a2c131e780c72202cf712c972e3"
< Status: 200 OK
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: application/json; charset=utf-8
< 
{ [data not shown]
######################################################################## 100.0%* Connection #0 to host sat.example.com left intact

我已经使用 REST::Client 来对抗可能不同的 API，没有任何问题，这是第一个真正让我感到悲伤的 API。解码响应后，我不确定从哪里开始进行故障排除。任何建议都会有所帮助。

print Dumper $client->GET($host)->{_res}->as_string;

的输出

$VAR1 = 'HTTP/1.1 200 OK
Cache-Control: max-age=0, private, must-revalidate
Connection: close
Date: Thu, 25 Apr 2019 19:45:48 GMT
ETag: W/"0eeb7a2c131e780c72202cf712c972e3-gzip"
Server: Apache/2.4.6 (Red Hat Enterprise Linux)
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 242
Content-Type: application/json; charset=utf-8
Apipie-Checksum: 2a54cbc5a3f59fad6e7e697ec609cda8
Client-Date: Thu, 25 Apr 2019 19:45:53 GMT
Client-Peer: 192.168.0.1:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/ST=North Carolina/L=Raleigh/O=Katello/OU=SomeOrgUnit/CN=sat.example.com
Client-SSL-Cert-Subject: /C=US/ST=North Carolina/O=Katello/OU=SomeOrgUnit/CN=sat.example.com
Client-SSL-Cipher: ECDHE-RSA-AES128-GCM-SHA256
Client-SSL-Socket-Class: IO::Socket::SSL
Content-Security-Policy: default-src \'self\'; child-src \'self\'; connect-src \'self\' ws: wss:; img-src \'self\' data: *.gravatar.com; script-src \'unsafe-eval\' \'unsafe-inline\' \'self\'; style-src \'unsafe-inline\' \'self\'
Foreman_api_version: 2
Foreman_version: 1.15.6.48
Set-Cookie: _session_id=ea51c42b7a65478d27b21c2fb02e4c4a; path=/; secure; HttpOnly
Status: 200 OK
Strict-Transport-Security: max-age=631152000; includeSubdomains
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: sameorigin
X-Permitted-Cross-Domain-Policies: none
X-Powered-By: Phusion Passenger 4.0.18
X-Request-Id: 528535f3-c8cf-4d8b-91db-1572e94a974a
X-Runtime: 4.600748
X-XSS-Protection: 1; mode=block

???j?0??~
?g9??   ???#?9?`dIm?m?-???uJ
    ???f4U?@?????
                     ???F??xګ X?;?\'r??/??3R?s?C?u?*?2_N??٧??????f\???????WA0????نp??T͖?l?Pȣ}?x??8?&?d?n??ߦ`??.???Tƙ?V?c?&????a???%?ZH·?aJ?0?yT?q? ?Jz??ճMO?\?????
';

Answer 1

根据你的问题所附的评论，我们可以说问题是服务器正在响应 gzip 压缩数据，而 REST::Client 包只是 returns 原始数据而不对其进行解码第一的。您可以在 REST::Client 对象上使用 $client->{_res} 直接访问基础 HTTP::Response 对象，并且可以使用 ->decoded_content 方法获取正确解压缩和字符集转换的数据。请注意，您需要在解码内容上使用 JSON::from_json() 而不是 JSON::decode_json()，因为作为解码的一部分，它已经从 UTF-8 字节序列转换为 Unicode 字符串。

总之，那么，你要的就是下面的

print Dumper JSON::from_json($client->GET($host)->{_res}->decoded_content);

直接访问底层对象并不漂亮，但除非您想修补 REST::Client，否则这是获得所需结果的最简单方法。

Perl REST::Client - 响应中的垃圾数据

Perl REST::Client - Garbage data in response

rest

perl

theforeman