s3n:// 处的主机名 URI 无效
Invalid HostName URI at s3n://
我在尝试从 s3 访问对象时遇到此错误 bucket.I 可以访问 s3 内顶层的对象 bucket.But 当我尝试访问文件夹内的文件时它不起作用.
Bucket/file.xml（有效）但 Bucket/folder/file.xml（无效）输出无效的主机名 uri 异常。
17:28:20,652 DEBUG HttpConnection:691 - Open connection to output-4ssdgf7c9ae-838df-40se1-a105-7c8srfgsg6db166.s3.amazonaws.com:443
17:28:20,949 DEBUG header:69 - >> "GET /%2Foutput HTTP/1.1[\r][\n]"
17:28:20,949 DEBUG HttpMethodBase:1235 - Adding Host request header
17:28:20,956 DEBUG header:69 - >> "Content-Type: [\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Date: Mon, 29 Aug 2016 11:58:20 GMT[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Authorization: AWS AsdggPZF42ULNsdsd:hJLLzNsxcvq44gNbbbndestXzL+XvP8cw=[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "User-Agent: JetS3t/0.6.1 (Linux/3.16.0-77-generic; amd64; en; JVM 1.8.0_101)[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Host: output-4e5sfdg9ae-8zsdf-40s1-a105-7c8easfddb166.s3.amazonaws.com[\r][\n]"
17:28:20,958 DEBUG header:69 - >> "[\r][\n]"
17:28:21,798 DEBUG header:69 - << "HTTP/1.1 404 Not Found[\r][\n]"
17:28:21,800 DEBUG header:69 - << "x-amz-request-id: F280A20CBDGD76D0[\r][\n]"
17:28:21,800 DEBUG header:69 - << "x-amz-id-2: Z8Pq68OsdDEnhcEvEtrhlxkQ9b+bXYAbIxxssamS6+8vsbD9LSDFpzxb0KDFFvXA4[\r][\n]"
17:28:21,800 DEBUG header:69 - << "Content-Type: application/xml[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Transfer-Encoding: chunked[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Date: Mon, 29 Aug 2016 11:58:20 GMT[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Server: AmazonS3[\r][\n]"
17:28:21,803 DEBUG HttpConnection:905 - Input data available
17:28:21,805 DEBUG RestS3Service:385 - Response for '/%2Foutput'. Content-Type: application/xml, Headers: [x-amz-request-id: F280A2SDF70D76D0
, x-amz-id-2: Z8Pq68OgBSDFGhcEvEtrhlxkQ9b+bXYAbIxRAmdddamS6+8vsbD9Ltdmxs83KAkvXA4
, Content-Type: application/xml
, Transfer-Encoding: chunked
, Date: Mon, 29 Aug 2016 11:58:20 GMT
, Server: AmazonS3
]
17:28:21,805 WARN RestS3Service:393 - Response '/%2Foutput' - Unexpected response code 404, expected 200
17:28:21,806 WARN RestS3Service:402 - Response '/%2Foutput' - Received error response with XML message
17:28:21,808 DEBUG content:83 - << "1"
17:28:21,808 DEBUG content:83 - << "0"
17:28:21,808 DEBUG content:83 - << "a"
17:28:21,808 DEBUG content:83 - << "[\r]"
17:28:21,809 DEBUG content:69 - << "[\n]"
17:28:21,809 DEBUG content:69 - << "<?xml version="1.0" encoding="UTF-8"?>[\n]"
17:28:21,809 DEBUG content:83 - << "<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>/output</Key><RequestId>F280A20DGG076D0</RequestId><HostId>Z8Pq68OgSD23hcEvEtrhlxkQ9b+bXYAERTY54RamS6+8vsbD9LwDFE342AkvXA4</HostId></Error>"
17:28:21,810 DEBUG content:83 - << "[\r]"
17:28:21,810 DEBUG content:69 - << "[\n]"
17:28:21,810 DEBUG content:83 - << "0"
17:28:21,810 DEBUG content:83 - << "[\r]"
17:28:21,811 DEBUG content:69 - << "[\n]"
17:28:21,811 DEBUG content:83 - << "[\r]"
17:28:21,811 DEBUG content:69 - << "[\n]"
17:28:21,811 DEBUG HttpMethodBase:921 - Resorting to protocol version default close connection policy
17:28:21,812 DEBUG HttpMethodBase:925 - Should NOT close connection, using HTTP/1.1
17:28:21,812 DEBUG HttpConnection:1177 - Releasing connection back to connection manager.
17:28:21,812 DEBUG MultiThreadedHttpConnectionManager:943 - Freeing connection, hostConfig=HostConfiguration[host=https://output-4e57c9ae-838f-40e1-a105-7c8ed86db166.s3.amazonaws.com]
17:28:21,813 DEBUG IdleConnectionHandler:75 - Adding connection at: 1472471901813
17:28:21,813 DEBUG MultiThreadedHttpConnectionManager:926 - Notifying no-one, there are no waiting threads
17:28:21,813 DEBUG HttpMethodReleaseInputStream:142 - Released HttpMethod as its response data stream is fully consumed
17:28:21,816 DEBUG RestS3Service:516 - Releasing HttpClient connection after error: S3 GET failed for '/%2Foutput'
17:28:21,817 DEBUG JobClient:982 - Creating splits at file:/tmp/hadoop-java-dev/mapred/staging/java-dev930738962/.staging/job_local93034362_0001
17:28:21,825 WARN LoadSnappy:46 - Snappy native library not loaded
17:28:21,827 DEBUG FileSystem:1441 - Creating filesystem for s3://lukup_assets/live_channel_xmls/10%20tv.xml
17:28:21,827 INFO JobClient:1023 - Cleaning up the staging area file:/tmp/hadoop-java-dev/mapred/staging/java-dev930738962/.staging/job_local930743452_0001
java.lang.IllegalArgumentException: Invalid hostname in URI s3://asset_fdp/live_records/400%20tv.xml
******end job
我收到上述异常：
java.lang.IllegalArgumentException: Invalid hostname in URI s3://asset_fdp/live_records/400%20tv.xml
上面'asset_fdp'是我的桶名，'live records'是桶里面的文件夹。
我猜这可能是您密钥的权限问题？
也许您可以访问 bucket_name，但不能访问 bucket_name/* ?
检查这个：
I need an Amazon S3 user with full access to a single bucket
但更重要的是，查看您的调试日志，我发现有些东西看起来有点像您的 AWS 私有密钥。我希望不是这样？如果是，您应该将其从 public 论坛中删除...
我在尝试从 s3 访问对象时遇到此错误 bucket.I 可以访问 s3 内顶层的对象 bucket.But 当我尝试访问文件夹内的文件时它不起作用. Bucket/file.xml（有效）但 Bucket/folder/file.xml（无效）输出无效的主机名 uri 异常。
17:28:20,652 DEBUG HttpConnection:691 - Open connection to output-4ssdgf7c9ae-838df-40se1-a105-7c8srfgsg6db166.s3.amazonaws.com:443
17:28:20,949 DEBUG header:69 - >> "GET /%2Foutput HTTP/1.1[\r][\n]"
17:28:20,949 DEBUG HttpMethodBase:1235 - Adding Host request header
17:28:20,956 DEBUG header:69 - >> "Content-Type: [\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Date: Mon, 29 Aug 2016 11:58:20 GMT[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Authorization: AWS AsdggPZF42ULNsdsd:hJLLzNsxcvq44gNbbbndestXzL+XvP8cw=[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "User-Agent: JetS3t/0.6.1 (Linux/3.16.0-77-generic; amd64; en; JVM 1.8.0_101)[\r][\n]"
17:28:20,957 DEBUG header:69 - >> "Host: output-4e5sfdg9ae-8zsdf-40s1-a105-7c8easfddb166.s3.amazonaws.com[\r][\n]"
17:28:20,958 DEBUG header:69 - >> "[\r][\n]"
17:28:21,798 DEBUG header:69 - << "HTTP/1.1 404 Not Found[\r][\n]"
17:28:21,800 DEBUG header:69 - << "x-amz-request-id: F280A20CBDGD76D0[\r][\n]"
17:28:21,800 DEBUG header:69 - << "x-amz-id-2: Z8Pq68OsdDEnhcEvEtrhlxkQ9b+bXYAbIxxssamS6+8vsbD9LSDFpzxb0KDFFvXA4[\r][\n]"
17:28:21,800 DEBUG header:69 - << "Content-Type: application/xml[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Transfer-Encoding: chunked[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Date: Mon, 29 Aug 2016 11:58:20 GMT[\r][\n]"
17:28:21,801 DEBUG header:69 - << "Server: AmazonS3[\r][\n]"
17:28:21,803 DEBUG HttpConnection:905 - Input data available
17:28:21,805 DEBUG RestS3Service:385 - Response for '/%2Foutput'. Content-Type: application/xml, Headers: [x-amz-request-id: F280A2SDF70D76D0
, x-amz-id-2: Z8Pq68OgBSDFGhcEvEtrhlxkQ9b+bXYAbIxRAmdddamS6+8vsbD9Ltdmxs83KAkvXA4
, Content-Type: application/xml
, Transfer-Encoding: chunked
, Date: Mon, 29 Aug 2016 11:58:20 GMT
, Server: AmazonS3
]
17:28:21,805 WARN RestS3Service:393 - Response '/%2Foutput' - Unexpected response code 404, expected 200
17:28:21,806 WARN RestS3Service:402 - Response '/%2Foutput' - Received error response with XML message
17:28:21,808 DEBUG content:83 - << "1"
17:28:21,808 DEBUG content:83 - << "0"
17:28:21,808 DEBUG content:83 - << "a"
17:28:21,808 DEBUG content:83 - << "[\r]"
17:28:21,809 DEBUG content:69 - << "[\n]"
17:28:21,809 DEBUG content:69 - << "<?xml version="1.0" encoding="UTF-8"?>[\n]"
17:28:21,809 DEBUG content:83 - << "<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>/output</Key><RequestId>F280A20DGG076D0</RequestId><HostId>Z8Pq68OgSD23hcEvEtrhlxkQ9b+bXYAERTY54RamS6+8vsbD9LwDFE342AkvXA4</HostId></Error>"
17:28:21,810 DEBUG content:83 - << "[\r]"
17:28:21,810 DEBUG content:69 - << "[\n]"
17:28:21,810 DEBUG content:83 - << "0"
17:28:21,810 DEBUG content:83 - << "[\r]"
17:28:21,811 DEBUG content:69 - << "[\n]"
17:28:21,811 DEBUG content:83 - << "[\r]"
17:28:21,811 DEBUG content:69 - << "[\n]"
17:28:21,811 DEBUG HttpMethodBase:921 - Resorting to protocol version default close connection policy
17:28:21,812 DEBUG HttpMethodBase:925 - Should NOT close connection, using HTTP/1.1
17:28:21,812 DEBUG HttpConnection:1177 - Releasing connection back to connection manager.
17:28:21,812 DEBUG MultiThreadedHttpConnectionManager:943 - Freeing connection, hostConfig=HostConfiguration[host=https://output-4e57c9ae-838f-40e1-a105-7c8ed86db166.s3.amazonaws.com]
17:28:21,813 DEBUG IdleConnectionHandler:75 - Adding connection at: 1472471901813
17:28:21,813 DEBUG MultiThreadedHttpConnectionManager:926 - Notifying no-one, there are no waiting threads
17:28:21,813 DEBUG HttpMethodReleaseInputStream:142 - Released HttpMethod as its response data stream is fully consumed
17:28:21,816 DEBUG RestS3Service:516 - Releasing HttpClient connection after error: S3 GET failed for '/%2Foutput'
17:28:21,817 DEBUG JobClient:982 - Creating splits at file:/tmp/hadoop-java-dev/mapred/staging/java-dev930738962/.staging/job_local93034362_0001
17:28:21,825 WARN LoadSnappy:46 - Snappy native library not loaded
17:28:21,827 DEBUG FileSystem:1441 - Creating filesystem for s3://lukup_assets/live_channel_xmls/10%20tv.xml
17:28:21,827 INFO JobClient:1023 - Cleaning up the staging area file:/tmp/hadoop-java-dev/mapred/staging/java-dev930738962/.staging/job_local930743452_0001
java.lang.IllegalArgumentException: Invalid hostname in URI s3://asset_fdp/live_records/400%20tv.xml
******end job
我收到上述异常：
java.lang.IllegalArgumentException: Invalid hostname in URI s3://asset_fdp/live_records/400%20tv.xml
上面'asset_fdp'是我的桶名，'live records'是桶里面的文件夹。
我猜这可能是您密钥的权限问题？
也许您可以访问 bucket_name，但不能访问 bucket_name/* ?
检查这个：
I need an Amazon S3 user with full access to a single bucket
但更重要的是，查看您的调试日志，我发现有些东西看起来有点像您的 AWS 私有密钥。我希望不是这样？如果是，您应该将其从 public 论坛中删除...