使用 C# 将数据传输到 aws s3 snowball

Question

我正在尝试使用滚雪球将所有数据导入 s3，但我需要跟踪每个文件的一些元数据。我有一些 c# 代码可以使用此元数据将文件直接上传到存储桶，工作正常。我想我可以使用雪球 s3 适配器连接到我的雪球并从我的 c# 代码上传数据，因为 this link 说：

You can use the S3 Adapter with existing Amazon S3 interfaces like the AWS SDKs, the AWS CLI, or your own custom Amazon S3 REST client.

我在 http://localhost:8090 上有适配器运行，所以我尝试通过传入一个 AmazonS3Config 对象并将该端点作为 ServiceUrl 来将其设置为 Amazon3Client 的端点，但是当我尝试使用 PutObjectRequest 或 TransferUtility 上传文件时，出现以下异常：

 Amazon.Runtime.AmazonServiceException: A WebException with status
 NameResolutionFailure was thrown. ---> System.Net.WebException: The
 remote name could not be resolved: 'BUCKET.localhost' at
 System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
 at System.Net.HttpWebRequest.GetRequestStream() at
 Amazon.Runtime.Internal.HttpRequest.GetRequestContent() at
 Amazon.Runtime.Internal.HttpHandler`1.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.RedirectHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.Unmarshaller.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.ErrorHandler.InvokeSync(IExecutionContext
 executionContext)
 --- End of inner exception stack trace --- at Amazon.Runtime.Internal.WebExceptionHandler.HandleException(IExecutionContext
 executionContext, WebException exception) at
 Amazon.Runtime.Internal.ExceptionHandler`1.Handle(IExecutionContext
 executionContext, Exception exception) at
 Amazon.Runtime.Internal.ErrorHandler.ProcessException(IExecutionContext
 executionContext, Exception exception) at
 Amazon.Runtime.Internal.ErrorHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.Signer.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.CredentialsRetriever.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.RetryHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3KmsHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.EndpointResolver.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3PostMarshallHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.Marshaller.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3PreMarshallHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.CallbackHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.PipelineHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.MetricsHandler.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.Internal.RuntimePipeline.InvokeSync(IExecutionContext
 executionContext) at
 Amazon.Runtime.AmazonServiceClient.Invoke(TRequest request,
 IMarshaller`2 marshaller, ResponseUnmarshaller unmarshaller) at
 Amazon.S3.AmazonS3Client.PutObject(PutObjectRequest request)

我尝试使用我的本地 ip 地址以及 127.0.0.1，但都出现了相同的错误。当我使用 amazon cli 将文档上传到 snowball 时，使用 --endpoint 更改端点，它有效（使用本地主机或我的本地 ip），所以显然它只是关于 .net sdk 的东西。但我真的更愿意使用我现有的 c# 代码，而不是编写一个命令行程序来执行此操作 - 从 cli 不会那么简单。

编辑：这是不起作用的 C# 代码：

var config = new AmazonS3Config
{
  ServiceURL = "http://localhost:8090"
};

var file = new FileInfo(FILENAME);

var pRequest = new PutObjectRequest
{
  BucketName = BUCKET,
  Key = S3FILENAME,
  StorageClass = S3StorageClass.StandardInfrequentAccess,
  FilePath = file.FullName
};

pRequest.Metadata.Add("x-amz-meta-local-date-modified", file.CreationTime.ToFileTimeUtc().ToString());

using (var client = new AmazonS3Client(config))
{
  client.PutObject(pRequest);
}

这是有效的命令行语句：

aws s3 cp FILENAME s3://BUCKET/S3FILENAME --endpoint http://localhost:8090

谁能帮帮我？

谢谢

Answer 1

问题是，默认情况下，AWS SDK 会将您的 BucketName 添加到服务 URL 之前，因此您最终会得到一个无效的 URL：http://BUCKET.localhost:8090.

您需要使用 ForcePathStyle 参数，以防止 BucketName 被添加到服务中URL:

var config = new AmazonS3Config
{
    ServiceURL = "http://localhost:8090",
    ForcePathStyle = true
};

使用 C# 将数据传输到 aws s3 snowball

Transfer data onto aws s3 snowball using c#

c#

amazon-web-services

amazon-s3

aws-sdk

aws-snowball