使用 C# 代码从 Data Lake Store 读取文件内容

Question

我正在尝试从 Azure Data Lake Store 文件中读取内容。但是在连接/打开连接时，连接本身失败并抛出异常

 var stream = _adlsFileSystemClient.FileSystem.Open(_adlsAccountName, "/folder1/"+file.PathSuffix);

获取异常为：

Exception of type 'Microsoft.Rest.Azure.CloudException' was thrown.

无法识别问题。如何解决这个问题？

谢谢

Answer 1

Exception of type 'Microsoft.Rest.Azure.CloudException' was thrown.

这是基本的异常信息。所以请尝试使用 fiddler 工具来捕获详细的错误信息。

根据你的代码，好像是找不到文件或者授权异常。

1.If 是 404 错误：请尝试使用以下格式并从 Azure 门户检查文件。

 var srcPath = "/mytempdir/tomtest.txt";
 var stream = adlsFileSystemClient.FileSystem.Open(adlsAccountName, srcPath);

2.If 这是授权异常，如果您使用带有客户端密码的服务到服务身份验证。请为 azure AD 应用程序分配操作该文件的权限。更多详情请参考document.

以下是我的测试代码和截图：

 var applicationId = "your application Id";
 var secretKey = "secret Key";
 var tenantId = "Your tenantId";
 var adlsAccountName = "adls account name";
 var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, applicationId, secretKey).Result;
 var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds);
 var srcPath = "/mytempdir/tomtest.txt";
 var destPath = @"F:\tom\testfile.txt";
 using (var stream = adlsFileSystemClient.FileSystem.Open(adlsAccountName, srcPath))
 using (var fileStream = new FileStream(destPath, FileMode.Create))
 {
    stream.CopyTo(fileStream);
 }

Packages.config

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Hyak.Common" version="1.0.2" targetFramework="net452" />
  <package id="Microsoft.Azure.Common" version="2.1.0" targetFramework="net452" />
  <package id="Microsoft.Azure.Common.Dependencies" version="1.0.0" targetFramework="net452" />
  <package id="Microsoft.Azure.Management.DataLake.Store" version="1.0.4" targetFramework="net452" />
  <package id="Microsoft.Azure.Management.DataLake.StoreFileSystem" version="0.9.6-preview" targetFramework="net452" />
  <package id="Microsoft.Azure.Management.DataLake.StoreUploader" version="1.0.1-preview" targetFramework="net452" />
  <package id="Microsoft.Bcl" version="1.1.9" targetFramework="net452" />
  <package id="Microsoft.Bcl.Async" version="1.0.168" targetFramework="net452" />
  <package id="Microsoft.Bcl.Build" version="1.0.14" targetFramework="net452" />
  <package id="Microsoft.IdentityModel.Clients.ActiveDirectory" version="2.28.3" targetFramework="net452" />
  <package id="Microsoft.Net.Http" version="2.2.22" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime" version="2.3.5" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.5" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure.Authentication" version="2.2.12" targetFramework="net452" />
  <package id="Newtonsoft.Json" version="6.0.8" targetFramework="net452" />
</packages>

使用 C# 代码从 Data Lake Store 读取文件内容

Read content of file from Data Lake Store using C# code

c#

azure

azure-data-lake