ADLS 客户端批量下载 - 下载文件但没有数据

Question

我想从 Azure Data Lake Store 下载一个目录到我的本地目录，我正在使用这种方法 - Bulk Download。问题是，当我运行我的代码按预期下载文件时，但文件没有数据并且它们的大小也显示 0KB。

我是这样使用方法的-

string srcPath = "/souce path included/"; 
string dstPath = "C:/Users/keprasad/Documents/Views/"; 
clientProd.BulkDownload(srcPath, dstPath);

我非常有信心我已经使用正确的凭据正确创建了客户端，因为它不会抛出错误并实际下载文件。任何人都知道这种情况何时会发生以及需要采取什么措施来补救？

PS：实际上我在 Whosebug 中的第一个问题（对未来感到兴奋！）

Answer 1

如果要使用服务主体访问Azure数据湖gen1，我们需要为sp配置正确的访问控制。详情请参考here and here.

例如。我想下载文件 (/Seattle/Portland/Data.txt)。我们需要为 sp 配置 ACL，如下所示。

代码

下载单个文件

 string appId = "";
            string appSecret = "";
            string domain = "";

            var serviceSettings = ActiveDirectoryServiceSettings.Azure;
            serviceSettings.TokenAudience = new Uri(@"https://datalake.azure.net/");
            
            var creds = await ApplicationTokenProvider.LoginSilentAsync(domain, appId, appSecret, serviceSettings);

            string accountName = "";
            AdlsClient client = AdlsClient.CreateClient($"{accountName}.azuredatalakestore.net", creds);
            string fileName = "/Seattle/Portland/data.txt";
            string line = null;
            using (var readStream = new StreamReader(client.GetReadStream(fileName)))
            {
                while ((line =  await readStream.ReadLineAsync()) != null) {

                    Console.WriteLine(line);
                }
               
            }
            Console.ReadLine();

批量下载

string appId = "";
            string appSecret = "";
            string domain = "";

            var serviceSettings = ActiveDirectoryServiceSettings.Azure;
            serviceSettings.TokenAudience = new Uri(@"https://datalake.azure.net/");

            var creds = await ApplicationTokenProvider.LoginSilentAsync(domain, appId, appSecret, serviceSettings);

            string accountName = "";
            AdlsClient client = AdlsClient.CreateClient($"{accountName}.azuredatalakestore.net", creds);
            string srcPath = "/Seattle/Portland/";
            string desPath = @"D:\sample";
            TransferStatus status =client.BulkDownload(srcPath, desPath);

ADLS 客户端批量下载 - 下载文件但没有数据

ADLS Client Bulk Download - downloading files but no data

c#

asp.net

azure-data-lake