使用 SSIS 以 Parquet/CSV/JSON 格式将 SQL 服务器 Table/View 加载到 Azure Data Lake。灵活的文件目标任务不断抛出错误
Load SQL Server Table/View to Azure Data Lake in Parquet / CSV / JSON format using SSIS. Flexible File Destination task keeps throwing error
我正在使用 OLEDB 源从 SQL Server 2012 读取并使用灵活文件目标任务以 Parquet 格式将其保存到 Azure Data lake。但是当 azure flexible file destination 任务在数据流中失败时，我不断收到以下错误
错误信息：
[Flexible File Destination] Error: Transfer data error : System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Azure.Storage.Common, Version=11.1.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.Azure.Storage.Common, Version=11.1.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35'
at Microsoft.Azure.Storage.Data.ExceptionHandler.SetStorageExceptionContext(Exception ex, Dictionary2 exData) at Microsoft.Azure.Storage.Data.ExceptionHandler.HandleException(Exception ex, String fileSystem, String path) at Microsoft.Azure.Storage.Data.ExceptionHandler.ExecuteAction[TResult](Func1 func, String fileSystem, String path)
at Microsoft.Azure.Storage.Data.BlobFSClient.IsFileSystemExist(String filesystem, Boolean swallowForbiddenError, Boolean useListPathApi)
at Microsoft.Azure.Storage.Data.BlobFSClient.CreateFileSystemIfNotExist(String filesystem, Boolean swallowForbiddenError, Boolean useListPathApiToCheckFileSystemExist)
at Microsoft.DataTransfer.Runtime.BlobFSMultipartSink..ctor(AzureBlobFSConnection connection, AzureBlobFSWriteInternalSettings writeSettings)
at Microsoft.DataTransfer.Runtime.AzureBlobFSConnection.CreateSink(IDictionary2 properties) at Microsoft.DataTransfer.ClientLibrary.MultipartWriteSink..ctor(IConnectorPlugin connector, IDictionary2 connectionProperties, IDictionary2 writeProperties) at Microsoft.SqlServer.IntegrationService.ExtensibleFileCommon.DataTransferClientHelper.CreateBinaryDataSink(IDictionary2 sinkProperties, Guid transferId, Guid activityId, BinaryTelemetryCollector& collector)
at Microsoft.SqlServer.IntegrationService.ExtensibleFileComponents.Destination.ExtensibleFileDest.TransferDataToFile()
我已经为 SQL Server 2012 安装了集成 azure 功能包。请帮忙，谢谢。
您必须交叉检查您在灵活文件目标编辑器中提供的文件夹路径和文件名。尽管如此，如果这导致相同的错误，我建议您安装新版本的 Microsoft SQL Azure 服务器集成服务功能包。
安装最新版SSIS后问题已解决，请检查此类问题。
我正在使用 OLEDB 源从 SQL Server 2012 读取并使用灵活文件目标任务以 Parquet 格式将其保存到 Azure Data lake。但是当 azure flexible file destination 任务在数据流中失败时，我不断收到以下错误
错误信息：
[Flexible File Destination] Error: Transfer data error : System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Azure.Storage.Common, Version=11.1.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified. File name: 'Microsoft.Azure.Storage.Common, Version=11.1.3.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' at Microsoft.Azure.Storage.Data.ExceptionHandler.SetStorageExceptionContext(Exception ex, Dictionary
2 exData) at Microsoft.Azure.Storage.Data.ExceptionHandler.HandleException(Exception ex, String fileSystem, String path) at Microsoft.Azure.Storage.Data.ExceptionHandler.ExecuteAction[TResult](Func1 func, String fileSystem, String path) at Microsoft.Azure.Storage.Data.BlobFSClient.IsFileSystemExist(String filesystem, Boolean swallowForbiddenError, Boolean useListPathApi) at Microsoft.Azure.Storage.Data.BlobFSClient.CreateFileSystemIfNotExist(String filesystem, Boolean swallowForbiddenError, Boolean useListPathApiToCheckFileSystemExist) at Microsoft.DataTransfer.Runtime.BlobFSMultipartSink..ctor(AzureBlobFSConnection connection, AzureBlobFSWriteInternalSettings writeSettings) at Microsoft.DataTransfer.Runtime.AzureBlobFSConnection.CreateSink(IDictionary2 properties) at Microsoft.DataTransfer.ClientLibrary.MultipartWriteSink..ctor(IConnectorPlugin connector, IDictionary2 connectionProperties, IDictionary2 writeProperties) at Microsoft.SqlServer.IntegrationService.ExtensibleFileCommon.DataTransferClientHelper.CreateBinaryDataSink(IDictionary2 sinkProperties, Guid transferId, Guid activityId, BinaryTelemetryCollector& collector) at Microsoft.SqlServer.IntegrationService.ExtensibleFileComponents.Destination.ExtensibleFileDest.TransferDataToFile()
我已经为 SQL Server 2012 安装了集成 azure 功能包。请帮忙，谢谢。
您必须交叉检查您在灵活文件目标编辑器中提供的文件夹路径和文件名。尽管如此，如果这导致相同的错误，我建议您安装新版本的 Microsoft SQL Azure 服务器集成服务功能包。
安装最新版SSIS后问题已解决，请检查此类问题。