U-SQL Activity 不是 Azure 数据工厂上的运行

Question

我正在使用 Azure 数据工厂将本地数据传输到 Azure Data Lake 存储。复制数据后，我在上传的文件上运行ning 一个 U-SQL 脚本以将其转换为新的 .csv 文件。如果运行来自 Visual Studio 或直接来自 Data Lake Analytics，我的 U-SQL 工作运行ning 很好。

但如果在 Azure 数据工厂中添加 activity。复制数据后，此 U-SQL 脚本 activity 立即失败。我尝试了很多方法，但无法解决问题。它给了我以下错误。

JSON我的U-SQLActivity的定义是

{
    "name": "Transform Data",
    "description": "This will transform work space data.",
    "type": "DataLakeAnalyticsU-SQL",
    "dependsOn": [
        {
            "activity": "Copy_workplace_groups_info_2018_03_19_09_32_csv",
            "dependencyConditions": [
                "Completed"
            ]
        }
    ],
    "policy": {
        "timeout": "7.00:00:00",
        "retry": 0,
        "retryIntervalInSeconds": 30,
        "secureOutput": false
    },
    "typeProperties": {
        "scriptPath": "Scripts/Script.usql",
        "scriptLinkedService": {
            "referenceName": "Destination_DataLakeStore_lc0",
            "type": "LinkedServiceReference"
        }
    },
    "linkedServiceName": {
        "referenceName": "AzureDataLakeAnalyticsForDF",
        "type": "LinkedServiceReference"
    }
}

整个管道的

JSON是

{
    "name": "CopyPipeline_d26",
    "properties": {
        "activities": [
            {
                "name": "Copy_workplace_groups_info_2018_03_19_09_32_csv",
                "type": "Copy",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false
                },
                "typeProperties": {
                    "source": {
                        "type": "FileSystemSource",
                        "recursive": false
                    },
                    "sink": {
                        "type": "AzureDataLakeStoreSink",
                        "copyBehavior": "MergeFiles"
                    },
                    "enableStaging": false,
                    "cloudDataMovementUnits": 0,
                    "enableSkipIncompatibleRow": true
                },
                "inputs": [
                    {
                        "referenceName": "workplace_groups_info_2018_03_19_09_32_csv_i_lc0",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "workplace_groups_info_2018_03_19_09_32_csv_o_lc0",
                        "type": "DatasetReference"
                    }
                ]
            },
            {
                "name": "Transform Data",
                "description": "This will transform work space data.",
                "type": "DataLakeAnalyticsU-SQL",
                "dependsOn": [
                    {
                        "activity": "Copy_workplace_groups_info_2018_03_19_09_32_csv",
                        "dependencyConditions": [
                            "Completed"
                        ]
                    }
                ],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false
                },
                "typeProperties": {
                    "scriptPath": "Scripts/Script.usql",
                    "scriptLinkedService": {
                        "referenceName": "Destination_DataLakeStore_lc0",
                        "type": "LinkedServiceReference"
                    }
                },
                "linkedServiceName": {
                    "referenceName": "AzureDataLakeAnalyticsForDF",
                    "type": "LinkedServiceReference"
                }
            }
        ],
        "parameters": {
            "windowStart": {
                "type": "String"
            },
            "windowEnd": {
                "type": "String"
            }
        }
    }
}

Answer 1

我通过使用 AppService 创建运行time 解决了这个问题。我遵循了以下步骤。

我在 Active Directory 中创建了一个 WebApp。
也为该 WebApp 分配 Azure Data Lake 权限。
在该应用程序中创建一个 public 键并记下它。它再也不会出现了。
记下该 WebApp 的应用程序 ID。
打开 Azure Data Lake Analytics 并将贡献者角色分配给在 Active Directory 中创建的 WebApp。
在创建运行时间。

它工作正常。 :)

U-SQL Activity 不是 Azure 数据工厂上的运行

U-SQL Activity is not running on Azure Data Factory

azure-data-factory

azure-data-lake

u-sql

azure-data-factory-2

U-SQL Activity 不是 Azure 数据工厂上的 运行

U-SQL Activity is not running on Azure Data Factory

azure-data-factory

azure-data-lake

u-sql

azure-data-factory-2

U-SQL Activity 不是 Azure 数据工厂上的运行