从 Azure 数据工厂管道调用 Webjob 抛出 HTTP 409 冲突异常错误

Question

我有一个 OnDemand 触发的网络作业，我想使用 HTTP 链接服务通过 ADF 复制 activity 触发它。这是链接服务：-

{
    "name": "LS_WebJob",
    "properties": {
        "hubName": "yas-cdp-adf_hub",
        "type": "Http",
        "typeProperties": {
            "url": "https://cust-app.scm.azurewebsites.net/api/triggeredwebjobs/ConsoleApplication1/run",
            "authenticationType": "Basic",
            "username": "$custdata-app",
            "password": "**********"
        }
    }
}

输入数据集

{
    "name": "ZZ_Inp_Webjob",
    "properties": {
        "published": false,
        "type": "Http",
        "linkedServiceName": "LS_WebJob",
        "typeProperties": {
            "requestMethod": "Post",
            "requestBody": "Hey Buddy"
        },
        "availability": {
            "frequency": "Day",
            "interval": 1,
            "style": "StartOfInterval"
        },
        "external": true,
        "policy": {}
    }
}

输出数据集

{
    "name": "ZZ_Out_WebJob",
    "properties": {
        "published": false,
        "type": "AzureBlob",
        "linkedServiceName": "LS_ABLB",
        "typeProperties": {
            "fileName": "webjob.json",
            "folderPath": "yc-cdp-container/Dummy/temp",
            "format": {
                "type": "TextFormat"
            }
        },
        "availability": {
            "frequency": "Day",
            "interval": 1,
            "style": "StartOfInterval"
        }
    }
}

流水线

{
"name": "ZZ-PL-WebJob",
"properties": {
    "description": "This pipeline copies data from an HTTP Marina WiFi Source URL to Azure blob",
    "activities": [
        {
            "type": "Copy",
            "typeProperties": {
                "source": {
                    "type": "HttpSource"
                },
                "sink": {
                    "type": "BlobSink",
                    "writeBatchSize": 0,
                    "writeBatchTimeout": "00:00:00"
                }
            },
            "inputs": [
                {
                    "name": "ZZ_Inp_Webjob"
                }
            ],
            "outputs": [
                {
                    "name": "ZZ_Out_Webjob"
                }
            ],
            "policy": {
                "timeout": "01:00:00",
                "concurrency": 1
            },
            "scheduler": {
                "frequency": "Day",
                "interval": 1,
                "style": "StartOfInterval"
            },
            "name": "WebjobSourceToAzureBlob",
            "description": "Copy from an HTTP source to an Azure blob"
        }
    ],
    "start": "2017-04-10T01:00:00Z",
    "end": "2017-04-10T01:00:00Z",
    "isPaused": false,
    "hubName": "yas-cdp-adf_hub",
    "pipelineMode": "Scheduled"
}
}

我的网络作业是一个简单的 C# 应用程序：-

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("My Team Rocks!");
        }
    }
}

当我执行管道时，网络作业被成功触发。但是管道失败并出现 HTTP 409 冲突错误。

Copy activity encountered a user error at Source side: ErrorCode=UserErrorFailedToReadHttpFile,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to read data from http source file.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (409) Conflict.,Source=System,'.

Answer 1

尝试将网关名称添加到 linked 服务 json。参考这个 link 。

从 Azure 数据工厂管道调用 Webjob 抛出 HTTP 409 冲突异常错误

Calling Webjob from Azure Data Factory pipeline throwing HTTP 409 conflict exception error

azure-webjobs

azure-data-factory