NULL 从管道传递到链接的服务 baseURL

Question

我有一个管道可以遍历文件并将它们复制到存储位置。

baseURL 和 relativeURL 存储在 json 文件中。

我可以读入这个文件，它是有效的。

我已将链接服务 baseURL 参数化，这在从链接服务和数据集进行测试时有效。

然而，当我尝试调试管道时，出现错误：

"code":"BadRequest"
"message":null
"target":"pipeline//runid/310b8ac1-2ce6-4c7c-a1ad-433ee9019e9b"
"details":null
"error":null

似乎从管道中的 activity 传递了空值而不是 baseURL。

我已经迭代了我的配置文件中的值并且正在读取它并且值是正确的。管道似乎没有为 baseURL 传递正确的值。

我是否必须修改管道后面的 json 代码才能使其正常工作？

如果有帮助，链接服务、数据集和管道的json如下：

--链接服务：

{
"name": "ls_http_opendata_ecdc_europe_eu",
"properties": {
    "parameters": {
        "baseURL": {
            "type": "string"
        }
    },
    "annotations": [],
    "type": "HttpServer",
    "typeProperties": {
        "url": "@linkedService().baseURL",
        "enableServerCertificateValidation": true,
        "authenticationType": "Anonymous"
    }
}
}

--数据集

{
"name": "ds_ecdc_raw_csv_http",
"properties": {
    "linkedServiceName": {
        "referenceName": "ls_http_opendata_ecdc_europe_eu",
        "type": "LinkedServiceReference"
    },
    "parameters": {
        "relativeURL": {
            "type": "string"
        },
        "baseURL": {
            "type": "string"
        }
    },
    "annotations": [],
    "type": "DelimitedText",
    "typeProperties": {
        "location": {
            "type": "HttpServerLocation",
            "relativeUrl": {
                "value": "@dataset().relativeURL",
                "type": "Expression"
            }
        },
        "columnDelimiter": ",",
        "escapeChar": "\",
        "firstRowAsHeader": true,
        "quoteChar": "\""
    },
    "schema": []
}
}

--管道

{
"name": "pl_ingest_ecdc_data",
"properties": {
    "activities": [
        {
            "name": "lookup ecdc filelist",
            "type": "Lookup",
            "dependsOn": [],
            "policy": {
                "timeout": "7.00:00:00",
                "retry": 0,
                "retryIntervalInSeconds": 30,
                "secureOutput": false,
                "secureInput": false
            },
            "userProperties": [],
            "typeProperties": {
                "source": {
                    "type": "JsonSource",
                    "storeSettings": {
                        "type": "AzureBlobStorageReadSettings",
                        "recursive": true,
                        "enablePartitionDiscovery": false
                    },
                    "formatSettings": {
                        "type": "JsonReadSettings"
                    }
                },
                "dataset": {
                    "referenceName": "ds_ecdc_file_list",
                    "type": "DatasetReference"
                },
                "firstRowOnly": false
            }
        },
        {
            "name": "execute copy for every record",
            "type": "ForEach",
            "dependsOn": [
                {
                    "activity": "lookup ecdc filelist",
                    "dependencyConditions": [
                        "Succeeded"
                    ]
                }
            ],
            "userProperties": [],
            "typeProperties": {
                "items": {
                    "value": "@activity('lookup ecdc filelist').output.value",
                    "type": "Expression"
                },
                "activities": [
                    {
                        "name": "Copy data1",
                        "type": "Copy",
                        "dependsOn": [],
                        "policy": {
                            "timeout": "7.00:00:00",
                            "retry": 0,
                            "retryIntervalInSeconds": 30,
                            "secureOutput": false,
                            "secureInput": false
                        },
                        "userProperties": [],
                        "typeProperties": {
                            "source": {
                                "type": "DelimitedTextSource",
                                "storeSettings": {
                                    "type": "HttpReadSettings",
                                    "requestMethod": "GET"
                                },
                                "formatSettings": {
                                    "type": "DelimitedTextReadSettings"
                                }
                            },
                            "sink": {
                                "type": "DelimitedTextSink",
                                "storeSettings": {
                                    "type": "AzureBlobFSWriteSettings"
                                },
                                "formatSettings": {
                                    "type": "DelimitedTextWriteSettings",
                                    "quoteAllText": true,
                                    "fileExtension": ".txt"
                                }
                            },
                            "enableStaging": false,
                            "translator": {
                                "type": "TabularTranslator",
                                "typeConversion": true,
                                "typeConversionSettings": {
                                    "allowDataTruncation": true,
                                    "treatBooleanAsNumber": false
                                }
                            }
                        },
                        "inputs": [
                            {
                                "referenceName": "DelimitedText1",
                                "type": "DatasetReference",
                                "parameters": {
                                    "sourceBaseURL": {
                                        "value": "@item().sourceBaseURL",
                                        "type": "Expression"
                                    },
                                    "sourceRelativeURL": {
                                        "value": "@item().sourceRelativeURL",
                                        "type": "Expression"
                                    }
                                }
                            }
                        ],
                        "outputs": [
                            {
                                "referenceName": "ds_ecdc_raw_csv_dl",
                                "type": "DatasetReference",
                                "parameters": {
                                    "fileName": {
                                        "value": "@item().sinkFileName",
                                        "type": "Expression"
                                    }
                                }
                            }
                        ]
                    }
                ]
            }
        }
    ],
    "concurrency": 1,
    "annotations": []
}
}

Answer 1

我重现了你的错误。

{"code":"BadRequest","message":null,"target":"pipeline//runid/abd35329-3625-490b-85cf-f6d0de3dac86","details":null,"error":null}

这是因为您没有将 baseURL 传递给源数据集中的 link 服务。请这样做：

和数据集 JSON 代码应该是这样的：

{
    "name": "ds_ecdc_raw_csv_http",
    "properties": {
        "linkedServiceName": {
            "referenceName": "ls_http_opendata_ecdc_europe_eu",
            "type": "LinkedServiceReference",
            "parameters": {
                "baseURL": {
                    "value": "@dataset().baseURL",
                    "type": "Expression"
                }
            }
        },
        "parameters": {
            "relativeURL": {
                "type": "string"
            },
            "baseURL": {
                "type": "string"
            }
        },
        "annotations": [],
        "type": "DelimitedText",
        "typeProperties": {
            "location": {
                "type": "HttpServerLocation",
                "relativeUrl": {
                    "value": "@dataset().relativeURL",
                    "type": "Expression"
                }
            },
            "columnDelimiter": ",",
            "escapeChar": "\",
            "firstRowAsHeader": true,
            "quoteChar": "\""
        },
        "schema": []
    }
}

NULL 从管道传递到链接的服务 baseURL

NULL being passed from pipeline to linked service baseURL

azure-data-factory

azure-data-factory-2

azure-data-factory-pipeline