持久函数是否适合大量活动?
Is durable functions suited for high amount of activities?
我有一个需要计算 500k 活动的场景。都是小算计。由于节流,我只能同时计算 30。
想象一下下面的简单示例:
[FunctionName("Crawl")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
WriteLine("In orchistration");
var outputs = new List<string>();
// here i get 1 million jobs
var jobs = await context.CallActivityAsync<List<Influencer>>("GetSocialAccountJobs", "");
var tasks = new Task<string>[jobs.Count];
var retryOptions = new RetryOptions(
firstRetryInterval: TimeSpan.FromSeconds(60), maxNumberOfAttempts: 3);
for (int i = 0; i < jobs.Count; i++)
{
tasks[i] = context.CallActivityWithRetryAsync<string>("Crawl_Hello", retryOptions, jobs[i].Id);
}
await Task.WhenAll(tasks);
return outputs;
}
每次调用 activity 时,都会调用此编排器函数。并循环所有活动,直到 et 找到一个未被调用的 activity。它将循环数百万次。我是不是遗漏了什么或者持久函数不适合这种情况?
持久函数 auto-scale when running in the Consumption and Elastic Premium plans. There are some documented performance targets 有助于部署持久函数。
具体来说,对于您的情况,您可能需要注意以下几点
Unlike fan-out, fan-in operations are limited to a single VM. If your application uses the fan-out, fan-in pattern and you are concerned about fan-in performance, consider sub-dividing the activity function fan-out across multiple sub-orchestrations.
因此,您可以触发具有较小批量作业的子协调器,而不是从单个协调器调用百万个活动,而后者又会调用 activity 函数。
我有一个需要计算 500k 活动的场景。都是小算计。由于节流,我只能同时计算 30。
想象一下下面的简单示例:
[FunctionName("Crawl")]
public static async Task<List<string>> RunOrchestrator(
[OrchestrationTrigger] DurableOrchestrationContext context)
{
WriteLine("In orchistration");
var outputs = new List<string>();
// here i get 1 million jobs
var jobs = await context.CallActivityAsync<List<Influencer>>("GetSocialAccountJobs", "");
var tasks = new Task<string>[jobs.Count];
var retryOptions = new RetryOptions(
firstRetryInterval: TimeSpan.FromSeconds(60), maxNumberOfAttempts: 3);
for (int i = 0; i < jobs.Count; i++)
{
tasks[i] = context.CallActivityWithRetryAsync<string>("Crawl_Hello", retryOptions, jobs[i].Id);
}
await Task.WhenAll(tasks);
return outputs;
}
每次调用 activity 时,都会调用此编排器函数。并循环所有活动,直到 et 找到一个未被调用的 activity。它将循环数百万次。我是不是遗漏了什么或者持久函数不适合这种情况?
持久函数 auto-scale when running in the Consumption and Elastic Premium plans. There are some documented performance targets 有助于部署持久函数。
具体来说,对于您的情况,您可能需要注意以下几点
Unlike fan-out, fan-in operations are limited to a single VM. If your application uses the fan-out, fan-in pattern and you are concerned about fan-in performance, consider sub-dividing the activity function fan-out across multiple sub-orchestrations.
因此,您可以触发具有较小批量作业的子协调器,而不是从单个协调器调用百万个活动,而后者又会调用 activity 函数。