这意味着什么，Spark 的内部公平调度器让长期存在的应用程序定义队列
What does it means, Spark’s internal Fair Scheduler lets long-lived applications define queues
我正在尝试了解 Spark 的作业调度并在 Learning Spark,
中得到了这一点
"Spark provides a mechanism through configurable intra-application
scheduling policies. Spark’s internal Fair Scheduler lets long-lived
applications define queues for prioritizing scheduling of tasks"
你能给我更多的细节吗？
如 fair scheduler documentation 中所述，您可以维护多个池，每个池都有自己的调度策略、最小 (minShare) 和相对 (weight) 资源分配。最后一位描述如下：
If you give a specific pool a weight of 2, for example, it will get 2x more resources as other active pools. Setting a high weight such as 1000 also makes it possible to implement priority between pools—in essence, the weight-1000 pool will always get to launch tasks first whenever it has jobs active.
我正在尝试了解 Spark 的作业调度并在 Learning Spark,
中得到了这一点"Spark provides a mechanism through configurable intra-application scheduling policies. Spark’s internal Fair Scheduler lets long-lived applications define queues for prioritizing scheduling of tasks"
你能给我更多的细节吗？
如 fair scheduler documentation 中所述，您可以维护多个池，每个池都有自己的调度策略、最小 (minShare) 和相对 (weight) 资源分配。最后一位描述如下：
If you give a specific pool a weight of 2, for example, it will get 2x more resources as other active pools. Setting a high weight such as 1000 also makes it possible to implement priority between pools—in essence, the weight-1000 pool will always get to launch tasks first whenever it has jobs active.