Yarn 上保留的内存是什么
What is Memory reserved on Yarn
我成功地在 Yarn 上启动了一个 spark 应用程序。但是内存使用情况有点奇怪，如下所示：
保留内存是什么意思？我怎样才能有效地使用所有可用内存？
提前致谢。
查看 Cloudera 的 this blog，其中解释了 YARN 中的新内存管理。
以下是相关内容：
... An implementation detail of this change that prevents applications from starving under this new flexibility is the notion of reserved containers. Imagine two jobs are running that each have enough tasks to saturate more than the entire cluster. One job wants each of its mappers to get 1GB, and another job wants its mappers to get 2GB. Suppose the first job starts and fills up the entire cluster. Whenever one of its task finishes, it will leave open a 1GB slot. Even though the second job deserves the space, a naive policy will give it to the first one because it’s the only job with tasks that fit. This could cause the second job to be starved indefinitely.
To prevent this unfortunate situation, when space on a node is offered to an application, if the application cannot immediately use it, it reserves it, and no other application can be allocated a container on that node until the reservation is fulfilled. Each node may have only one reserved container. The total reserved memory amount is reported in the ResourceManager UI. A high number means that it may take longer for new jobs to get space. ,,,
当容器被分配给某个没有足够资源（cpu 或内存）的节点管理器节点时，容器将变为保留状态。
我成功地在 Yarn 上启动了一个 spark 应用程序。但是内存使用情况有点奇怪，如下所示：
保留内存是什么意思？我怎样才能有效地使用所有可用内存？
提前致谢。
查看 Cloudera 的 this blog，其中解释了 YARN 中的新内存管理。
以下是相关内容：
... An implementation detail of this change that prevents applications from starving under this new flexibility is the notion of reserved containers. Imagine two jobs are running that each have enough tasks to saturate more than the entire cluster. One job wants each of its mappers to get 1GB, and another job wants its mappers to get 2GB. Suppose the first job starts and fills up the entire cluster. Whenever one of its task finishes, it will leave open a 1GB slot. Even though the second job deserves the space, a naive policy will give it to the first one because it’s the only job with tasks that fit. This could cause the second job to be starved indefinitely. To prevent this unfortunate situation, when space on a node is offered to an application, if the application cannot immediately use it, it reserves it, and no other application can be allocated a container on that node until the reservation is fulfilled. Each node may have only one reserved container. The total reserved memory amount is reported in the ResourceManager UI. A high number means that it may take longer for new jobs to get space. ,,,
当容器被分配给某个没有足够资源（cpu 或内存）的节点管理器节点时，容器将变为保留状态。