ElasticSearch ILM 不删除索引

Question

我已经为我的 fluentd.* 索引设置了一个简单的 ILM 策略，以便在（用于测试 - ）短时间后删除。

ILM:

PUT _ilm/policy/fluentd
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "1d",
            "max_size": "1gb"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "4d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

索引模板：

PUT _template/fluentd
{
  "order": 0,
  "index_patterns": [
    "fluentd.*"
  ],
  "settings": {
    "index": {
      "lifecycle": {
        "name": "fluentd"
      }
    }
  },
  "aliases": {
    "fluent": {}
  }
}

使用这些设置，我预计 ES 会删除超过 5-6 天的索引，但 ES 中仍然有 3 周前的索引。目前，它表示有 108 个链接索引到此 ILM 策略。

它实际上在做什么，它似乎什么也没做...如何在 x 天后删除索引？

我首先尝试使用 "index template" 但它没用，它不会将设置应用于每个索引（也许是，但仅在创建时？？？？）。

然后我手动将 ILM 放在索引上（另一个错误：你不能 select 所有索引并点击 "add ILM policy" - 你需要一个一个地添加策略）这需要我点击了大约 600 次。

现在的问题是，我已经定义了 "hot" 阶段，但它没有触发（这是错误的？）- 因为热阶段没有触发（我设置为 "rollover after 1 day after index creation"） - 删除阶段也没有。当我移除热阶段并仅使用删除再次应用 ILM 进行索引时 - 它起作用了！但是添加和删除所有这些都是错误的，我得到了糟糕的信息，到处都是错误。

我不明白为什么在更改 ILM 策略时必须删除 ILM 并将其重新应用于每个索引。 1000% 不方便。

ES 确实需要做一些工作，它仍然是测试版，我得到了很多状态代码 500，尽管我直接在 Elastic Cloud 上使用最新版本。

Answer 1

With these settings, I expected ES to delete indices older than 5-6 days, but there are still indices from 3 weeks ago in ES. Currently, it says there are 108 linked indices to this ILM policy.

根据您的设置，删除阶段从回滚后第 4 天开始。如果你想在“索引创建”后的第 4 天开始删除阶段，你需要从热阶段删除翻转操作：

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "4d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

I tried first to use the "index template" but it's useless, it does not apply settings to each index (maybe yes but only on creation????).

是的，它适用于创建索引。

Then I put the ILM on the index by hand (another bug: you can't select all index and hit "add ILM policy" - you need to add the policy one by one) which required me to click about 600 times.

Kibana 不允许您将 ILM 策略应用于所有索引，但 elasticsearch API 允许！只需打开 kibana 开发工具和运行以下请求：

PUT fluentd.*/_settings
{
  "index": {
    "lifecycle": {
      "name": "fluentd"
    }
  }
}

Now the problem was, I had "hot" phase defined but it didn't trigger (it's buggy?) - because the hot phase didn't trigger (i set to to "rollover after 1 day after index creation") - the delete phase didn't either. When I removed the hot phase and applied the ILM to index again with only delete - it worked! but adding and removing all this is buggy, I get Ooops, something went wrong errors here and there.

如果未触发翻转阶段，ILM 将无法进行。

I don't understand why I have to remove the ILM and reapply it to each index when I change something in the ILM policy. It's 1000% inconvenient.

因为 ILM 定义缓存在每个索引上。查看文档：https://www.elastic.co/guide/en/elasticsearch/reference/current/ilm-index-lifecycle.html#ilm-phase-execution

Answer 2

有点晚了，但也许会对某些人有所帮助。

另一个原因可能是刚才提到的here:

ILM is not really intended to be used on 1m lifecycle. I Do not believe You will achieve your desired behavior. My understanding is that ILM is an opportunistic background task it is not preemptive so it is not going to execute on the exact time frame.

It's designed to work on the order of hours or days not minutes.

我的索引也有同样的情况，我检查了 - 索引被删除了，但后来我把它们放了起来。

ElasticSearch ILM 不删除索引

ElasticSearch ILM not deleting indices

elasticsearch

elasticsearch-indices

elastic-cloud