对于嵌套的配置文件，从 OmegaConf 自定义解析器解析的元组的 Hydra 实例化失败

Question

OmegaConf 允许您注册自定义解析器。这是解析元组的示例。

def resolve_tuple(*args):
    return tuple(args)

OmegaConf.register_new_resolver("tuple", resolve_tuple)

这可用于将具有类似 ${tuple:1,2} 结构的配置文件中的值解析为元组 (1, 2)。连同 hydra.utils.instantiate 这可用于创建包含或使用元组的对象。例如：

config.yaml

obj:
  tuple: ${tuple:1,2}

test.py

import hydra
import hydra.utils as hu

from omegaconf import OmegaConf


def resolve_tuple(*args):
    return tuple(args)


OmegaConf.register_new_resolver('tuple', resolve_tuple)


@hydra.main(config_path='conf', config_name='config_test')
def main(cfg):
    obj = hu.instantiate(cfg.obj, _convert_='partial')
    print(obj)


if __name__ == '__main__':
    main()

运行这个例子returns:

$ python test.py
{'tuple': (1, 2)}

但是，假设您有一个复杂得多的配置结构。您可能想像这样使用插值从其他文件引入配置。

tuple/base.yaml

tuple: ${tuple:1,2}

config.yaml

defaults:
  - tuple: base
  - _self_

obj:
  tuple: ${tuple}

运行这个例子你得到一个错误：

$ python test.py
Error executing job with overrides: []
Traceback (most recent call last):
  File "test.py", line 16, in main
    obj = hu.instantiate(cfg.obj, _convert_='partial')
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 175, in instantiate
    OmegaConf.resolve(config)
omegaconf.errors.UnsupportedValueType: Value 'tuple' is not a supported primitive type

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

hydra 的完整回溯是：

Error executing job with overrides: []
Traceback (most recent call last):
  File "test.py", line 21, in <module>
    main()
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/main.py", line 52, in decorated_main
    config_name=config_name,
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/utils.py", line 378, in _run_hydra
    lambda: hydra.run(
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/utils.py", line 211, in run_and_report

    return func()
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/utils.py", line 381, in <lambda>
    overrides=args.overrides,
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/hydra.py", line 111, in run
    _ = ret.return_value
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/core/utils.py", line 233, in return_value
    raise self._return_value
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/core/utils.py", line 160, in run_job
    ret.return_value = task_function(task_cfg)
  File "test.py", line 17, in main
    model = hu.instantiate(cfg.obj, _convert_='partial')
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 175, in instantiate
    OmegaConf.resolve(config)
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/omegaconf.py", line 792, in resolve
    omegaconf._impl._resolve(cfg)
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/_impl.py", line 40, in _resolve
    _resolve_container_value(cfg, k)
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/_impl.py", line 19, in _resolve_container_value
    _resolve(resolved)
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/_impl.py", line 40, in _resolve
    _resolve_container_value(cfg, k)
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/_impl.py", line 23, in _resolve_container_value
    node._set_value(resolved._value())
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/nodes.py", line 44, in _set_value
    self._val = self.validate_and_convert(value)
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/nodes.py", line 57, in validate_and_convert
    return self._validate_and_convert_impl(value)
  File "/Users/me/anaconda3/envs/my_env/lib/python3.7/site-packages/omegaconf/nodes.py", line 134, in _validate_and_convert_impl
    f"Value '{t.__name__}' is not a supported primitive type"
omegaconf.errors.UnsupportedValueType: Value 'tuple' is not a supported primitive type

如果你真的深入研究跟踪中的 omegaconf 代码，你会发现配置对象 allow_objects 有一个标志，在通过的示例中为 True 和 None 在没有的例子中。有趣的是，在调用 Omegaconf.resolve(config) 之前的 _instantaite2.py 文件中设置了几个标志，一个是 allow_objects as True.

这些 interpolated/resolved 值的预期行为是否从单独的文件填充以覆盖此标志？如果是这样，是否有某种方法可以确保 allow_objects 标志对于所有已解析和内插的值都是（或保持）为真？

Answer 1

我认为有些混淆，因为您将 tuple 这个词用于多个不同的目的:)

这是一个适合我的例子：

# my_app.py
import hydra
import hydra.utils as hu

from omegaconf import OmegaConf

def resolve_tuple(*args):
    return tuple(args)

OmegaConf.register_new_resolver('as_tuple', resolve_tuple)

@hydra.main(config_path='conf', config_name='config')
def main(cfg):
    obj = hu.instantiate(cfg.obj, _convert_='partial')
    print(obj)

if __name__ == '__main__':
    main()

# conf/config.yaml
defaults:
  - subdir: base
  - _self_

obj:
  a_tuple: ${subdir.my_tuple}

# conf/subdir/base.yaml
my_tuple: ${as_tuple:1,2}

$ python my_app.py  # at the command line:
{'a_tuple': (1, 2)}

这里的主要区别是我们有 a_tuple: ${subdir.my_tuple} 而不是 a_tuple: ${my_tuple}。

备注：

OmegaConf 可能会在未来的某个时候将元组作为第一个 class 类型支持。这是相关问题：https://github.com/omry/omegaconf/issues/392
您提到的 allow_objects 标志未记录在案，其行为可能会发生变化。

对于嵌套的配置文件，从 OmegaConf 自定义解析器解析的元组的 Hydra 实例化失败

Hydra instantiation of tuple resolved from OmegaConf custom resolver fails for nested config files

python

fb-hydra

omegaconf