dbt pre-hooks 不能 render/find 宏?
dbt pre-hooks cannot render/find macros?
dbt_project.yml 模型预挂钩无法呈现宏。以下实施:
# gold/dbt_project.yml
config-version: 2
...
models:
gold:
+pre-hooks: "{{ incremental_failsafe() }}"
以及以下宏:
-- gold/macros/utils/incremental_failsafe.sql
{% macro incremental_failsafe() %}
{# /*
Used to wrap AND clauses that should only be run on dev/CI dbt runs.
For example, to limit the number of records scanned on an incremental
table. This macro will return True unless the target profile is 'prod':
`dbt run --target prod`
*/ #}
{% if target.name == "prod" %}
{% set incremental_failsafe = False %}
{% else %}
{% set incremental_failsafe = True %}
{% endif %}
{{ log("Running with incremental_failsafe: " ~ incremental_failsafe, info=True) }}
{{ return(incremental_failsafe) }}
{% endmacro %}
结果如下:
(snowflake) Teghans-MacBook-Pro:gold tnightengale$ dbt compile
Running with dbt=0.17.2
Encountered an error:
Compilation Error
Could not render {{ incremental_failsafe() }}: 'incremental_failsafe' is undefined
我知道 dbt 0.17.0 围绕预挂钩存在问题,如 this slack exchange 中所述。尽管我是 运行 0.17.2(不是 0.17.0),但我已经尝试按照该对话中的建议在 yml 中同时使用 +pre-hook:
和 pre-hook:
规范但无济于事.欢迎任何见解!
编辑:解决方案
老实说,答案是如此愚蠢和明显:错误是由于 +pre-hooks:
而不是 +pre-hook:
。然而,建议的答案让我重新看了一下。我将它标记为正确的,因为它提供了很多关于钩子的有用上下文。干杯!
第一印象 - 也许您混淆了特定模型的 模型 pre-hook 和 运行 pre-hook应该去?
编辑:在上面纠正我自己 - Asker 正在尝试执行模型配置 pre-hooks,而不是 运行 pre-hooks。我现在认为您需要将 pre-hook 移动到与名称空间配置相同的优先级别。见下文:
dbt_project.yml
# A run pre-hook that applies to all runs would go here:
on-run-start:
- "{{ example_pre_run_macro() }}"
models:
# a pre-hooks to all models goes here *before* but equivalent depth to the namespace:
+pre-hook: "{{ incremental_failsafe() }}"
namespace:
# Below configures models found in models/events/
events:
+enabled: true
+materialized: view
# a pre-hook to a single model / model directory goes here:
+pre-hook: "{{ example_pre_model_macro() }}"
GitLab 的 dbt 存储库是如何设置所有这些的一个很好的例子:Gitlab dbt_project.yml
TLDR:像这样设置您的 dbt_project.yml
# gold/dbt_project.yml
config-version: 2
...
models:
+pre-hook: "{{ incremental_failsafe() }}"
gold:
<models config here>
dbt_project.yml 模型预挂钩无法呈现宏。以下实施:
# gold/dbt_project.yml
config-version: 2
...
models:
gold:
+pre-hooks: "{{ incremental_failsafe() }}"
以及以下宏:
-- gold/macros/utils/incremental_failsafe.sql
{% macro incremental_failsafe() %}
{# /*
Used to wrap AND clauses that should only be run on dev/CI dbt runs.
For example, to limit the number of records scanned on an incremental
table. This macro will return True unless the target profile is 'prod':
`dbt run --target prod`
*/ #}
{% if target.name == "prod" %}
{% set incremental_failsafe = False %}
{% else %}
{% set incremental_failsafe = True %}
{% endif %}
{{ log("Running with incremental_failsafe: " ~ incremental_failsafe, info=True) }}
{{ return(incremental_failsafe) }}
{% endmacro %}
结果如下:
(snowflake) Teghans-MacBook-Pro:gold tnightengale$ dbt compile
Running with dbt=0.17.2
Encountered an error:
Compilation Error
Could not render {{ incremental_failsafe() }}: 'incremental_failsafe' is undefined
我知道 dbt 0.17.0 围绕预挂钩存在问题,如 this slack exchange 中所述。尽管我是 运行 0.17.2(不是 0.17.0),但我已经尝试按照该对话中的建议在 yml 中同时使用 +pre-hook:
和 pre-hook:
规范但无济于事.欢迎任何见解!
编辑:解决方案
老实说,答案是如此愚蠢和明显:错误是由于 +pre-hooks:
而不是 +pre-hook:
。然而,建议的答案让我重新看了一下。我将它标记为正确的,因为它提供了很多关于钩子的有用上下文。干杯!
第一印象 - 也许您混淆了特定模型的 模型 pre-hook 和 运行 pre-hook应该去?
编辑:在上面纠正我自己 - Asker 正在尝试执行模型配置 pre-hooks,而不是 运行 pre-hooks。我现在认为您需要将 pre-hook 移动到与名称空间配置相同的优先级别。见下文:
dbt_project.yml
# A run pre-hook that applies to all runs would go here:
on-run-start:
- "{{ example_pre_run_macro() }}"
models:
# a pre-hooks to all models goes here *before* but equivalent depth to the namespace:
+pre-hook: "{{ incremental_failsafe() }}"
namespace:
# Below configures models found in models/events/
events:
+enabled: true
+materialized: view
# a pre-hook to a single model / model directory goes here:
+pre-hook: "{{ example_pre_model_macro() }}"
GitLab 的 dbt 存储库是如何设置所有这些的一个很好的例子:Gitlab dbt_project.yml
TLDR:像这样设置您的 dbt_project.yml
# gold/dbt_project.yml
config-version: 2
...
models:
+pre-hook: "{{ incremental_failsafe() }}"
gold:
<models config here>