为 Postgres 13 中的高更新表调整 FILLFACTOR
Tuning FILLFACTOR for high-UPDATE tables in Postgres 13
HOT 和 FILLFACTOR 结果
我有一些高更新 tables，我已将 FILLFACTOR 调整为 95%，我正在检查它们。我认为我的设置不正确，也不清楚如何智能地调整它们。我又浏览了 Laurenz Albe 的有用博客 post 上的热门更新
https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/
...和清晰的源代码自述：
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/heap/README.HOT
下面是一个查询，改编自博客 post，用于检查系统中 table 的状态，以及一些示例输出：
SELECT relname,
n_tup_upd as total_update_count,
n_tup_hot_upd as hot_update_count,
coalesce(div_safe(n_tup_upd, n_tup_hot_upd),0) as total_by_hot,
coalesce(div_safe(n_tup_hot_upd, n_tup_upd),0) as hot_by_total
FROM pg_stat_user_tables
order by 4 desc;
一些结果：
relname total_update_count hot_update_count total_by_hot hot_by_total
rollups 369418 128 2886.0781 0.00034649097
q_event 71781 541 132.68207 0.007536813
analytic_scan 2104727 34304 61.35515 0.016298551
clinic 4424 77 57.454544 0.017405063
facility_location 179636 6489 27.683157 0.03612305
target_snapshot 494 18 27.444445 0.036437247
inv 1733021 78234 22.151762 0.045143135
我不确定我在这里寻找什么比例。谁能告诉我如何阅读这些结果，或者阅读什么来弄清楚如何解释它们？
这些更新是 HOTable 吗？
我在这个问题的初稿中没有提到这个基本点。我检查了几个月前的补丁，我 运行 SET (fillfactor = 95) 然后 VACUUM (FULL, VERBOSE, ANALYZE) 我的 table 中的 13 个。 （VERBOSE 在那里，因为我有一些 table 不能 VACUUM 因为几个月前的过程需要清理，这就是我发现问题的方式。pg_stat_activity 是我的朋友。）
但是，至少大多数 touch 索引列...但具有相同的值。像 1 = 1，所以值没有变化。我一直在想那是HOTable。如果我错了，无赖。如果不是，我主要希望澄清 fillfactor、n_tup_upd 和 n_tup_hot_upd.
之间关系的确切目标是什么
SELECT relname,
n_tup_upd as total_update_count,
n_tup_hot_upd as hot_update_count,
coalesce(div_safe(n_tup_upd, n_tup_hot_upd),0) as total_by_hot,
coalesce(div_safe(n_tup_hot_upd, n_tup_upd),0) as hot_by_total,
(select value::integer from table_get_options('data',relname) where option = 'fillfactor') as fillfactor_setting
FROM pg_stat_user_tables
WHERE relname IN (
'activity',
'analytic_productivity',
'analytic_scan',
'analytic_sterilizer_load',
'analytic_sterilizer_loadinv',
'analytic_work',
'assembly',
'data_file_info',
'inv',
'item',
'print_job',
'q_event')
order by 4 desc;
结果：
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| relname | total_update_count | hot_update_count | total_divided_by_hot | hot_divided_by_total | fillfactor_setting |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| q_event | 71810 | 553 | 129.85533 | 0.0077008773 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_scan | 2109206 | 34536 | 61.072678 | 0.016373934 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| inv | 1733176 | 78387 | 22.110502 | 0.045227375 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| item | 630586 | 32110 | 19.638306 | 0.05092089 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_sterilizer_loadinv | 76976539 | 5206806 | 14.783831 | 0.06764147 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_work | 8117050 | 608847 | 13.331839 | 0.07500841 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| assembly | 90580 | 7281 | 12.4405985 | 0.08038198 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_sterilizer_load | 19249 | 2997 | 6.422756 | 0.1556964 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| activity | 3795 | 711 | 5.3375525 | 0.18735178 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_productivity | 106486 | 25899 | 4.1115875 | 0.24321507 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| print_job | 1414 | 388 | 3.6443298 | 0.27439886 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| data_file_info | 402086 | 285663 | 1.4075537 | 0.7104525 | 90 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
（我只是在 https://www.tablesgenerator.com/text_tables 上寻找并找到了一个在线 table 生成器来帮助解决此类示例。使用起来有点笨拙，但比构建等宽字体要快手动对齐文本。)
FILLFACTOR 和 HOT 更新率
我想我可以通过改编来自 https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/ 的 Laurenz Albe 的代码来稍微解决这个问题。我在这里所做的就是制作一个脚本，该脚本构建一个 table，FILLFACTOR 为 10、20、30.....100%，然后以相同的方式为每个百分比更新它.每次创建 table 时，它都会填充 256 条记录，然后每条记录更新 10 次。更新将非索引字段设置回自身，因此实际上没有值更改：
UPDATE mytable SET val = val;
结果如下：
+------------+---------------+-------------+--------------+
| FILLFACTOR | total_updates | hot_updates | total_to_hot |
+------------+---------------+-------------+--------------+
| 10 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 20 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 30 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 40 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 50 | 2350 | 2223 | 1.06 |
+------------+---------------+-------------+--------------+
| 60 | 2350 | 2188 | 1.07 |
+------------+---------------+-------------+--------------+
| 70 | 2350 | 1883 | 1.25 |
+------------+---------------+-------------+--------------+
| 80 | 2350 | 1574 | 1.49 |
+------------+---------------+-------------+--------------+
| 90 | 2350 | 1336 | 1.76 |
+------------+---------------+-------------+--------------+
| 100 | 2350 | 987 | 2.38 |
+------------+---------------+-------------+--------------+
由此看来，当total_to_hot比率上升时，可能对增加FILLFACTOR有好处。
https://www.postgresql.org/docs/13/monitoring-stats.html
n_tup_upd 计算 所有 更新，包括 HOT 更新，而 n_tup_hot_upd 仅计算 HOT 更新。但它似乎并没有计算“如果我们没有 运行 页面上没有空间的话，这可能是一个热更新。”那太好了，但似乎也有很多要求。 （也许跟踪它的成本更高？）
这是脚本。我对每个 FILLFACTOR.
的测试进行了编辑和重新 运行
-- Set up the table for the test
DROP TABLE IF EXISTS mytable;
CREATE TABLE mytable (
id integer PRIMARY KEY,
val integer NOT NULL
) WITH (autovacuum_enabled = off);
-- Change the FILLFACTOR. The default is 100.
ALTER TABLE mytable SET (fillfactor = 10); -- The only part that changes between runs.
-- Seed the data
INSERT INTO mytable
SELECT *, 0
FROM generate_series(1, 235) AS n;
-- Thrash the data
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
-- How does it look?
SELECT n_tup_upd as total_updates,
n_tup_hot_upd as hot_updates,
div_safe(n_tup_upd, n_tup_hot_upd) as total_to_hot
FROM pg_stat_user_tables
WHERE relname = 'mytable';
检查 FILLFACTOR 设置
作为旁注，我想快速打电话检查 table 上的 FILLFACTOR 设置，结果比我想象的要复杂得多。我写了一个有效的函数，但可能会看到一些改进......如果有人有建议的话。我这样称呼它：
select * from table_get_options('foo', 'bar');
或
select * from table_get_options('foo','bar') where option = 'fillfactor';
这是代码，如果有人可以提供改进：
CREATE OR REPLACE FUNCTION dba.table_get_options(text,text)
RETURNS TABLE (
schema_name text,
table_name text,
option text,
value text
)
LANGUAGE SQL AS
$BODY$
WITH
packed_options AS (
select pg_class.relname as table_name,
btrim(pg_options_to_table(pg_class.reloptions)::text, '()') as option_kvp -- Convert to text (fillfactor,95), then strip off ( and )
from pg_class
join pg_namespace
on pg_namespace.oid = pg_class.relnamespace
where pg_namespace.nspname =
and relname =
and reloptions is not null
),
unpacked_options AS (
select as schema_name,
as table_name,
split_part(option_kvp, ',', 1) as option,
split_part(option_kvp, ',', 2) as value
from packed_options
)
select * from unpacked_options;
$BODY$;
这些数字表明您的策略无效，并且绝大多数更新都不是 HOT。您还说明了原因：即使您将索引列更新为原始值，您也不会获得热更新。
解决方案是通过在 UPDATE 语句中包含索引列来区分，仅当它确实被修改时。
95 的 fillfactor 也相当高，除非您的表的行非常小。也许使用 90 或 85 之类的设置会获得更好的结果。
HOT 和 FILLFACTOR 结果
我有一些高更新 tables，我已将 FILLFACTOR 调整为 95%，我正在检查它们。我认为我的设置不正确，也不清楚如何智能地调整它们。我又浏览了 Laurenz Albe 的有用博客 post 上的热门更新
https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/
...和清晰的源代码自述：
https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/access/heap/README.HOT
下面是一个查询，改编自博客 post，用于检查系统中 table 的状态，以及一些示例输出：
SELECT relname,
n_tup_upd as total_update_count,
n_tup_hot_upd as hot_update_count,
coalesce(div_safe(n_tup_upd, n_tup_hot_upd),0) as total_by_hot,
coalesce(div_safe(n_tup_hot_upd, n_tup_upd),0) as hot_by_total
FROM pg_stat_user_tables
order by 4 desc;
一些结果：
relname total_update_count hot_update_count total_by_hot hot_by_total
rollups 369418 128 2886.0781 0.00034649097
q_event 71781 541 132.68207 0.007536813
analytic_scan 2104727 34304 61.35515 0.016298551
clinic 4424 77 57.454544 0.017405063
facility_location 179636 6489 27.683157 0.03612305
target_snapshot 494 18 27.444445 0.036437247
inv 1733021 78234 22.151762 0.045143135
我不确定我在这里寻找什么比例。谁能告诉我如何阅读这些结果，或者阅读什么来弄清楚如何解释它们？
这些更新是 HOTable 吗？
我在这个问题的初稿中没有提到这个基本点。我检查了几个月前的补丁，我 运行 SET (fillfactor = 95) 然后 VACUUM (FULL, VERBOSE, ANALYZE) 我的 table 中的 13 个。 （VERBOSE 在那里，因为我有一些 table 不能 VACUUM 因为几个月前的过程需要清理，这就是我发现问题的方式。pg_stat_activity 是我的朋友。）
但是，至少大多数 touch 索引列...但具有相同的值。像 1 = 1，所以值没有变化。我一直在想那是HOTable。如果我错了，无赖。如果不是，我主要希望澄清 fillfactor、n_tup_upd 和 n_tup_hot_upd.
SELECT relname,
n_tup_upd as total_update_count,
n_tup_hot_upd as hot_update_count,
coalesce(div_safe(n_tup_upd, n_tup_hot_upd),0) as total_by_hot,
coalesce(div_safe(n_tup_hot_upd, n_tup_upd),0) as hot_by_total,
(select value::integer from table_get_options('data',relname) where option = 'fillfactor') as fillfactor_setting
FROM pg_stat_user_tables
WHERE relname IN (
'activity',
'analytic_productivity',
'analytic_scan',
'analytic_sterilizer_load',
'analytic_sterilizer_loadinv',
'analytic_work',
'assembly',
'data_file_info',
'inv',
'item',
'print_job',
'q_event')
order by 4 desc;
结果：
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| relname | total_update_count | hot_update_count | total_divided_by_hot | hot_divided_by_total | fillfactor_setting |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| q_event | 71810 | 553 | 129.85533 | 0.0077008773 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_scan | 2109206 | 34536 | 61.072678 | 0.016373934 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| inv | 1733176 | 78387 | 22.110502 | 0.045227375 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| item | 630586 | 32110 | 19.638306 | 0.05092089 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_sterilizer_loadinv | 76976539 | 5206806 | 14.783831 | 0.06764147 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_work | 8117050 | 608847 | 13.331839 | 0.07500841 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| assembly | 90580 | 7281 | 12.4405985 | 0.08038198 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_sterilizer_load | 19249 | 2997 | 6.422756 | 0.1556964 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| activity | 3795 | 711 | 5.3375525 | 0.18735178 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| analytic_productivity | 106486 | 25899 | 4.1115875 | 0.24321507 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| print_job | 1414 | 388 | 3.6443298 | 0.27439886 | 95 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
| data_file_info | 402086 | 285663 | 1.4075537 | 0.7104525 | 90 |
+-----------------------------+--------------------+------------------+----------------------+----------------------+--------------------+
（我只是在 https://www.tablesgenerator.com/text_tables 上寻找并找到了一个在线 table 生成器来帮助解决此类示例。使用起来有点笨拙，但比构建等宽字体要快手动对齐文本。)
FILLFACTOR 和 HOT 更新率
我想我可以通过改编来自 https://www.cybertec-postgresql.com/en/hot-updates-in-postgresql-for-better-performance/ 的 Laurenz Albe 的代码来稍微解决这个问题。我在这里所做的就是制作一个脚本，该脚本构建一个 table，FILLFACTOR 为 10、20、30.....100%，然后以相同的方式为每个百分比更新它.每次创建 table 时，它都会填充 256 条记录，然后每条记录更新 10 次。更新将非索引字段设置回自身，因此实际上没有值更改：
UPDATE mytable SET val = val;
结果如下：
+------------+---------------+-------------+--------------+
| FILLFACTOR | total_updates | hot_updates | total_to_hot |
+------------+---------------+-------------+--------------+
| 10 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 20 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 30 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 40 | 2350 | 2350 | 1.00 |
+------------+---------------+-------------+--------------+
| 50 | 2350 | 2223 | 1.06 |
+------------+---------------+-------------+--------------+
| 60 | 2350 | 2188 | 1.07 |
+------------+---------------+-------------+--------------+
| 70 | 2350 | 1883 | 1.25 |
+------------+---------------+-------------+--------------+
| 80 | 2350 | 1574 | 1.49 |
+------------+---------------+-------------+--------------+
| 90 | 2350 | 1336 | 1.76 |
+------------+---------------+-------------+--------------+
| 100 | 2350 | 987 | 2.38 |
+------------+---------------+-------------+--------------+
由此看来，当total_to_hot比率上升时，可能对增加FILLFACTOR有好处。
https://www.postgresql.org/docs/13/monitoring-stats.html
n_tup_upd 计算 所有 更新，包括 HOT 更新，而 n_tup_hot_upd 仅计算 HOT 更新。但它似乎并没有计算“如果我们没有 运行 页面上没有空间的话，这可能是一个热更新。”那太好了，但似乎也有很多要求。 （也许跟踪它的成本更高？）
这是脚本。我对每个 FILLFACTOR.
-- Set up the table for the test
DROP TABLE IF EXISTS mytable;
CREATE TABLE mytable (
id integer PRIMARY KEY,
val integer NOT NULL
) WITH (autovacuum_enabled = off);
-- Change the FILLFACTOR. The default is 100.
ALTER TABLE mytable SET (fillfactor = 10); -- The only part that changes between runs.
-- Seed the data
INSERT INTO mytable
SELECT *, 0
FROM generate_series(1, 235) AS n;
-- Thrash the data
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
UPDATE mytable SET val = val;
SELECT pg_sleep(1);
-- How does it look?
SELECT n_tup_upd as total_updates,
n_tup_hot_upd as hot_updates,
div_safe(n_tup_upd, n_tup_hot_upd) as total_to_hot
FROM pg_stat_user_tables
WHERE relname = 'mytable';
检查 FILLFACTOR 设置
作为旁注，我想快速打电话检查 table 上的 FILLFACTOR 设置，结果比我想象的要复杂得多。我写了一个有效的函数，但可能会看到一些改进......如果有人有建议的话。我这样称呼它：
select * from table_get_options('foo', 'bar');
或
select * from table_get_options('foo','bar') where option = 'fillfactor';
这是代码，如果有人可以提供改进：
CREATE OR REPLACE FUNCTION dba.table_get_options(text,text)
RETURNS TABLE (
schema_name text,
table_name text,
option text,
value text
)
LANGUAGE SQL AS
$BODY$
WITH
packed_options AS (
select pg_class.relname as table_name,
btrim(pg_options_to_table(pg_class.reloptions)::text, '()') as option_kvp -- Convert to text (fillfactor,95), then strip off ( and )
from pg_class
join pg_namespace
on pg_namespace.oid = pg_class.relnamespace
where pg_namespace.nspname =
and relname =
and reloptions is not null
),
unpacked_options AS (
select as schema_name,
as table_name,
split_part(option_kvp, ',', 1) as option,
split_part(option_kvp, ',', 2) as value
from packed_options
)
select * from unpacked_options;
$BODY$;
这些数字表明您的策略无效，并且绝大多数更新都不是 HOT。您还说明了原因：即使您将索引列更新为原始值，您也不会获得热更新。
解决方案是通过在 UPDATE 语句中包含索引列来区分，仅当它确实被修改时。
95 的 fillfactor 也相当高，除非您的表的行非常小。也许使用 90 或 85 之类的设置会获得更好的结果。