如何防止繁琐的查询在 Postgrex 中超时？

Question

我正在运行查询并使用 Postgrex 将结果加载到流中，如下所示：

{:ok, host_pid} = Postgrex.start_link(hostname: "somewhere.hostname.io", username: "myuser", password: "mypass", database: "mydb")

Postgrex.transaction(host_pid, fn(conn) ->
    # do query that takes 5 seconds
    # with timeout set to be really big
    query = Postgrex.prepare!(conn, "", "SELECT pg_sleep(5)", timeout: 50_000)
    stream = Postgrex.stream(conn, query)
    result_to_iodata = fn(%Postgrex.Result{rows: rows}) -> format_query_result(rows) end
    Enum.into(stream, File.stream!("eg"), result_to_iodata)
end)

但我收到以下错误：

localhost$ mix run lib/MyPostgrexScript.exs 
** (DBConnection.ConnectionError) connection not available and request was dropped 
from queue after 2950ms. You can configure how long requests wait in the queue 
using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information
    (db_connection) lib/db_connection.ex:836: DBConnection.transaction/3
    lib/MyPostgrexScript.exs:3: MyPostgrexModule.sleep/0
    (elixir) lib/code.ex:767: Code.require_file/2
    (mix) lib/mix/tasks/run.ex:147: Mix.Tasks.Run.run/5

因为我想做一些繁琐的查询，到运行肯定需要超过 2950 毫秒，我想知道如何配置 Postgrex 让我的查询花费更多时间。我在 https://hexdocs.pm/postgrex/Postgrex.html#transaction/3 阅读了 :timeout 选项，但我不确定如何包含它，或者它是否是我正在寻找的。

非常感谢任何指导，谢谢！

Answer 1

I read about the :timeout option at https://hexdocs.pm/postgrex/Postgrex.html#transaction/3 but I'm not sure to how include it,

像这样（见最后一行）：

Postgrex.transaction(
    host_pid,

    fn(conn) ->
        # do query that takes 5 seconds
        # with timeout set to be really big
        query = Postgrex.prepare!(conn, "", "SELECT pg_sleep(5)", timeout: 50_000)
        stream = Postgrex.stream(conn, query)
        result_to_iodata = fn(%Postgrex.Result{rows: rows}) ->
                                 format_query_result(rows) 
                           end
        Enum.into(stream, File.stream!("eg"), result_to_iodata)
    end,

    timeout: 30_000  #30 seconds
)

每当长生不老药文档定义这样的函数时：

func_name(arg1, ...argN, opts \ [] )

opts是关键字列表，例如：

[{:a, 1}, {:b, 2}]

但是，如果关键字列表是函数调用中的最后一个参数，则关键字列表可以这样写：

func(arg1, arg2, a: 1, b: 2)

并且函数定义将接收三个个参数

arg1, arg2, [{:a, 1}, {:b, 2}]

在任何情况下，:timeout 的默认值为：

:timeout - Transaction timeout (default: 15000);

错误提示：

connection not available and request was dropped from queue after 2950ms

因为 2950 < 15000 :timeout 值似乎不是错误的来源。

错误信息继续：

connection not available.... You can configure how long requests wait in the queue using :queue_target and :queue_interval. See DBConnection.start_link/2 for more information

This 解释了如何配置这些超时：

In config/<env>.exs (where <env> is dev, test, or prod):
config :my_app, MyApp.Repo,
  adapter: Ecto.Adapters.Postgres,
  pool_size: 10,
  migration_timestamps: [type: :utc_datetime_usec],
  migration_lock: nil,
  queue_target: 5000
Is what we had to do recently due to increased number of errors in production.

Also,

Handling requests is done through a queue. When DBConnection is started, there are two relevant options to control the queue:
:queue_target in milliseconds, defaults to 50ms
:queue_interval in milliseconds, defaults to 1000ms
Our goal is to wait at most :queue_target for a connection. If all connections checked out during a :queue_interval takes more than :queue_target, then we double the :queue_target. If checking out connections take longer than the new target, then we start dropping messages.

For example, by default our target is 50ms. If all connections checkouts take longer than 50ms for a whole second, we double the target to 100ms and we start dropping messages if the time to checkout goes above the new limit.

This allows us to better plan for overloads as we can refuse requests before they are sent to the database, which would otherwise increase the burden on the database, making the overload worse.

但是，如果您没有触及这些默认值，那么我想知道您为什么会看到 2950ms 在错误消息中而不是接近 50 毫秒或 1000 毫秒。

如何防止繁琐的查询在 Postgrex 中超时？

How do I prevent a cumbersome query from timing out in Postgrex?

elixir

ecto

postgrex