Asyncpg 池等待所有结果
Asyncpg pool waits for all results
我使用 asyncpg 进行繁重的查询，通常需要大约 15-20 秒的时间来计算。我正在试验池的最小和最大大小，并注意到一个奇怪的行为。当我尝试发出多个请求时，它们是同时执行的，但结果是 return 批处理。批处理始终与池大小相同。此行为会导致第一次响应时间过长，例如 50 秒左右。第一批之后，我需要再等 50 秒才能得到其他结果。似乎 asyncpg 等待从池中收集所有结果并 return 它们，而不是在它们准备好后立即 returning 结果。
有没有办法让单个请求一执行就得到结果？也许我缺少某些设置？
有一段时间我可以保持 25 个连接，但这似乎是一种 hack，我真的不喜欢。
如果这很重要，我会使用 DSN 创建池。
db_pool = await asyncpg.create_pool(loop=self._loop, **params)
这是我用来向数据库发出请求的代码：
async def first(self, query: str, values: List, db_name: str = 'default'):
pool = self.get_pool(db_name)
async with pool.acquire() as conn:
return await conn.fetchrow(query, *values)
现在我正在测试默认池大小（最大 10，10 分钟）并同时发送 25 个请求。
result for 336997: 45.43796348571777 s
result for 368406: 45.43796348571777 s
result for 288307: 45.81912803649902 s
result for 283236: 46.499717235565186 s
result for 296000: 49.140310764312744 s
result for 304671: 49.20531177520752 s
result for 283685: 49.26837992668152 s
result for 283772: 49.3363881111145 s
result for 283720: 49.3753764629364 s
result for 294811: 49.39737892150879 s
result for 325604: 112.60201215744019 s
result for 336997: 112.60101222991943 s
result for 283028: 112.62509346008301 s
result for 291122: 113.41229104995728 s
result for 281105: 115.48561716079712 s
result for 304874: 115.59060764312744 s
result for 281875: 115.73372554779053 s
result for 283219: 115.73472547531128 s
result for 312094: 116.00303101539612 s
result for 312094: 116.0290174484253 s
result for 368406: 157.77449679374695 s
result for 325604: 157.77449679374695 s
result for 281932: 157.79654741287231 s
result for 290687: 157.79554748535156 s
result for 304874: 158.38678884506226 s
如果我将最大值和最小值设置为 25，当我得到这些结果时：
result for 368406: 96.23042106628418 s
result for 368406: 96.22842144966125 s
result for 283236: 97.59920930862427 s
result for 304671: 99.69211030006409 s
result for 281932: 107.54676508903503 s
result for 283685: 107.95523738861084 s
result for 281875: 108.28549408912659 s
result for 283720: 108.39060115814209 s
result for 283028: 108.388601064682 s
result for 296000: 108.44459056854248 s
result for 283772: 108.55759739875793 s
result for 291122: 108.59360837936401 s
result for 294811: 108.90663266181946 s
result for 336997: 109.07162356376648 s
result for 325604: 109.07562470436096 s
result for 325604: 109.17762279510498 s
result for 336997: 109.30463767051697 s
result for 312094: 109.40563464164734 s
result for 312094: 109.40663456916809 s
result for 281105: 109.63970899581909 s
result for 290687: 109.66070008277893 s
result for 304874: 109.66170001029968 s
result for 288307: 109.68773555755615 s
result for 304874: 109.68273568153381 s
result for 283219: 109.68573522567749 s
5/5 连接带来糟糕的结果
result for 288307: 38.87580060958862 s
result for 325604: 38.87280225753784 s
result for 283219: 38.87380290031433 s
result for 283772: 38.98385691642761 s
result for 304671: 39.80011057853699 s
result for 368406: 94.94180464744568 s
result for 283720: 94.94180464744568 s
result for 296000: 94.98080492019653 s
result for 294811: 95.45388603210449 s
result for 283685: 95.85135459899902 s
result for 291122: 156.6862394809723 s
result for 325604: 156.68523907661438 s
result for 336997: 156.68724012374878 s
result for 304874: 156.8772156238556 s
result for 283236: 157.72206735610962 s
result for 336997: 219.94082736968994 s
result for 368406: 219.96384191513062 s
result for 312094: 220.39882922172546 s
result for 281105: 220.8170771598816 s
result for 283028: 221.18352794647217 s
result for 290687: 283.105571269989 s
result for 281932: 283.10657024383545 s
result for 304874: 283.105571269989 s
result for 281875: 283.21556425094604 s
result for 312094: 283.7060844898224 s
每个查询只在等待自己的结果。如果所有查询花费的工作量大致相同，那么它们（实际上是 运行 的查询）都在大约相同的时间后完成。无论如何也不是那么相似，从 45.4 到 49.4 的差距并不小。如果它以某种方式故意同步它们，您会认为它会做得更好。
For some time I could keep 25 connections, but it seems like a hack, which I really don't like.
你更愿意做什么？使用较低的数字？更高的？一个每分钟都在变化的随机数？您应该限制连接池。为什么 25 是黑客攻击而其他一些数字却不是？
这是数据库端的问题，我们不得不增加资源来解决这个问题。
我使用 asyncpg 进行繁重的查询，通常需要大约 15-20 秒的时间来计算。我正在试验池的最小和最大大小，并注意到一个奇怪的行为。当我尝试发出多个请求时，它们是同时执行的，但结果是 return 批处理。批处理始终与池大小相同。此行为会导致第一次响应时间过长，例如 50 秒左右。第一批之后，我需要再等 50 秒才能得到其他结果。似乎 asyncpg 等待从池中收集所有结果并 return 它们，而不是在它们准备好后立即 returning 结果。
有没有办法让单个请求一执行就得到结果？也许我缺少某些设置？
有一段时间我可以保持 25 个连接，但这似乎是一种 hack，我真的不喜欢。
如果这很重要，我会使用 DSN 创建池。
db_pool = await asyncpg.create_pool(loop=self._loop, **params)
这是我用来向数据库发出请求的代码：
async def first(self, query: str, values: List, db_name: str = 'default'):
pool = self.get_pool(db_name)
async with pool.acquire() as conn:
return await conn.fetchrow(query, *values)
现在我正在测试默认池大小（最大 10，10 分钟）并同时发送 25 个请求。
result for 336997: 45.43796348571777 s
result for 368406: 45.43796348571777 s
result for 288307: 45.81912803649902 s
result for 283236: 46.499717235565186 s
result for 296000: 49.140310764312744 s
result for 304671: 49.20531177520752 s
result for 283685: 49.26837992668152 s
result for 283772: 49.3363881111145 s
result for 283720: 49.3753764629364 s
result for 294811: 49.39737892150879 s
result for 325604: 112.60201215744019 s
result for 336997: 112.60101222991943 s
result for 283028: 112.62509346008301 s
result for 291122: 113.41229104995728 s
result for 281105: 115.48561716079712 s
result for 304874: 115.59060764312744 s
result for 281875: 115.73372554779053 s
result for 283219: 115.73472547531128 s
result for 312094: 116.00303101539612 s
result for 312094: 116.0290174484253 s
result for 368406: 157.77449679374695 s
result for 325604: 157.77449679374695 s
result for 281932: 157.79654741287231 s
result for 290687: 157.79554748535156 s
result for 304874: 158.38678884506226 s
如果我将最大值和最小值设置为 25，当我得到这些结果时：
result for 368406: 96.23042106628418 s
result for 368406: 96.22842144966125 s
result for 283236: 97.59920930862427 s
result for 304671: 99.69211030006409 s
result for 281932: 107.54676508903503 s
result for 283685: 107.95523738861084 s
result for 281875: 108.28549408912659 s
result for 283720: 108.39060115814209 s
result for 283028: 108.388601064682 s
result for 296000: 108.44459056854248 s
result for 283772: 108.55759739875793 s
result for 291122: 108.59360837936401 s
result for 294811: 108.90663266181946 s
result for 336997: 109.07162356376648 s
result for 325604: 109.07562470436096 s
result for 325604: 109.17762279510498 s
result for 336997: 109.30463767051697 s
result for 312094: 109.40563464164734 s
result for 312094: 109.40663456916809 s
result for 281105: 109.63970899581909 s
result for 290687: 109.66070008277893 s
result for 304874: 109.66170001029968 s
result for 288307: 109.68773555755615 s
result for 304874: 109.68273568153381 s
result for 283219: 109.68573522567749 s
5/5 连接带来糟糕的结果
result for 288307: 38.87580060958862 s
result for 325604: 38.87280225753784 s
result for 283219: 38.87380290031433 s
result for 283772: 38.98385691642761 s
result for 304671: 39.80011057853699 s
result for 368406: 94.94180464744568 s
result for 283720: 94.94180464744568 s
result for 296000: 94.98080492019653 s
result for 294811: 95.45388603210449 s
result for 283685: 95.85135459899902 s
result for 291122: 156.6862394809723 s
result for 325604: 156.68523907661438 s
result for 336997: 156.68724012374878 s
result for 304874: 156.8772156238556 s
result for 283236: 157.72206735610962 s
result for 336997: 219.94082736968994 s
result for 368406: 219.96384191513062 s
result for 312094: 220.39882922172546 s
result for 281105: 220.8170771598816 s
result for 283028: 221.18352794647217 s
result for 290687: 283.105571269989 s
result for 281932: 283.10657024383545 s
result for 304874: 283.105571269989 s
result for 281875: 283.21556425094604 s
result for 312094: 283.7060844898224 s
每个查询只在等待自己的结果。如果所有查询花费的工作量大致相同，那么它们（实际上是 运行 的查询）都在大约相同的时间后完成。无论如何也不是那么相似，从 45.4 到 49.4 的差距并不小。如果它以某种方式故意同步它们，您会认为它会做得更好。
For some time I could keep 25 connections, but it seems like a hack, which I really don't like.
你更愿意做什么？使用较低的数字？更高的？一个每分钟都在变化的随机数？您应该限制连接池。为什么 25 是黑客攻击而其他一些数字却不是？
这是数据库端的问题，我们不得不增加资源来解决这个问题。