FastApi sqlalchemy 连接在运行中被关闭

Question

我有一个带有异步sqlalchemy的异步FastApi应用程序，源代码（不会提供schemas.py因为没有必要）：

database.py

from sqlalchemy import (
    Column,
    String,
)
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm.decl_api import DeclarativeMeta

from app.config import settings


engine = create_async_engine(settings.DATABASE_URL)
Base: DeclarativeMeta = declarative_base()
async_session = sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)


class Titles(Base):
    __tablename__ = "titles"
    id = Column(String(100), primary_key=True)
    title = Column(String(100), unique=True)


async def get_session() -> AsyncSession:
    async with async_session() as session:
        yield session

routers.py

import .database
from fastapi_utils.cbv import cbv
from fastapi_utils.inferring_router import InferringRouter


router = InferringRouter()


async def get_titles(session: AsyncSession):
    results = await session.execute(select(database.Titles)))
    return results.scalars().all()


@cbv(router)
class TitlesView:
    session: AsyncSession = Depends(database.get_session)

    @router.get("/titles", status_code=HTTP_200_OK)
    async def get(self) -> List[TitlesSchema]:
        results = await get_titles(self.session)
        return [TitlesSchema.from_orm(result) for result in results]

main.py

from fastapi import FastAPI

from app.routers import router 


def create_app() -> FastAPI:
    app = FastAPI()
    app .include_router(routers, prefix="/", tags=["Titles"])

    return printer_app


app = create_app()

它运行 docker:

CMD ["uvicorn", "main:app", "--reload", "--host", "0.0.0.0", "--port", "8000", "--limit-max-requests", "10000"]

并且它在 docker 中也有默认设置的 Postgres 数据库。这一切都在 docker-swarm 上运行。一开始工作正常，接受所有请求。但是如果你放15-30分钟（我没算），然后再request，就不行了：

<class 'asyncpg.exceptions.ConnectionDoesNotExistError'>: connection was closed in the middle of operation

紧接着我发送了下一个请求，它没有抛出错误。会是什么呢？如何摆脱 ConnectionDoesNotExistError？

Answer 1

我引用一下here的回答，我觉得可能有用。全部归功于 q210。

In our case, the root cause was that ipvs, used by swarm to route packets, have default expiration time for idle connections set to 900 seconds. So if connection had no activity for more than 15 minutes, ipvs broke it. 900 seconds is significantly less than default linux tcp keepalive setting (7200 seconds) used by most of the services that can send keepalive tcp packets to keep connections from going idle.

The same problem is described here moby/moby#31208

To fix this we had to set the following in postgresql.conf:
tcp_keepalives_idle = 600               # TCP_KEEPIDLE, in seconds;
                                       # 0 selects the system default
tcp_keepalives_interval = 30            # TCP_KEEPINTVL, in seconds;
                                       # 0 selects the system default
tcp_keepalives_count = 10               # TCP_KEEPCNT;
                                       # 0 selects the system default
These settings are forcing PostgreSQL to keep connections from going idle by sending keepalive packets more often than ipvs default setting (that we can't change in docker-swarm, sadly).

I guess the same could be achieved by changing corresponding linux settings (net.ipv4.tcp_keepalive_time and the like), 'cause PostgreSQL uses them by default, but in our case changing these was a bit more cumbersome.

Answer 2

我使用 pool_pre_ping 设置解决了这个问题：

engine = create_engine(DB_URL, pool_pre_ping=True)

https://docs.sqlalchemy.org/en/14/core/pooling.html

FastApi sqlalchemy 连接在运行中被关闭

FastApi sqlalchemy Connection was closed in the middle of operation

python

sqlalchemy

asyncpg

fastapi

uvicorn

database.py

routers.py

main.py