Alembic：使用“version_table_schema”时 'relation "public.alembic_version" does not exist'
Alembic: 'relation "public.alembic_version" does not exist' when using `version_table_schema`
我正在为 Alembic 编写一些自定义代码，以使我的数据库在我的项目开发环境中始终保持更新。项目涉及数据库如下：
- 共享数据的
public 模式
- 每个客户端一个模式 "database"
- 作为所有客户端模式（组织）
prototype的一个模式
此刻，我并不担心多个客户端模式，只担心 public 和 prototype 模式是最新的。我的 env.py 脚本适用于 public 模式，但不适用于 prototype，因为在使用 [=19] 时，alembic 试图使用 public 中的 table 版本=].
所以，我想我可以使用 version_table_schema 选项在 public 模式中维护一个版本 table，在 prototype 模式中维护一个版本。但是，一旦我开始使用它，我就会在尝试升级时收到“关系 "public.alembic_version" 不存在”错误。
我看到的唯一区别是，当我使用 version_table_schema 设置为适当的模式时，生成的修订脚本实际上包含一行到 op.drop_table('alembic_version')。该行仅在使用 version_table_schema 时存在。
我希望我只是漏掉了一些小东西。
以下是所有相关的源文件，供参考。唯一的外部依赖应该是 appcontext 和 getDeclarativeBase。 appcontext 仅用于配置并且绝对有效（数据库连接正常）。 getDeclarativeBase 是一种动态获取模式的声明性基础（和相关元数据）的方法。根据调试输出，这似乎也能正常工作。元数据对象本身在构造时与正确的模式相关联。
运行 迁移的包装函数。 autoMigratePublic() 和 autoMigrateOrgProto() 在这种情况下是有问题的方法
""" A wrapper around Alembic that will assist with database migrations.
Our database system is soemwhat complex to upgrade. There is a public
schema which we use for shared things (such as authentication and job
management). Then there is the org prototype.
There are also individual schemas for each client org. These are trickier.
Also useful:
https://github.com/zzzeek/alembic/blob/0e3319bb36f1612e41a8d7da5a48ce1ca33a0b2b/alembic/config.py#L476
"""
import logging
import os
import time
import alembic.config
import alembic.util
CONFIG_ORG_PROTO = os.path.join("migrations", "alembic_orgproto.ini")
CONFIG_PUBLIC = os.path.join("migrations", "alembic_public.ini")
log = logging.getLogger("sys.migrations")
def autoMigratePublic():
try:
freezePublic()
except alembic.util.CommandError:
log.warn("[public] Problem creating migration revision. Skipping as this sometimes just means that we are working with a new DB.")
upgradePublic()
return
def autoMigrateOrgProto():
try:
freezeOrgProto()
except alembic.util.CommandError:
log.warn("[orgproto] Problem creating migration revision. Skipping as this sometimes just means that we are working with a new DB.")
upgradeOrgProto()
return
def freezePublic():
log.info("[public] Checking the database for revisions...")
alembicArgs = [
"--config", CONFIG_PUBLIC,
"--raiseerr",
"revision",
"--autogenerate",
"--message", "autogenerate {0}".format(makeRevisionName()),
]
runAlembic(alembicArgs)
return
def freezeOrgProto():
log.info("[orgproto] Checking the database for revisions...")
alembicArgs = [
"--config", CONFIG_ORG_PROTO,
"--raiseerr",
"revision",
"--autogenerate",
"--message", "autogenerate {0}".format(makeRevisionName()),
]
runAlembic(alembicArgs)
return
def makeRevisionName():
return time.strftime('%Y-%m-%d %H:%M:%S')
def upgradePublic():
log.info("[public] Performing database upgrade...")
alembicArgs = [
"--config", CONFIG_PUBLIC,
"--raiseerr",
"upgrade",
"head",
]
runAlembic(alembicArgs)
return
def upgradeOrgProto():
log.info("[orgproto] Performing database upgrade...")
alembicArgs = [
"--config", CONFIG_ORG_PROTO,
"--raiseerr",
"upgrade",
"head",
]
runAlembic(alembicArgs)
return
def upgradeOrg(schemaName):
log.info("[%s] Performing database upgrade...", schemaName)
alembicArgs = [
"--config", CONFIG_ORG_PROTO,
"--raiseerr",
"upgrade",
"head",
"-x", "schema={0}".format(schemaName),
]
runAlembic(alembicArgs)
return
def runAlembic(args):
return alembic.config.main(args)
Class 用于执行迁移。 env.py 文件
中引用了此 class
import copy
import logging
import os
import re
import traceback
from logging.config import fileConfig
from sqlalchemy import create_engine
import core.context.appcontext
from core.database.declarative import getDeclarativeBase
logging.getLogger("alembic").setLevel(logging.DEBUG)
#==============================================================================
class Migrator(object):
def __init__(self):
from alembic import context
self.context = context
self.config = context.config
self.log = logging.getLogger("sys.migrations")
self.sys = core.context.appcontext.instance()
self.schema = self.config.get_main_option("schema")
if self.context.get_x_argument("schema"):
self.schema = self.context.get_x_argument("schema")
return
def start(self):
import core.database.tables # Make sure the metadata is defined
if self.context.is_offline_mode():
self.log.error("[%s] Can't run migrations offline", self.schema)
return
self.doMigration()
return
def doMigration(self):
targetMetadata = getDeclarativeBase(self.schema).metadata
engine = create_engine(self.sys.getConnectionUrl(), echo=False)
self.log.info("[%s] Engine: %s", self.schema, engine)
self.log.debug("[%s] Metadata: %s", self.schema, targetMetadata)
for t in targetMetadata.sorted_tables:
self.log.debug(" - %s", t)
conn = engine.connect()
try:
self.context.configure(
conn,
version_table_schema=self.schema,
target_metadata=targetMetadata,
process_revision_directives=self.process_revision_directives,
)
with self.context.begin_transaction():
self.context.run_migrations()
finally:
conn.close()
return
def process_revision_directives(self, context, revision, directives):
""" Used to prevent creating empty migrations
See: http://alembic.readthedocs.org/en/latest/cookbook.html#don-t-generate-empty-migrations-with-autogenerate
"""
if self.config.cmd_opts.autogenerate:
script = directives[0]
if script.upgrade_ops.is_empty():
self.log.debug("[%s] Auto-generated migration is empty. No migration file will be created.", self.schema)
directives[:] = []
else:
self.log.info("[%s] Creating new auto-generated migration revision.", self.schema)
return
样本env.py.public和原型升级的内容相同
from migrations.migrator import Migrator
Migrator().start()
示例配置 ini public 和原型迁移使用几乎完全相同的文件
# A generic, single database configuration.
[alembic]
# path to migration scripts
script_location = migrations/orgproto
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
# max length of characters to apply to the
# "slug" field
#truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; this defaults
# to alembic/versions. When using multiple version
# directories, initial revisions must be specified with --version-path
# version_locations = %(here)s/bar %(here)s/bat alembic/versions
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = <Not Loaded>
schema = orgproto
Mako 文件 我使用的是默认值。
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
def upgrade():
${upgrades if upgrades else "pass"}
def downgrade():
${downgrades if downgrades else "pass"}
我在 sqlalchemy-alembic google group 上收到了解决该问题的回复。将它张贴在这里以防它对其他人有帮助：
you're likely hitting the very confusing schema rules that apply to
Postgresql. See
http://docs.sqlalchemy.org/en/rel_1_0/dialects/postgresql.html#remote-schema-table-introspection-and-postgresql-search-path
for details. Short answer is that schema of "blank" and schema of
"public" are two different things on the Python side, leading to a lot
of confusion.
In order to convince autogenerate to not affect alembic_version at all
no matter where it pops up, you probably need to create an exclusion
rule using include_object:
http://alembic.readthedocs.org/en/latest/api/runtime.html?highlight=include_object#alembic.runtime.environment.EnvironmentContext.configure.params.include_object
def include_object(object, name, type_, reflected, compare_to):
if (type_ == "table" and name == 'alembic_version'):
return False
else:
return True
在我们的案例中，帮助将 alembic_version 添加到 migrations/alembic.ini 的 [alembic:exclude] 部分：
[alembic:exclude]
- tables = spatial_ref_sys,topology,layer
+ tables = spatial_ref_sys,topology,layer,alembic_version
migrations/env.py 中的下一个：
context.configure(
version_table_schema='public',
# other params
)
我正在为 Alembic 编写一些自定义代码，以使我的数据库在我的项目开发环境中始终保持更新。项目涉及数据库如下：
- 共享数据的
public模式 - 每个客户端一个模式 "database"
- 作为所有客户端模式（组织）
prototype的一个模式
此刻，我并不担心多个客户端模式，只担心 public 和 prototype 模式是最新的。我的 env.py 脚本适用于 public 模式，但不适用于 prototype，因为在使用 [=19] 时，alembic 试图使用 public 中的 table 版本=].
所以，我想我可以使用 version_table_schema 选项在 public 模式中维护一个版本 table，在 prototype 模式中维护一个版本。但是，一旦我开始使用它，我就会在尝试升级时收到“关系 "public.alembic_version" 不存在”错误。
我看到的唯一区别是，当我使用 version_table_schema 设置为适当的模式时，生成的修订脚本实际上包含一行到 op.drop_table('alembic_version')。该行仅在使用 version_table_schema 时存在。
我希望我只是漏掉了一些小东西。
以下是所有相关的源文件，供参考。唯一的外部依赖应该是 appcontext 和 getDeclarativeBase。 appcontext 仅用于配置并且绝对有效（数据库连接正常）。 getDeclarativeBase 是一种动态获取模式的声明性基础（和相关元数据）的方法。根据调试输出，这似乎也能正常工作。元数据对象本身在构造时与正确的模式相关联。
运行 迁移的包装函数。 autoMigratePublic() 和 autoMigrateOrgProto() 在这种情况下是有问题的方法
""" A wrapper around Alembic that will assist with database migrations.
Our database system is soemwhat complex to upgrade. There is a public
schema which we use for shared things (such as authentication and job
management). Then there is the org prototype.
There are also individual schemas for each client org. These are trickier.
Also useful:
https://github.com/zzzeek/alembic/blob/0e3319bb36f1612e41a8d7da5a48ce1ca33a0b2b/alembic/config.py#L476
"""
import logging
import os
import time
import alembic.config
import alembic.util
CONFIG_ORG_PROTO = os.path.join("migrations", "alembic_orgproto.ini")
CONFIG_PUBLIC = os.path.join("migrations", "alembic_public.ini")
log = logging.getLogger("sys.migrations")
def autoMigratePublic():
try:
freezePublic()
except alembic.util.CommandError:
log.warn("[public] Problem creating migration revision. Skipping as this sometimes just means that we are working with a new DB.")
upgradePublic()
return
def autoMigrateOrgProto():
try:
freezeOrgProto()
except alembic.util.CommandError:
log.warn("[orgproto] Problem creating migration revision. Skipping as this sometimes just means that we are working with a new DB.")
upgradeOrgProto()
return
def freezePublic():
log.info("[public] Checking the database for revisions...")
alembicArgs = [
"--config", CONFIG_PUBLIC,
"--raiseerr",
"revision",
"--autogenerate",
"--message", "autogenerate {0}".format(makeRevisionName()),
]
runAlembic(alembicArgs)
return
def freezeOrgProto():
log.info("[orgproto] Checking the database for revisions...")
alembicArgs = [
"--config", CONFIG_ORG_PROTO,
"--raiseerr",
"revision",
"--autogenerate",
"--message", "autogenerate {0}".format(makeRevisionName()),
]
runAlembic(alembicArgs)
return
def makeRevisionName():
return time.strftime('%Y-%m-%d %H:%M:%S')
def upgradePublic():
log.info("[public] Performing database upgrade...")
alembicArgs = [
"--config", CONFIG_PUBLIC,
"--raiseerr",
"upgrade",
"head",
]
runAlembic(alembicArgs)
return
def upgradeOrgProto():
log.info("[orgproto] Performing database upgrade...")
alembicArgs = [
"--config", CONFIG_ORG_PROTO,
"--raiseerr",
"upgrade",
"head",
]
runAlembic(alembicArgs)
return
def upgradeOrg(schemaName):
log.info("[%s] Performing database upgrade...", schemaName)
alembicArgs = [
"--config", CONFIG_ORG_PROTO,
"--raiseerr",
"upgrade",
"head",
"-x", "schema={0}".format(schemaName),
]
runAlembic(alembicArgs)
return
def runAlembic(args):
return alembic.config.main(args)
Class 用于执行迁移。 env.py 文件
中引用了此 classimport copy
import logging
import os
import re
import traceback
from logging.config import fileConfig
from sqlalchemy import create_engine
import core.context.appcontext
from core.database.declarative import getDeclarativeBase
logging.getLogger("alembic").setLevel(logging.DEBUG)
#==============================================================================
class Migrator(object):
def __init__(self):
from alembic import context
self.context = context
self.config = context.config
self.log = logging.getLogger("sys.migrations")
self.sys = core.context.appcontext.instance()
self.schema = self.config.get_main_option("schema")
if self.context.get_x_argument("schema"):
self.schema = self.context.get_x_argument("schema")
return
def start(self):
import core.database.tables # Make sure the metadata is defined
if self.context.is_offline_mode():
self.log.error("[%s] Can't run migrations offline", self.schema)
return
self.doMigration()
return
def doMigration(self):
targetMetadata = getDeclarativeBase(self.schema).metadata
engine = create_engine(self.sys.getConnectionUrl(), echo=False)
self.log.info("[%s] Engine: %s", self.schema, engine)
self.log.debug("[%s] Metadata: %s", self.schema, targetMetadata)
for t in targetMetadata.sorted_tables:
self.log.debug(" - %s", t)
conn = engine.connect()
try:
self.context.configure(
conn,
version_table_schema=self.schema,
target_metadata=targetMetadata,
process_revision_directives=self.process_revision_directives,
)
with self.context.begin_transaction():
self.context.run_migrations()
finally:
conn.close()
return
def process_revision_directives(self, context, revision, directives):
""" Used to prevent creating empty migrations
See: http://alembic.readthedocs.org/en/latest/cookbook.html#don-t-generate-empty-migrations-with-autogenerate
"""
if self.config.cmd_opts.autogenerate:
script = directives[0]
if script.upgrade_ops.is_empty():
self.log.debug("[%s] Auto-generated migration is empty. No migration file will be created.", self.schema)
directives[:] = []
else:
self.log.info("[%s] Creating new auto-generated migration revision.", self.schema)
return
样本env.py.public和原型升级的内容相同
from migrations.migrator import Migrator
Migrator().start()
示例配置 ini public 和原型迁移使用几乎完全相同的文件
# A generic, single database configuration.
[alembic]
# path to migration scripts
script_location = migrations/orgproto
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
# max length of characters to apply to the
# "slug" field
#truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; this defaults
# to alembic/versions. When using multiple version
# directories, initial revisions must be specified with --version-path
# version_locations = %(here)s/bar %(here)s/bat alembic/versions
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = <Not Loaded>
schema = orgproto
Mako 文件 我使用的是默认值。
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
def upgrade():
${upgrades if upgrades else "pass"}
def downgrade():
${downgrades if downgrades else "pass"}
我在 sqlalchemy-alembic google group 上收到了解决该问题的回复。将它张贴在这里以防它对其他人有帮助：
you're likely hitting the very confusing schema rules that apply to Postgresql. See http://docs.sqlalchemy.org/en/rel_1_0/dialects/postgresql.html#remote-schema-table-introspection-and-postgresql-search-path for details. Short answer is that schema of "blank" and schema of "public" are two different things on the Python side, leading to a lot of confusion.
In order to convince autogenerate to not affect alembic_version at all no matter where it pops up, you probably need to create an exclusion rule using include_object: http://alembic.readthedocs.org/en/latest/api/runtime.html?highlight=include_object#alembic.runtime.environment.EnvironmentContext.configure.params.include_object
def include_object(object, name, type_, reflected, compare_to): if (type_ == "table" and name == 'alembic_version'): return False else: return True
在我们的案例中，帮助将 alembic_version 添加到 migrations/alembic.ini 的 [alembic:exclude] 部分：
[alembic:exclude]
- tables = spatial_ref_sys,topology,layer
+ tables = spatial_ref_sys,topology,layer,alembic_version
migrations/env.py 中的下一个：
context.configure(
version_table_schema='public',
# other params
)