在没有模型的情况下在数据存储中查询

query in datastore without a model

我正在尝试在 appengine 中创建一个搜索键列表的应用程序,然后我使用此列表从数据存储中删除这些记录,此服务必须是通用服务,因此我无法使用模型只需按种类名称搜索,是否可以通过应用程序引擎功能进行搜索?

在我的代码下面,但它需要我有一个模型。

导入httplib 导入日志 从日期时间导入日期时间,timedelta

import webapp2
from google.appengine.api import urlfetch
from google.appengine.ext import ndb

DEFAULT_PAGE_SIZE = 100000
DATE_PATTERN = "%Y-%m-%dT%H:%M:%S"


def get_date(amount):
    date = datetime.today() - timedelta(days=30 * amount)
    date = date.replace(hour=0, minute=0, second=0)
    return date


class Purge(webapp2.RequestHandler):

    def get(self):
        kind = self.request.get('kind')
        datefield = self.request.get('datefield')
        amount = self.request.get('amount', default_value=3)
        date = get_date(amount)

        logging.info('Executando purge para Entity {}, mantendo periodo de {} meses.'.format(kind, amount))

        # cria a query
        query = ndb.Query(kind=kind, namespace='development')

        logging.info('Setando o filtro [{} <= {}]'.format(datefield, date.strftime(DATE_PATTERN)))

        # cria um filtro
        query.filter(ndb.DateTimeProperty(datefield) <= date)

        query.fetch_page(DEFAULT_PAGE_SIZE)

        while True:
            # executa a consulta
            keys = query.fetch(keys_only=True)

            logging.info('Encontrados {} {} a serem exluidos'.format(len(keys), kind))

            # exclui usando as keys
            ndb.delete_multi(keys)

            if len(keys) < DEFAULT_PAGE_SIZE:
                logging.info('Nao existem mais registros a serem excluidos')
                break


app = webapp2.WSGIApplication(
    [
        ('/cloud-datastore-purge', Purge),
    ], debug=True)

追踪

Traceback (most recent call last):
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/p~telefonica-dev-155211/cloud-datastore-purge-python:20180629t150020.410785498982375644/purge.py", line 38, in get
    query.fetch_page(_DEFAULT_PAGE_SIZE)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/utils.py", line 160, in positional_wrapper
    return wrapped(*args, **kwds)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 1362, in fetch_page
    return self.fetch_page_async(page_size, **q_options).get_result()   
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
    self.check_success()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 1380, in _fetch_page_async
    while (yield it.has_next_async()):
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
    value = gen.throw(exc.__class__, exc, tb)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 1793, in has_next_async
    yield self._fut
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 890, in helper
    batch, i, ent = yield inq.getq()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 969, in run_to_queue
    batch = yield rpc
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
    result = rpc.get_result()
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2951, in __query_result_hook
    self.__results = self._process_results(query_result.result_list())
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2984, in _process_results
    for result in results]
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 194, in pb_to_query_result
    return self.pb_to_entity(pb)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 690, in pb_to_entity
    modelclass = Model._lookup_model(kind, self.default_model)
  File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3101, in _lookup_model
    kind)
KindError: No model class found for kind 'Test'. Did you forget to import it?

在设置fetch_page的那一行发现了问题。

删除此行

query.fetch_page(DEFAULT_PAGE_SIZE)

为此

keys = query.fetch(limit=_DEFAULT_LIMIT, keys_only=True)

要运行 环境中没有可用模型 class 的数据存储区查询,您可以使用 google.appengine.api.datastore.Query class from the low-level datastore API

有关其他想法,请参阅

如果目标只是擦除实体而不考虑其种类,那么除了您自己指定 kinds/models 之外,您还有一些选择。

  • 您可以从您的模型文件中获取各种类型的列表并遍历它们,请参阅How to delete all the entries from google datastore?

  • 使用通用数据存储客户端库(不是 ndb),您可以从数据存储本身获取种类列表,请参阅 但我依稀记得一些问题 可能 转化为该库不再适用于第一代标准环境,所以我不能 100% 确定这一点。