Python KeyError: <peewee.IntegerField object at ... > using peewee insert_many()

Question

作为练习，我从 API 中提取数据并将其插入到 psql 数据库中。我最初遵循每次拉取 1000 个条目的默认限制，但我决定尝试获取大约 40K 行的所有数据。经过一些实验，我可以拉出 4800，但随后我得到以下信息：

Traceback (most recent call last):
  File "data_pull.py", line 19, in <module>
    postgres_db.Bike_Count.insert_many(data).execute()
  File "/usr/local/lib/python3.5/dist-packages/peewee.py", line 3516, in execute
    cursor = self._execute()
  File "/usr/local/lib/python3.5/dist-packages/peewee.py", line 2901, in _execute
    sql, params = self.sql()
  File "/usr/local/lib/python3.5/dist-packages/peewee.py", line 3484, in sql
    return self.compiler().generate_insert(self)
  File "/usr/local/lib/python3.5/dist-packages/peewee.py", line 2084, in generate_insert
    value = row_dict[field]
KeyError: <peewee.IntegerField object at 0x7f5b32c2c7f0>

data_pull.py

import json, requests, peewee
import postgres_db


endpoint =  'https://data.seattle.gov/resource/4xy5-26gy.json?$limit=4800'

response = requests.get(endpoint, headers={'X-App-Token': '(REMOVED)'})
if response.status_code == 200:
    data = json.loads(response.text)


postgres_db.Bike_Count.create_table(True)
postgres_db.Bike_Count.insert_many(data).execute()

postgres_db.py

import peewee


psql_db = peewee.PostgresqlDatabase('database', user='my_username')

class Bike_Count(peewee.Model):
    date = peewee.DateTimeField()
    fremont_bridge_sb = peewee.IntegerField()
    fremont_bridge_nb = peewee.IntegerField()

    class Meta:
        database = psql_db

我查看了在线表格，认为其中的条目存在问题，但我找不到任何明显的内容。感谢您的帮助。

Answer 1

我在本地尝试了您的代码（删除了应用程序令牌和 4800 限制）并且它按预期工作：

  id  |        date         | fremont_bridge_sb | fremont_bridge_nb 
------+---------------------+-------------------+-------------------
    1 | 2017-01-09 06:00:00 |                28 |                55
    2 | 2017-01-04 20:00:00 |                19 |                10
    3 | 2017-01-18 13:00:00 |                18 |                18
    4 | 2017-01-06 11:00:00 |                22 |                15
    5 | 2017-01-27 11:00:00 |                39 |                38
    6 | 2017-01-08 14:00:00 |                 6 |                10
    7 | 2017-01-06 23:00:00 |                 8 |                 3
    8 | 2017-01-27 13:00:00 |                45 |                35
...

当我运行它附加了 LIMIT 时，我注意到 API 返回的其中一行只包含一个 date 键（缺少 fremont_bridge_nb 和 fremont_bridge_sb 个字段）。

Peewee 要求批量插入时每一行都有相同的键，所以问题是 peewee 期望找到所有 3 个键。

Python KeyError: <peewee.IntegerField object at ... > using peewee insert_many()

Python KeyError: <peewee.IntegerField object at ... > using peewee insert_many()

python

postgresql

socrata

peewee