pandas json 使用特定 json 属性规范化密钥错误

Question

我有一个 json 作为：

mytestdata = {
    "success": True,
    "message": "",
    "data": {
        "totalCount": 95,
        "goal": [
            {
                "user_id": 123455,
                "user_email": "john.smith@test.com",
                "user_first_name": "John",
                "user_last_name": "Smith",
                "people_goals": [
                    {
                        "goal_id": 545555,
                        "goal_name": "test goal name",
                        "goal_owner": "123455",
                        "goal_narrative": "",
                        "goal_type": {
                            "id": 1,
                            "name": "Team"
                        },
                        "goal_create_at": "1595874095",
                        "goal_modified_at": "1595874095",
                        "goal_created_by": "123455",
                        "goal_updated_by": "123455",
                        "goal_start_date": "1593561600",
                        "goal_target_date": "1601424000",
                        "goal_progress": "34",
                        "goal_progress_color": "#ff9933",
                        "goal_status": "1",
                        "goal_permission": "internal,team",
                        "goal_category": [],
                        "goal_owner_full_name": "John Smith",
                        "goal_team_id": "766754",
                        "goal_team_name": "",
                        "goal_workstreams": []
                    }
                ]
            }
        ]
    }
}

我正在尝试显示“people_goals”中的所有详细信息以及“user_last_name”、“user_first_name”、“user_email”、“[=31” =]" 和 json_normalize。到目前为止，我可以使用代码

显示“people_goals”、“user_first_name”、“user_email”

df2 = pd.json_normalize(data=mytestdata['data'], record_path=['goal', 'people_goals'], 
meta=[['goal','user_first_name'], ['goal','user_last_name'], ['goal','user_email']], errors='ignore')

但是我在尝试将 ['goal'、'user_id'] 包含在 meta=[] 中时遇到问题错误是：

TypeError                                 Traceback (most recent call last)
<ipython-input-192-b7a124a075a0> in <module>
      7 df2 = pd.json_normalize(data=mytestdata['data'], record_path=['goal', 'people_goals'], 
      8                         meta=[['goal','user_first_name'], ['goal','user_last_name'], ['goal','user_email'], ['goal','user_id']],
----> 9                         errors='ignore')
     10 
     11 # df2 = pd.json_normalize(data=mytestdata['data'], record_path=['goal', 'people_goals'])

我看到 'user_id' 的唯一区别是它不是字符串我在这里遗漏了什么吗？

Answer 1

您的代码可以在我的平台上运行。由于两个原因，我已经不再使用 record_path 和 meta 参数。 a) 它们很难解决 b) pandas

版本之间存在兼容性问题

因此我现在使用多次使用json_normalize()的方法逐步扩展JSON。或者使用 pd.Series。已将两者作为示例。

df = pd.json_normalize(data=mytestdata['data']).explode("goal")
df = pd.concat([df, df["goal"].apply(pd.Series)], axis=1).drop(columns="goal").explode("people_goals")
df = pd.concat([df, df["people_goals"].apply(pd.Series)], axis=1).drop(columns="people_goals")
df = pd.concat([df, df["goal_type"].apply(pd.Series)], axis=1).drop(columns="goal_type")
df.T

df2 = pd.json_normalize(pd.json_normalize(
    pd.json_normalize(data=mytestdata['data']).explode("goal").to_dict(orient="records")
).explode("goal.people_goals").to_dict(orient="records"))
df2.T

print(df.T.to_string())

输出

                                        0
totalCount                             95
user_id                            123455
user_email            john.smith@test.com
user_first_name                      John
user_last_name                      Smith
goal_id                            545555
goal_name                  test goal name
goal_owner                         123455
goal_narrative                           
goal_create_at                 1595874095
goal_modified_at               1595874095
goal_created_by                    123455
goal_updated_by                    123455
goal_start_date                1593561600
goal_target_date               1601424000
goal_progress                          34
goal_progress_color               #ff9933
goal_status                             1
goal_permission             internal,team
goal_category                          []
goal_owner_full_name           John Smith
goal_team_id                       766754
goal_team_name                           
goal_workstreams                       []
id                                      1
name                                 Team

pandas json 使用特定 json 属性规范化密钥错误

pandas json normalize key error with a particular json attribute

json

pandas

dataframe

typeerror

normalize