如何使用休息调用将一个数据框的架构应用于另一个空数据框
how to apply schema of one dataframe to another empty dataframe using rest call
我在铸造厂有两个数据集:
df1 & df2,
df1 有带架构的数据。
df2 是没有应用架构的空数据框。
使用数据代理我能够从 df1 中提取模式
{
"foundrySchema": {
"fieldSchemaList": [
{...
}
],
"primaryKey": null,
"dataFrameReaderClass": "n/a",
"customMetadata": {}
},
"rows": []
}
如何通过 rest 调用将此架构应用于空数据帧 df2?
下面的 foundry 示例展示了如何提交空事务,
此示例未显示如何应用架构
curl -X POST \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d '{}' \
"${CATALOG_URL}/api/catalog/datasets/${DATASET_RID}/transactions/${TRANSACTION_RID}/commit"
这是一个 Python 函数,用于为具有已提交事务的数据集上传架构:
from urllib.parse import quote_plus
import requests
def upload_dataset_schema(dataset_rid: str,
transaction_rid: str, schema: dict, token: str, branch='master'):
"""
Uploads the foundry dataset schema for a dataset, transaction, branch combination
Args:
dataset_rid: The rid of the dataset
transaction_rid: The rid of the transaction
schema: The foundry schema
branch: The branch
Returns: None
"""
base_url = "https://foundry-instance/foundry-metadata/api"
response = requests.post(f"{base_url}/schemas/datasets/"
f"{dataset_rid}/branches/{quote_plus(branch)}",
params={'endTransactionRid': transaction_rid},
json=schema,
headers={
'content-type': "application/json",
'authorization': f"Bearer {token}",
}
)
response.raise_for_status()
我在铸造厂有两个数据集: df1 & df2, df1 有带架构的数据。
df2 是没有应用架构的空数据框。
使用数据代理我能够从 df1 中提取模式
{
"foundrySchema": {
"fieldSchemaList": [
{...
}
],
"primaryKey": null,
"dataFrameReaderClass": "n/a",
"customMetadata": {}
},
"rows": []
}
如何通过 rest 调用将此架构应用于空数据帧 df2?
下面的 foundry 示例展示了如何提交空事务, 此示例未显示如何应用架构
curl -X POST \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: application/json" \
-d '{}' \
"${CATALOG_URL}/api/catalog/datasets/${DATASET_RID}/transactions/${TRANSACTION_RID}/commit"
这是一个 Python 函数,用于为具有已提交事务的数据集上传架构:
from urllib.parse import quote_plus
import requests
def upload_dataset_schema(dataset_rid: str,
transaction_rid: str, schema: dict, token: str, branch='master'):
"""
Uploads the foundry dataset schema for a dataset, transaction, branch combination
Args:
dataset_rid: The rid of the dataset
transaction_rid: The rid of the transaction
schema: The foundry schema
branch: The branch
Returns: None
"""
base_url = "https://foundry-instance/foundry-metadata/api"
response = requests.post(f"{base_url}/schemas/datasets/"
f"{dataset_rid}/branches/{quote_plus(branch)}",
params={'endTransactionRid': transaction_rid},
json=schema,
headers={
'content-type': "application/json",
'authorization': f"Bearer {token}",
}
)
response.raise_for_status()