NodeJS、DocumentDB(通过 Docooment)批量插入
NodeJS, DocumentDB (Via Docooment) Bulk Insert
我正在编写一个脚本来使用 NodeJS 和 Docooment 将一些批量导入(从 CSV)到 documentDB。
有没有办法进行批量插入?
我知道它基于 Mongoose,但有关 Mongoose 批量插入的示例与特定 MongoDB 功能相关。
我建议选择以下选项之一:
DocumentDB 的数据迁移工具
将数据从 CSV 批量导入 DocumentDB 的最简单方法是使用 DocumentDB 的迁移工具。您可以找到有关如何使用它的详细信息 here。
通过存储过程以编程方式(数据库端脚本)
如果您想以编程方式批量导入数据,我建议您查看 DocumentDB 的存储过程 - 它允许您在数据库服务器本身上执行批处理和排序操作(避免需要发出多个网络请求)。
这里有一个sample stored procedure用于批量导入数据:
/**
* This script called as stored procedure to import lots of documents in one batch.
* The script sets response body to the number of docs imported and is called multiple times
* by the client until total number of docs desired by the client is imported.
* @param {Object[]} docs - Array of documents to import.
*/
function bulkImport(docs) {
var collection = getContext().getCollection();
var collectionLink = collection.getSelfLink();
// The count of imported docs, also used as current doc index.
var count = 0;
// Validate input.
if (!docs) throw new Error("The array is undefined or null.");
var docsLength = docs.length;
if (docsLength == 0) {
getContext().getResponse().setBody(0);
}
// Call the CRUD API to create a document.
tryCreate(docs[count], callback);
// Note that there are 2 exit conditions:
// 1) The createDocument request was not accepted.
// In this case the callback will not be called, we just call setBody and we are done.
// 2) The callback was called docs.length times.
// In this case all documents were created and we don't need to call tryCreate anymore. Just call setBody and we are done.
function tryCreate(doc, callback) {
var isAccepted = collection.createDocument(collectionLink, doc, callback);
// If the request was accepted, callback will be called.
// Otherwise report current count back to the client,
// which will call the script again with remaining set of docs.
// This condition will happen when this stored procedure has been running too long
// and is about to get cancelled by the server. This will allow the calling client
// to resume this batch from the point we got to before isAccepted was set to false
if (!isAccepted) getContext().getResponse().setBody(count);
}
// This is called when collection.createDocument is done and the document has been persisted.
function callback(err, doc, options) {
if (err) throw err;
// One more document has been inserted, increment the count.
count++;
if (count >= docsLength) {
// If we have created all documents, we are done. Just set the response.
getContext().getResponse().setBody(count);
} else {
// Create next document.
tryCreate(docs[count], callback);
}
}
}
您可以找到有关 DocumentDB 数据库端编程(存储过程、触发器和 UDF)的参考文档here。
我正在编写一个脚本来使用 NodeJS 和 Docooment 将一些批量导入(从 CSV)到 documentDB。
有没有办法进行批量插入?
我知道它基于 Mongoose,但有关 Mongoose 批量插入的示例与特定 MongoDB 功能相关。
我建议选择以下选项之一:
DocumentDB 的数据迁移工具
将数据从 CSV 批量导入 DocumentDB 的最简单方法是使用 DocumentDB 的迁移工具。您可以找到有关如何使用它的详细信息 here。
通过存储过程以编程方式(数据库端脚本)
如果您想以编程方式批量导入数据,我建议您查看 DocumentDB 的存储过程 - 它允许您在数据库服务器本身上执行批处理和排序操作(避免需要发出多个网络请求)。
这里有一个sample stored procedure用于批量导入数据:
/**
* This script called as stored procedure to import lots of documents in one batch.
* The script sets response body to the number of docs imported and is called multiple times
* by the client until total number of docs desired by the client is imported.
* @param {Object[]} docs - Array of documents to import.
*/
function bulkImport(docs) {
var collection = getContext().getCollection();
var collectionLink = collection.getSelfLink();
// The count of imported docs, also used as current doc index.
var count = 0;
// Validate input.
if (!docs) throw new Error("The array is undefined or null.");
var docsLength = docs.length;
if (docsLength == 0) {
getContext().getResponse().setBody(0);
}
// Call the CRUD API to create a document.
tryCreate(docs[count], callback);
// Note that there are 2 exit conditions:
// 1) The createDocument request was not accepted.
// In this case the callback will not be called, we just call setBody and we are done.
// 2) The callback was called docs.length times.
// In this case all documents were created and we don't need to call tryCreate anymore. Just call setBody and we are done.
function tryCreate(doc, callback) {
var isAccepted = collection.createDocument(collectionLink, doc, callback);
// If the request was accepted, callback will be called.
// Otherwise report current count back to the client,
// which will call the script again with remaining set of docs.
// This condition will happen when this stored procedure has been running too long
// and is about to get cancelled by the server. This will allow the calling client
// to resume this batch from the point we got to before isAccepted was set to false
if (!isAccepted) getContext().getResponse().setBody(count);
}
// This is called when collection.createDocument is done and the document has been persisted.
function callback(err, doc, options) {
if (err) throw err;
// One more document has been inserted, increment the count.
count++;
if (count >= docsLength) {
// If we have created all documents, we are done. Just set the response.
getContext().getResponse().setBody(count);
} else {
// Create next document.
tryCreate(docs[count], callback);
}
}
}
您可以找到有关 DocumentDB 数据库端编程(存储过程、触发器和 UDF)的参考文档here。