频繁更改 mongodb 中所有记录值的最有效方法是什么
What is most efficiently way to change value of all record in mongodb freequently
假设我有如下模型
import {model, Schema, Types} from 'mongoose'
import { IResource } from '../interfaces'
const ResourceSchema = new Schema<IResource>({
user : {type : Schema.Types.ObjectId, ref : 'users'},
type : {type: Schema.Types.ObjectId , ref : 'resource_datas'},
building : {type : Schema.Types.ObjectId , ref : 'buildings'},
lastUpdate : {type : Date , default : Date.now},
value : {type : Number, default : 500},
valuePerHours : {type : Number, default : 0}
})
const Resources = model<IResource>('resources' , ResourceSchema)
export default Resources
和示例记录:
[
{
user : User_ID,
type : Type_ID,
building : Building_ID,
lastUpdate : Date("2022-03-21T08:32:40.866+00:00"),
value : 500,
valuePerHours : 120
},
{
user : User_ID,
type : Type_ID,
building : Building_ID,
lastUpdate : Date("2022-03-21T08:22:40.866+00:00"),
value : 540,
valuePerHours : 150
},
{
user : User_ID,
type : Type_ID,
building : Building_ID,
lastUpdate : Date("2022-03-21T08:36:40.866+00:00"),
value : 1200,
valuePerHours : 180
},
]
如何尽可能频繁地更新字段“值”?
目前我比较喜欢下面
while (true) {
const resources = await Resources.find({})
.limit(100)
.sort({lastUpdate : 1})
.populate('building')
const promiseSave = []
for (let index = 0; index < resources.length; index++) {
const resource = resources[index];
const now = Date.now()
const diffTime = (now - new Date(resource.lastUpdate).getTime()) / 1000
const percentDiffTimePerHour = diffTime / 3600000
const generate = resource.valuePerHours
const valueAfterDiff = generate * percentDiffTimePerHour
resource.value += valueAfterDiff
resource.lastUpdate = now
promiseSave.push(resource.save())
}
await Promise.all(promiseSave)
}
//update value follow lastUpdate and valuePerHours
每一轮查找100条记录并保存,大约需要200ms
如果我有 100 万条记录,更新所有记录需要 2000 秒。
我可以每小时或每天更新一次。但更好的是它应该几乎是“实时的”。
有什么更好的方法吗?感谢阅读。
您不需要任何循环或花哨的逻辑。您可以 运行 一个简单的更新聚合管道:
db.Resources.updateMany({}, [
{
$set: {
lastUpdate: "$$NOW",
value: {
$sum: ["$value", {
$multiply: [{
$divide: [
{ $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "second" } },
60 * 60 * 1000
]
}, "$valuePerHours"]
}]
}
}
}
])
或更短一点:
db.Resources.updateMany({}, [
{
$set: {
lastUpdate: "$$NOW",
value: {
$sum: ["$value", {
$multiply: [
{ $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "second" } },
"$valuePerHours",
1 / 60 / 60 / 1000
]
}]
}
}
}
])
这会更新 value
秒分辨率。如果您需要毫秒分辨率,请使用
db.Resources.updateMany({}, [
{
$set: {
lastUpdate: "$$NOW",
value: {
$sum: ["$value", {
$multiply: [{
$divide: [
{ $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "millisecond" } },
60 * 60 * 1000 * 1000
]
}, "$valuePerHours"]
}]
}
}
}
])
假设我有如下模型
import {model, Schema, Types} from 'mongoose'
import { IResource } from '../interfaces'
const ResourceSchema = new Schema<IResource>({
user : {type : Schema.Types.ObjectId, ref : 'users'},
type : {type: Schema.Types.ObjectId , ref : 'resource_datas'},
building : {type : Schema.Types.ObjectId , ref : 'buildings'},
lastUpdate : {type : Date , default : Date.now},
value : {type : Number, default : 500},
valuePerHours : {type : Number, default : 0}
})
const Resources = model<IResource>('resources' , ResourceSchema)
export default Resources
和示例记录:
[
{
user : User_ID,
type : Type_ID,
building : Building_ID,
lastUpdate : Date("2022-03-21T08:32:40.866+00:00"),
value : 500,
valuePerHours : 120
},
{
user : User_ID,
type : Type_ID,
building : Building_ID,
lastUpdate : Date("2022-03-21T08:22:40.866+00:00"),
value : 540,
valuePerHours : 150
},
{
user : User_ID,
type : Type_ID,
building : Building_ID,
lastUpdate : Date("2022-03-21T08:36:40.866+00:00"),
value : 1200,
valuePerHours : 180
},
]
如何尽可能频繁地更新字段“值”?
目前我比较喜欢下面
while (true) {
const resources = await Resources.find({})
.limit(100)
.sort({lastUpdate : 1})
.populate('building')
const promiseSave = []
for (let index = 0; index < resources.length; index++) {
const resource = resources[index];
const now = Date.now()
const diffTime = (now - new Date(resource.lastUpdate).getTime()) / 1000
const percentDiffTimePerHour = diffTime / 3600000
const generate = resource.valuePerHours
const valueAfterDiff = generate * percentDiffTimePerHour
resource.value += valueAfterDiff
resource.lastUpdate = now
promiseSave.push(resource.save())
}
await Promise.all(promiseSave)
}
//update value follow lastUpdate and valuePerHours
每一轮查找100条记录并保存,大约需要200ms 如果我有 100 万条记录,更新所有记录需要 2000 秒。 我可以每小时或每天更新一次。但更好的是它应该几乎是“实时的”。 有什么更好的方法吗?感谢阅读。
您不需要任何循环或花哨的逻辑。您可以 运行 一个简单的更新聚合管道:
db.Resources.updateMany({}, [
{
$set: {
lastUpdate: "$$NOW",
value: {
$sum: ["$value", {
$multiply: [{
$divide: [
{ $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "second" } },
60 * 60 * 1000
]
}, "$valuePerHours"]
}]
}
}
}
])
或更短一点:
db.Resources.updateMany({}, [
{
$set: {
lastUpdate: "$$NOW",
value: {
$sum: ["$value", {
$multiply: [
{ $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "second" } },
"$valuePerHours",
1 / 60 / 60 / 1000
]
}]
}
}
}
])
这会更新 value
秒分辨率。如果您需要毫秒分辨率,请使用
db.Resources.updateMany({}, [
{
$set: {
lastUpdate: "$$NOW",
value: {
$sum: ["$value", {
$multiply: [{
$divide: [
{ $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "millisecond" } },
60 * 60 * 1000 * 1000
]
}, "$valuePerHours"]
}]
}
}
}
])