频繁更改 mongodb 中所有记录值的最有效方法是什么

Question

假设我有如下模型

import {model, Schema, Types} from 'mongoose'
import { IResource } from '../interfaces'

const ResourceSchema = new Schema<IResource>({
    user : {type : Schema.Types.ObjectId, ref : 'users'},
    type : {type: Schema.Types.ObjectId , ref : 'resource_datas'},
    building : {type : Schema.Types.ObjectId , ref : 'buildings'},
    lastUpdate : {type : Date , default : Date.now},
    value : {type : Number, default : 500},
    valuePerHours : {type : Number, default : 0}
})

const Resources = model<IResource>('resources' , ResourceSchema)

export default Resources

和示例记录：

[
   {
      user : User_ID,
      type : Type_ID,
      building : Building_ID,
      lastUpdate : Date("2022-03-21T08:32:40.866+00:00"),
      value : 500,
      valuePerHours : 120
   },
   {
      user : User_ID,
      type : Type_ID,
      building : Building_ID,
      lastUpdate : Date("2022-03-21T08:22:40.866+00:00"),
      value : 540,
      valuePerHours : 150
   },
   {
      user : User_ID,
      type : Type_ID,
      building : Building_ID,
      lastUpdate : Date("2022-03-21T08:36:40.866+00:00"),
      value : 1200,
      valuePerHours : 180
   },
]

如何尽可能频繁地更新字段“值”？

目前我比较喜欢下面

while (true) {
        const resources = await Resources.find({})
    .limit(100)
    .sort({lastUpdate : 1})
    .populate('building')

    const promiseSave = []
    for (let index = 0; index < resources.length; index++) {
        const resource = resources[index];
        const now = Date.now()
        const diffTime = (now - new Date(resource.lastUpdate).getTime()) / 1000
        const percentDiffTimePerHour = diffTime / 3600000
        const generate = resource.valuePerHours
        const valueAfterDiff = generate * percentDiffTimePerHour
        resource.value += valueAfterDiff
        resource.lastUpdate = now         
        promiseSave.push(resource.save())
    }
    await Promise.all(promiseSave)
 }

//update value follow lastUpdate and valuePerHours

每一轮查找100条记录并保存，大约需要200ms 如果我有 100 万条记录，更新所有记录需要 2000 秒。我可以每小时或每天更新一次。但更好的是它应该几乎是“实时的”。有什么更好的方法吗？感谢阅读。

Answer 1

您不需要任何循环或花哨的逻辑。您可以运行一个简单的更新聚合管道：

db.Resources.updateMany({}, [
   {
      $set: {
         lastUpdate: "$$NOW",
         value: {
            $sum: ["$value", {
               $multiply: [{
                  $divide: [
                     { $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "second" } },
                     60 * 60 * 1000
                  ]
               }, "$valuePerHours"]
            }]
         }
      }
   }
])

或更短一点：

db.Resources.updateMany({}, [
   {
      $set: {
         lastUpdate: "$$NOW",
         value: {
            $sum: ["$value", {
               $multiply: [
                  { $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "second" } },
                  "$valuePerHours",
                  1 / 60 / 60 / 1000
               ]
            }]
         }
      }
   }
])

这会更新 value 秒分辨率。如果您需要毫秒分辨率，请使用

db.Resources.updateMany({}, [
   {
      $set: {
         lastUpdate: "$$NOW",
         value: {
            $sum: ["$value", {
               $multiply: [{
                  $divide: [
                     { $dateDiff: { startDate: "$lastUpdate", endDate: "$$NOW", unit: "millisecond" } },
                     60 * 60 * 1000 * 1000
                  ]
               }, "$valuePerHours"]
            }]
         }
      }
   }
])

频繁更改 mongodb 中所有记录值的最有效方法是什么

What is most efficiently way to change value of all record in mongodb freequently

node.js

mongodb

mongoose