无法通过 mgo 中的排序查询获得结果

Failing to get results with sort query in mgo

我想了解我的用例有什么问题。我正在尝试使用这样的 mgo 排序查询来查询 MongoDB:

conn := _Mongo.Connect() // custom mongo connection
defer conn.Close()
c := conn.DB(dbname).C(collname)
index := mgo.Index{
    Key:        skeys,
    Unique:     false,
    Background: true,
}
e := c.EnsureIndex(index)
if e != nil {
   panic(e)
}
err := c.Find(spec).Sort(skeys...).All(&out)

如您所见,我确实在传递的选择键 (skey) 上调用了 ensure index。但是在大型查询的运行时,我收到以下错误:

Executor error: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.

我查了一下我的collection确实有索引,但是我觉得mgo没有用

谢谢, 瓦伦丁

P.S。以下是 mongo 对相关查询的解释输出:

Explain: bson.M{"queryPlanner":bson.M{"parsedQuery":bson.M{"$and":[]interface {}{bson.M{"das.record":bson.M{"$eq":1}}, bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}}}, "winningPlan":bson.M{"stage":"SORT", "sortPattern":bson.M{"dataset.name":1}, "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "inputStage":bson.M{"stage":"FETCH", "filter":bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}, "inputStage":bson.M{"isSparse":false, "isPartial":false, "indexVersion":2, "direction":"forward", "stage":"IXSCAN", "keyPattern":bson.M{"das.record":1}, "indexName":"das.record_1", "isMultiKey":false, "isUnique":false, "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}}}}}, "rejectedPlans":[]interface {}{bson.M{"stage":"SORT", "sortPattern":bson.M{"dataset.name":1}, "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "inputStage":bson.M{"stage":"FETCH", "filter":bson.M{"das.record":bson.M{"$eq":1}}, "inputStage":bson.M{"keyPattern":bson.M{"qhash":1}, "indexName":"qhash_1", "isMultiKey":false, "isSparse":false, "stage":"IXSCAN", "isUnique":false, "isPartial":false, "indexVersion":2, "direction":"forward", "indexBounds":bson.M{"qhash":[]interface {}{"[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"}}}}}}, bson.M{"stage":"SORT", "sortPattern":bson.M{"dataset.name":1}, "inputStage":bson.M{"stage":"KEEP_MUTATIONS", "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "inputStage":bson.M{"stage":"FETCH", "inputStage":bson.M{"stage":"AND_SORTED", "inputStages":[]interface {}{bson.M{"keyPattern":bson.M{"qhash":1}, "isPartial":false, "indexVersion":2, "indexBounds":bson.M{"qhash":[]interface {}{"[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"}}, "direction":"forward", "stage":"IXSCAN", "indexName":"qhash_1", "isMultiKey":false, "isUnique":false, "isSparse":false}, bson.M{"stage":"IXSCAN", "isMultiKey":false, "isUnique":false, "isSparse":false, "isPartial":false, "direction":"forward", "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}, "keyPattern":bson.M{"das.record":1}, "indexName":"das.record_1", "indexVersion":2}}}}}}}, bson.M{"sortPattern":bson.M{"dataset.name":1}, "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "inputStage":bson.M{"stage":"FETCH", "filter":bson.M{"$and":[]interface {}{bson.M{"das.record":bson.M{"$eq":1}}, bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}}}, "inputStage":bson.M{"direction":"forward", "keyPattern":bson.M{"dataset.name":1}, "indexName":"dataset.name_1", "isMultiKey":true, "isSparse":false, "indexBounds":bson.M{"dataset.name":[]interface {}{"[MinKey, MaxKey]"}}, "stage":"IXSCAN", "isUnique":false, "isPartial":false, "indexVersion":2}}}, "stage":"SORT"}}, "plannerVersion":1, "namespace":"das.cache", "indexFilterSet":false}, "executionStats":bson.M{"executionTimeMillis":317, "totalDocsExamined":30972, "executionSuccess":false, "errorMessage":"Exec error resulting in state FAILURE :: caused by :: errmsg: \"Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.\"", "totalKeysExamined":30972, "executionStages":bson.M{"isEOF":0, "sortPattern":bson.M{"dataset.name":1}, "needTime":30973, "saveState":497, "nReturned":0, "restoreState":497, "executionTimeMillisEstimate":150, "memLimit":33554432, "advanced":0, "needYield":0, "invalidates":0, "memUsage":33554660, "inputStage":bson.M{"advanced":30972, "needTime":1, "saveState":497, "restoreState":497, "isEOF":0, "invalidates":0, "inputStage":bson.M{"stage":"FETCH", "works":30972, "restoreState":497, "invalidates":0, "alreadyHasObj":0, "inputStage":bson.M{"dupsDropped":0, "seenInvalidated":0, "stage":"IXSCAN", "needTime":0, "isMultiKey":false, "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}, "keysExamined":30972, "direction":"forward", "restoreState":497, "isEOF":0, "indexName":"das.record_1", "isSparse":false, "indexVersion":2, "seeks":1, "dupsTested":0, "executionTimeMillisEstimate":35, "advanced":30972, "needYield":0, "saveState":497, "keyPattern":bson.M{"das.record":1}, "nReturned":30972, "works":30972, "invalidates":0, "isUnique":false, "isPartial":false}, "saveState":497, "filter":bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}, "advanced":30972, "isEOF":0, "docsExamined":30972, "nReturned":30972, "executionTimeMillisEstimate":68, "needTime":0, "needYield":0}, "stage":"SORT_KEY_GENERATOR", "nReturned":30972, "executionTimeMillisEstimate":138, "works":30973, "needYield":0}, "stage":"SORT", "works":30974}, "allPlansExecution":[]interface {}{bson.M{"nReturned":0, "executionTimeMillisEstimate":69, "totalKeysExamined":10491, "totalDocsExamined":10491, "executionStages":bson.M{"stage":"SORT", "executionTimeMillisEstimate":69, "works":10492, "restoreState":335, "isEOF":0, "memUsage":11812527, "inputStage":bson.M{"advanced":10491, "needTime":1, "saveState":335, "restoreState":335, "isEOF":0, "stage":"SORT_KEY_GENERATOR", "executionTimeMillisEstimate":69, "works":10492, "invalidates":0, "inputStage":bson.M{"needTime":0, "invalidates":0, "isEOF":0, "docsExamined":10491, "alreadyHasObj":0, "stage":"FETCH", "filter":bson.M{"das.record":bson.M{"$eq":1}}, "advanced":10491, "needYield":0, "saveState":335, "inputStage":bson.M{"invalidates":0, "isMultiKey":false, "seeks":1, "dupsTested":0, "needTime":0, "isEOF":0, "works":10491, "needYield":0, "restoreState":335, "keyPattern":bson.M{"qhash":1}, "indexName":"qhash_1", "isSparse":false, "stage":"IXSCAN", "nReturned":10491, "seenInvalidated":0, "indexVersion":2, "indexBounds":bson.M{"qhash":[]interface {}{"[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"}}, "advanced":10491, "isPartial":false, "isUnique":false, "direction":"forward", "keysExamined":10491, "dupsDropped":0, "executionTimeMillisEstimate":10, "saveState":335}, "nReturned":10491, "executionTimeMillisEstimate":55, "works":10491, "restoreState":335}, "nReturned":10491, "needYield":0}, "needTime":10492, "memLimit":33554432, "nReturned":0, "advanced":0, "needYield":0, "saveState":335, "invalidates":0, "sortPattern":bson.M{"dataset.name":1}}}, bson.M{"nReturned":0, "executionTimeMillisEstimate":11, "totalKeysExamined":10491, "totalDocsExamined":5245, "executionStages":bson.M{"restoreState":335, "inputStage":bson.M{"saveState":335, "restoreState":335, "inputStage":bson.M{"nReturned":5245, "executionTimeMillisEstimate":11, "advanced":5245, "needYield":0, "restoreState":335, "invalidates":0, "inputStage":bson.M{"isEOF":0, "alreadyHasObj":0, "works":10491, "needTime":5246, "restoreState":335, "advanced":5245, "saveState":335, "nReturned":5245, "needYield":0, "invalidates":0, "inputStage":bson.M{"stage":"AND_SORTED", "executionTimeMillisEstimate":0, "works":10491, "saveState":335, "advanced":5245, "needTime":5246, "nReturned":5245, "needYield":0, "isEOF":0, "flagged":0, "restoreState":335, "invalidates":0, "failedAnd_0":0, "failedAnd_1":0, "inputStages":[]interface {}{bson.M{"stage":"IXSCAN", "nReturned":5246, "invalidates":0, "isUnique":false, "keysExamined":5246, "dupsDropped":0, "seeks":1, "dupsTested":0, "executionTimeMillisEstimate":0, "advanced":5246, "restoreState":335, "keyPattern":bson.M{"qhash":1}, "indexName":"qhash_1", "indexVersion":2, "indexBounds":bson.M{"qhash":[]interface {}{"[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"}}, "needTime":0, "needYield":0, "saveState":335, "isEOF":0, "isMultiKey":false, "direction":"forward", "works":5246, "isSparse":false, "isPartial":false, "seenInvalidated":0}, bson.M{"advanced":5245, "isEOF":0, "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}, "seeks":1, "stage":"IXSCAN", "nReturned":5245, "needTime":0, "needYield":0, "isSparse":false, "indexVersion":2, "isPartial":false, "direction":"forward", "executionTimeMillisEstimate":0, "works":5245, "saveState":335, "invalidates":0, "keyPattern":bson.M{"das.record":1}, "isMultiKey":false, "keysExamined":5245, "restoreState":335, "indexName":"das.record_1", "isUnique":false, "dupsTested":0, "dupsDropped":0, "seenInvalidated":0}}}, "stage":"FETCH", "executionTimeMillisEstimate":11, "docsExamined":5245}, "stage":"SORT_KEY_GENERATOR", "works":10492, "needTime":5247, "saveState":335, "isEOF":0}, "stage":"KEEP_MUTATIONS", "nReturned":5245, "needYield":0, "needTime":5247, "isEOF":0, "invalidates":0, "executionTimeMillisEstimate":11, "works":10492, "advanced":5245}, "executionTimeMillisEstimate":11, "works":10492, "needYield":0, "invalidates":0, "memUsage":6102195, "stage":"SORT", "nReturned":0, "advanced":0, "needTime":10492, "saveState":335, "isEOF":0, "sortPattern":bson.M{"dataset.name":1}, "memLimit":33554432}}, bson.M{"nReturned":0, "executionTimeMillisEstimate":70, "totalKeysExamined":10491, "totalDocsExamined":10491, "executionStages":bson.M{"executionTimeMillisEstimate":70, "saveState":335, "sortPattern":bson.M{"dataset.name":1}, "works":10492, "inputStage":bson.M{"advanced":10485, "needYield":0, "executionTimeMillisEstimate":58, "works":10492, "needTime":7, "saveState":335, "restoreState":335, "isEOF":0, "invalidates":0, "inputStage":bson.M{"executionTimeMillisEstimate":36, "saveState":335, "docsExamined":10491, "nReturned":10485, "restoreState":335, "invalidates":0, "needYield":0, "filter":bson.M{"$and":[]interface {}{bson.M{"das.record":bson.M{"$eq":1}}, bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}}}, "advanced":10485, "alreadyHasObj":0, "stage":"FETCH", "needTime":6, "isEOF":0, "inputStage":bson.M{"keyPattern":bson.M{"dataset.name":1}, "isPartial":false, "indexVersion":2, "nReturned":10491, "executionTimeMillisEstimate":24, "advanced":10491, "saveState":335, "invalidates":0, "indexBounds":bson.M{"dataset.name":[]interface {}{"[MinKey, MaxKey]"}}, "dupsDropped":0, "seenInvalidated":0, "dupsTested":10491, "works":10491, "needTime":0, "isMultiKey":true, "isUnique":false, "keysExamined":10491, "needYield":0, "isEOF":0, "direction":"forward", "seeks":1, "stage":"IXSCAN", "restoreState":335, "indexName":"dataset.name_1", "isSparse":false}, "works":10491}, "stage":"SORT_KEY_GENERATOR", "nReturned":10485}, "stage":"SORT", "nReturned":0, "advanced":0, "memUsage":11787104, "needTime":10492, "needYield":0, "restoreState":335, "isEOF":0, "invalidates":0, "memLimit":33554432}}, bson.M{"executionTimeMillisEstimate":59, "totalKeysExamined":10491, "totalDocsExamined":10491, "executionStages":bson.M{"works":10492, "advanced":0, "needTime":10492, "isEOF":0, "invalidates":0, "sortPattern":bson.M{"dataset.name":1}, "restoreState":335, "memUsage":11812527, "inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", "works":10492, "advanced":10491, "inputStage":bson.M{"inputStage":bson.M{"isMultiKey":false, "isSparse":false, "indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}, "seeks":1, "advanced":10491, "needTime":0, "saveState":335, "indexName":"das.record_1", "dupsTested":0, "restoreState":335, "isPartial":false, "dupsDropped":0, "isUnique":false, "indexVersion":2, "direction":"forward", "keysExamined":10491, "stage":"IXSCAN", "nReturned":10491, "needYield":0, "isEOF":0, "seenInvalidated":0, "executionTimeMillisEstimate":11, "works":10491, "invalidates":0, "keyPattern":bson.M{"das.record":1}}, "stage":"FETCH", "works":10491, "advanced":10491, "isEOF":0, "invalidates":0, "alreadyHasObj":0, "filter":bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}, "nReturned":10491, "executionTimeMillisEstimate":21, "needTime":0, "needYield":0, "saveState":335, "restoreState":335, "docsExamined":10491}, "saveState":335, "restoreState":335, "isEOF":0, "invalidates":0, "nReturned":10491, "executionTimeMillisEstimate":47, "needTime":1, "needYield":0}, "stage":"SORT", "executionTimeMillisEstimate":59, "needYield":0, "nReturned":0, "saveState":335, "memLimit":33554432}, "nReturned":0}}, "errorCode":96, "nReturned":0}, "serverInfo":bson.M{"version":"3.6.3", "gitVersion":"9586e557d54ef70f9ca4b43c26892cd55257e1a5", "host":"vkair", "port":8230}}

特别是,它说:

"inputStage":bson.M{"direction":"forward", "keyPattern":bson.M{"dataset.name":1},
"indexName":"dataset.name_1", "isMultiKey":true, "isSparse":false, 
"indexBounds":bson.M{"dataset.name":[]interface {}{"[MinKey, MaxKey]"}},     
"stage":"IXSCAN", "isUnique":false, "isPartial":false, "indexVersion":2}}}, 
"stage":"SORT"}}, 
"plannerVersion":1, "namespace":"das.cache", "indexFilterSet":false}, 
"executionStats":bson.M{"executionTimeMillis":317, "totalDocsExamined":30972,
"executionSuccess":false,
"errorMessage":"Exec error resulting in state FAILURE :: caused by :: errmsg: \"Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.

P.S.S.获胜的计划是(mgo 的输出):

"winningPlan":bson.M{"stage":"SORT", 
"sortPattern":bson.M{"dataset.name":1}, 
"inputStage":bson.M{"stage":"SORT_KEY_GENERATOR", 
"inputStage":bson.M{"stage":"FETCH", 
"filter":bson.M{"qhash":bson.M{"$eq":"7b32b92becab9b5de06fa8ac85011133"}}, 
"inputStage":bson.M{"isSparse":false, 
"isPartial":false, "indexVersion":2, 
"direction":"forward", "stage":"IXSCAN", 
"keyPattern":bson.M{"das.record":1}, 
"indexName":"das.record_1",
"isMultiKey":false, "isUnique":false, 
"indexBounds":bson.M{"das.record":[]interface {}{"[1, 1]"}}}}}}

并且,来自 mongo shell

的输出
                "winningPlan" : {
                    "stage" : "SORT",
                    "sortPattern" : {
                            "dataset.name" : 1
                    },
                    "inputStage" : {
                            "stage" : "SORT_KEY_GENERATOR",
                            "inputStage" : {
                                    "stage" : "FETCH",
                                    "filter" : {
                                            "qhash" : {
                                                    "$eq" : "7b32b92becab9b5de06fa8ac85011133"
                                            }
                                    },
                                    "inputStage" : {
                                            "stage" : "IXSCAN",
                                            "keyPattern" : {
                                                    "das.record" : 1
                                            },
                                            "indexName" : "das.record_1",
                                            "isMultiKey" : false,
                                            "isUnique" : false,
                                            "isSparse" : false,
                                            "isPartial" : false,
                                            "indexVersion" : 2,
                                            "direction" : "forward",
                                            "indexBounds" : {
                                                    "das.record" : [
                                                            "[1.0, 1.0]"
                                                    ]
                                            }
                                    }
                            }
                    }
            }

据我所知,mgo 和 mongo shell 的获胜计划是相同的。如果我在 mongo shell 中执行相同的查询,我会得到以下结果:

> db.cache.find({"qhash":"7b32b92becab9b5de06fa8ac85011133", "das.record":1}).sort({"dataset.name":1})
Error: error: {
    "ok" : 0,
    "errmsg" : "Executor error during find command: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.",
    "code" : 96,
    "codeName" : "OperationFailed"}

而如果我计算聚合,它 returns 结果很快:

db.cache.find({"qhash":"7b32b92becab9b5de06fa8ac85011133", "das.record":1}).sort({"dataset.name":1}).count()
69936

所以看起来 mongo 本身在获取结果时使用获胜计划(索引)有问题。

更新1 正如万建议的那样,我创建了新的复合索引

db.cache.createIndex({"qhash":1, "das.record":1, "dataset.name":1})

但是并没有解决问题。如果我放置查询

db.cache.find({"qhash":"7b32b92becab9b5de06fa8ac85011133", "das.record":1}).sort({"dataset.name":1}).explain()

中标方案依然使用das.record索引,见

{
    "queryPlanner" : {
            "plannerVersion" : 1,
            "namespace" : "das.cache",
            "indexFilterSet" : false,
            "parsedQuery" : {
                    "$and" : [
                            {
                                    "das.record" : {
                                            "$eq" : 1
                                    }
                            },
                            {
                                    "qhash" : {
                                            "$eq" : "7b32b92becab9b5de06fa8ac85011133"
                                    }
                            }
                    ]
            },
            "winningPlan" : {
                    "stage" : "SORT",
                    "sortPattern" : {
                            "dataset.name" : 1
                    },
                    "inputStage" : {
                            "stage" : "SORT_KEY_GENERATOR",
                            "inputStage" : {
                                    "stage" : "FETCH",
                                    "filter" : {
                                            "qhash" : {
                                                    "$eq" : "7b32b92becab9b5de06fa8ac85011133"
                                            }
                                    },
                                    "inputStage" : {
                                            "stage" : "IXSCAN",
                                            "keyPattern" : {
                                                    "das.record" : 1
                                            },
                                            "indexName" : "das.record_1",
                                            "isMultiKey" : false,
                                            "isUnique" : false,
                                            "isSparse" : false,
                                            "isPartial" : false,
                                            "indexVersion" : 2,
                                            "direction" : "forward",
                                            "indexBounds" : {
                                                    "das.record" : [
                                                            "[1.0, 1.0]"
                                                    ]
                                            }
                                    }
                            }
                    }
            },
            "rejectedPlans" : [
                    {
                            "stage" : "SORT",
                            "sortPattern" : {
                                    "dataset.name" : 1
                            },
                            "inputStage" : {
                                    "stage" : "SORT_KEY_GENERATOR",
                                    "inputStage" : {
                                            "stage" : "FETCH",
                                            "filter" : {
                                                    "das.record" : {
                                                            "$eq" : 1
                                                    }
                                            },
                                            "inputStage" : {
                                                    "stage" : "IXSCAN",
                                                    "keyPattern" : {
                                                            "qhash" : 1
                                                    },
                                                    "indexName" : "qhash_1",
                                                    "isMultiKey" : false,
                                                    "isUnique" : false,
                                                    "isSparse" : false,
                                                    "isPartial" : false,
                                                    "indexVersion" : 2,
                                                    "direction" : "forward",
                                                    "indexBounds" : {
                                                            "qhash" : [
                                                                    "[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"
                                                            ]
                                                    }
                                            }
                                    }
                            }
                    },
                    {
                            "stage" : "SORT",
                            "sortPattern" : {
                                    "dataset.name" : 1
                            },
                            "inputStage" : {
                                    "stage" : "SORT_KEY_GENERATOR",
                                    "inputStage" : {
                                            "stage" : "FETCH",
                                            "inputStage" : {
                                                    "stage" : "IXSCAN",
                                                    "keyPattern" : {
                                                            "qhash" : 1,
                                                            "das.record" : 1,
                                                            "dataset.name" : 1
                                                    },
                                                    "indexName" : "qhash_1_das.record_1_dataset.name_1",
                                                    "isMultiKey" : true,
                                                    "isUnique" : false,
                                                    "isSparse" : false,
                                                    "isPartial" : false,
                                                    "indexVersion" : 2,
                                                    "direction" : "forward",
                                                    "indexBounds" : {
                                                            "qhash" : [
                                                                    "[\"7b32b92becab9b5de06fa8ac85011133\", \"7b32b92becab9b5de06fa8ac85011133\"]"
                                                            ],
                                                            "das.record" : [
                                                                    "[1.0, 1.0]"
                                                            ],
                                                            "dataset.name" : [
                                                                    "[MinKey, MaxKey]"
                                                            ]
                                                    }
                                            }
                                    }
                            }
                    },
                    {
                            "stage" : "SORT",
                            "sortPattern" : {
                                    "dataset.name" : 1
                            },
                            "inputStage" : {
                                    "stage" : "SORT_KEY_GENERATOR",
                                    "inputStage" : {
                                            "stage" : "FETCH",
                                            "filter" : {
                                                    "$and" : [
                                                            {
                                                                    "das.record" : {
                                                                            "$eq" : 1
                                                                    }
                                                            },
                                                            {
                                                                    "qhash" : {
                                                                            "$eq" : "7b32b92becab9b5de06fa8ac85011133"
                                                                    }
                                                            }
                                                    ]
                                            },
                                            "inputStage" : {
                                                    "stage" : "IXSCAN",
                                                    "keyPattern" : {
                                                            "dataset.name" : 1
                                                    },
                                                    "indexName" : "dataset.name_1",
                                                    "isMultiKey" : true,
                                                    "isUnique" : false,
                                                    "isSparse" : false,
                                                    "isPartial" : false,
                                                    "indexVersion" : 2,
                                                    "direction" : "forward",
                                                    "indexBounds" : {
                                                            "dataset.name" : [
                                                                    "[MinKey, MaxKey]"
                                                            ]
                                                    }
                                            }
                                    }
                            }
                    }
            ]
    },
    "serverInfo" : {
            "host" : "vkair",
            "port" : 8230,
            "version" : "3.6.3",
            "gitVersion" : "9586e557d54ef70f9ca4b43c26892cd55257e1a5"
    },
    "ok" : 1
}

Executor error: OperationFailed: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.

在MongoDB中,排序操作可以根据索引中的顺序检索文档来获取排序顺序。如果查询规划器无法从索引中获取排序顺序,它将在内存中对结果进行排序。使用索引按排序顺序检索结果比检索结果并在内存中排序更有效。此外,不使用索引的排序操作在使用 32 兆字节内存时将中止。

根据您发布的 explain 的输出,这是因为您在 das.record 上有一个索引(获胜计划,用于查询),但是您是根据 dataset.name a) 丢失计划或 b) 没有这样的索引。

An index 可以支持对索引键模式的非前缀子集进行排序操作。为此,查询必须在排序键之前的所有前缀键上包含相等条件。例如,如果您的查询是:

db.cache.find({"qhash":"7b32b92becab9b5de06fa8ac85011133", "das.record":1}).sort({"dataset.name":1})

然后利用索引进行过滤和排序,你应该有一个compound index

{"qhash":1, "das.record":1, "dataset.name":1 }

查看手册的更多信息Tutorial: sort and non-prefix subset of an index

您也可以从阅读中受益:MongoDB Indexing Strategies and Blog: Optimizing MongoDB Compound Indexes

As far as I can tell the winning plans from mgo and mongo shell are the same

给定相同的查询,输出应该是相同的。我要求 mongo shell 输出的原因是删除 Go 方面,即您的代码来自调试层。

更新

I created additional index { "v" : 2, "key" : { "qhash" : 1, "das.record" : 1, "dataset.name" : 1 }, "name" : "qhash_1_das.record_1_dataset.name_1", "ns" : "das.cache" } but the MongoDB does not use it

根据您的解释输出,您可以发现新索引 qhash_1_das.record_1_dataset.name_1 列在 rejectedPlan 部分下。造成这种情况的潜在原因很少,但在这种情况下可能的原因是 dataset.name 是一个数组或具有某个数组值。由 isMultiKey:truemin, max 的索引范围表示。

dataset.name 上的 multikey index 可能导致了意外的缓慢,因此被拒绝为最快的查询计划。

如果您不打算在 dataset.name 中获取数组值,请查找有问题的文档和 alter/remove 数组值。此外,您还需要重新创建索引qhash_1_das.record_1_dataset.name_1

如果 dataset.name 实际上包含数组,则基于多键的排序性能会降低。您可能必须删除其他索引,选择另一个具有单值的字段进行排序,操作拆分 dataset.name 的值,或者重新设计您的 dataset 文档结构。