如何减少聚合管道中的重复数据?
How can I reduce the duplicate data from my aggregation pipeline?
我有一个管道非常适合我的需要...但我认为有一些冗余数据可以从管道中删除。
预期输出
这就是我想要的输出结果
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": [
{
"_id": "5ecb263c166b8500047c1411",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1422",
"what": "Log OUT"
}
]
}
当前输出
这是当前输出的样子
{
"docs": [
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": {
"_id": "5ecb263c166b8500047c1411",
"what": "Log IN"
},
"historyIndex": 0
},
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": {
"_id": "5ecb263c166b8500047c1422",
"what": "Log OUT"
},
"historyIndex": 1
}
]
}
用户文档
在现实生活中会有比这更多的用户……当然……
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
}
历史文档
同样,为简单起见,我保持数据简短
[
{
"_id": "5ecb263c166b8500047c1411",
"userId": "5ecee2189fdd1b0004056936",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1422",
"userId": "5ecee2189fdd1b0004056999",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1433",
"userId": "5ecee2189fdd1b0004056936",
"what": "Log OUT"
},
{
"_id": "5ecb263c166b8500047c1444",
"userId": "5ecee2189fdd1b0004056999",
"what": "Log OUT"
}
]
mongoose-aggregate-paginate-v2 中间件
我也在使用 mongoose-aggregate-paginate-v2,但我 认为 不是我的问题,但返回结果时肯定会起作用。它需要将文档展平,以便对它们进行计数和分页:
"totalDocs": 941,
"limit": 500,
"page": 1,
"totalPages": 2,
"pagingCounter": 1,
"hasPrevPage": false,
"hasNextPage": true,
"prevPage": null,
"nextPage": 2
流水线
这是我的管道
var agg_match = {
$match:
{
_id: mongoose.Types.ObjectId(userId)
}
};
var agg_lookup = {
$lookup: {
from: 'it_userhistories',
localField: '_id',
foreignField: 'userId',
as: 'history'
}
}
var agg_unwind = {
$unwind: {
path: "$history",
preserveNullAndEmptyArrays: true,
includeArrayIndex: 'historyIndex',
}
}
var agg = [
agg_match,
agg_lookup,
agg_unwind,
agg_project,
];
var pageAndLimit = {
page:page,
limit:limit
}
User.aggregatePaginate(myAggregate, pageAndLimit)
您可以使用 $map
运算符来执行此操作。以下查询会有所帮助(我没有在管道中包含匹配阶段,您可以轻松地包含它):
db.user.aggregate([
{
$lookup: {
from: "history",
localField: "_id",
foreignField: "userId",
as: "history"
}
},
{
$project: {
name: 1,
history: {
$map: {
input: "$history",
as: "h",
in: {
_id: "$$h._id",
what: "$$h.what"
}
}
}
}
}
])
我有一个管道非常适合我的需要...但我认为有一些冗余数据可以从管道中删除。
预期输出
这就是我想要的输出结果
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": [
{
"_id": "5ecb263c166b8500047c1411",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1422",
"what": "Log OUT"
}
]
}
当前输出
这是当前输出的样子
{
"docs": [
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": {
"_id": "5ecb263c166b8500047c1411",
"what": "Log IN"
},
"historyIndex": 0
},
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
"history": {
"_id": "5ecb263c166b8500047c1422",
"what": "Log OUT"
},
"historyIndex": 1
}
]
}
用户文档
在现实生活中会有比这更多的用户……当然……
{
"_id": "5ecee2189fdd1b0004056936",
"name": "Mike",
}
历史文档
同样,为简单起见,我保持数据简短
[
{
"_id": "5ecb263c166b8500047c1411",
"userId": "5ecee2189fdd1b0004056936",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1422",
"userId": "5ecee2189fdd1b0004056999",
"what": "Log IN"
},
{
"_id": "5ecb263c166b8500047c1433",
"userId": "5ecee2189fdd1b0004056936",
"what": "Log OUT"
},
{
"_id": "5ecb263c166b8500047c1444",
"userId": "5ecee2189fdd1b0004056999",
"what": "Log OUT"
}
]
mongoose-aggregate-paginate-v2 中间件
我也在使用 mongoose-aggregate-paginate-v2,但我 认为 不是我的问题,但返回结果时肯定会起作用。它需要将文档展平,以便对它们进行计数和分页:
"totalDocs": 941,
"limit": 500,
"page": 1,
"totalPages": 2,
"pagingCounter": 1,
"hasPrevPage": false,
"hasNextPage": true,
"prevPage": null,
"nextPage": 2
流水线
这是我的管道
var agg_match = {
$match:
{
_id: mongoose.Types.ObjectId(userId)
}
};
var agg_lookup = {
$lookup: {
from: 'it_userhistories',
localField: '_id',
foreignField: 'userId',
as: 'history'
}
}
var agg_unwind = {
$unwind: {
path: "$history",
preserveNullAndEmptyArrays: true,
includeArrayIndex: 'historyIndex',
}
}
var agg = [
agg_match,
agg_lookup,
agg_unwind,
agg_project,
];
var pageAndLimit = {
page:page,
limit:limit
}
User.aggregatePaginate(myAggregate, pageAndLimit)
您可以使用 $map
运算符来执行此操作。以下查询会有所帮助(我没有在管道中包含匹配阶段,您可以轻松地包含它):
db.user.aggregate([
{
$lookup: {
from: "history",
localField: "_id",
foreignField: "userId",
as: "history"
}
},
{
$project: {
name: 1,
history: {
$map: {
input: "$history",
as: "h",
in: {
_id: "$$h._id",
what: "$$h.what"
}
}
}
}
}
])