Mongodb 聚合 - 排序使得查询非常慢

问题描述 投票：0回答：3

Mongodb 3.2，安装在centos 6上，有足够的RAM和磁盘。我有一个包含 10K 文档的集合，其结构如下：

{
  "id":5752034,
  "score":7.6,
  "name":"ASUS X551 15.6-inch Laptop", 
  "categoryId":"803",
  "positiveAspects":[{
                       "id":30030525,
                       "name":"price",
                       "score":9.8,
                       "frequency":139,
                       "rank":100098
                     },
                     {
                       "id":30028399,
                       "name":"use",
                       "score":9.9,
                       "frequency":99,
                       "rank":100099
                     }
                     .
                     .
                ]
}

对于每个文档，嵌套数组 PositiveAspects 有几百个元素。

该藏品有以下索引：

{ "v" : 1, "key" : { "_id" : 1 }, "name" : "_id_", "ns" : "proddb.product_trees" }
{ "v" : 1, "key" : { "positiveAspects.id" : 1.0, "positiveAspects.score" : 1.0 }, "name" : "positiveAspects.id_1_positiveAspects.score_1", "ns" : "proddb.product_trees" }
{ "v" : 1, "key" : { "categoryId" : 1.0, "score" : 1.0 }, "name" : "categoryId_1_score_1", "ns" : "proddb.product_trees" }
{ "v" : 1, "key" : { "rank" : -1.0 }, "name" : "rank_-1", "ns" : "proddb.product_trees" }
{ "v" : 1, "key" : { "positiveAspects.rank" : -1.0 }, "name" : "positiveAspects.rank_-1", "ns" : "proddb.product_trees" }

我想运行以下聚合，大约需要 40 秒：

{  
  aggregate:"product_trees",
  pipeline:[  
  {  
     $match:{  
        categoryId:"803",
        score:{  
           $gte:8.0
        }
     }
  },
  {  
     $unwind:"$positiveAspects"
  },
  {  
     $match:{  
        positiveAspects.id:30030525,
        positiveAspects.score:{  
           $gte:9.0
        }
     }
  },
  {  
     $sort:{  
        positiveAspects.rank:-1
     }
  },
  {  
     $project:{  
        _id:0,
        score:1,
        id:1,
        name:1,
        positiveAspects:1
     }
  },
  {  
     $limit:10
  }
 ]
}

解释如下：

2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Beginning planning...
=============================
Options = NO_BLOCKING_SORT INDEX_INTERSECTION
Canonical query:
ns=proddb.product_treesTree: $and
    categoryId == "803"
    score $gte 8.0
Sort: {}
Proj: {}
=============================
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Index 0 is kp: { _id: 1 } unique name: '_id_' io: { v: 1, key: { _id: 1 }, name: "_id_", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Index 1 is kp: { positiveAspects.id: 1.0, positiveAspects.score: 1.0 } multikey name: 'positiveAspects.id_1_positiveAspects.score_1' io: { v: 1, key: { positiveAspects.id: 1.0, positiveAspects.score: 1.0 }, name: "positiveAspects.id_1_positiveAspects.score_1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Index 2 is kp: { categoryId: 1.0, score: 1.0 } name: 'categoryId_1_score_1' io: { v: 1, key: { categoryId: 1.0, score: 1.0 }, name: "categoryId_1_score_1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Index 3 is kp: { rank: -1.0 } name: 'rank_-1' io: { v: 1, key: { rank: -1.0 }, name: "rank_-1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Index 4 is kp: { positiveAspects.rank: -1.0 } multikey name: 'positiveAspects.rank_-1' io: { v: 1, key: { positiveAspects.rank: -1.0 }, name: "positiveAspects.rank_-1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Predicate over field 'score'
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Predicate over field 'categoryId'
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Relevant index 0 is kp: { categoryId: 1.0, score: 1.0 } name: 'categoryId_1_score_1' io: { v: 1, key: { categoryId: 1.0, score: 1.0 }, name: "categoryId_1_score_1", ns: "proddb.product_trees" }
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Rated tree:
$and
    categoryId == "803"  || First: 0 notFirst: full path: categoryId
    score $gte 8.0  || First: notFirst: 0 full path: score
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Tagging memoID 1
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Enumerator: memo just before moving:
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] About to build solntree from tagged tree:
$and
    categoryId == "803"  || Selected Index #0 pos 0
    score $gte 8.0  || Selected Index #0 pos 1
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Planner: adding solution:
FETCH
---fetched = 1
---sortedByDiskLoc = 0
---getSort = [{ categoryId: 1 }, { categoryId: 1, score: 1 }, { score: 1 }, ]
---Child:
------IXSCAN
---------keyPattern = { categoryId: 1.0, score: 1.0 }
---------direction = 1
---------bounds = field #0['categoryId']: ["803", "803"], field #1['score']: [8.0, inf.0]
---------fetched = 0
---------sortedByDiskLoc = 0
---------getSort = [{ categoryId: 1 }, { categoryId: 1, score: 1 }, { score: 1 }, ]
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Planner: outputted 1 indexed solutions.
2016-06-01T16:10:49.140-0500 D QUERY    [conn47] Only one plan is available; it will be run but will not be cached. query: { categoryId: "803", score: { $gte: 8.0 } } sort: {} projection: {}, planSummary: IXSCAN { categoryId: 1.0, score: 1.0 }
2016-06-01T16:11:27.170-0500 I COMMAND  [conn47] command proddb.product_trees command: aggregate { aggregate: "product_trees", pipeline: [ { $match: { categoryId: "803", score: { $gte: 8.0 } } }, { $unwind: "$positiveAspects" }, { $match: { positiveAspects.id: 30030525, positiveAspects.score: { $gte: 9.0 } } }, { $sort: { positiveAspects.rank: -1 } }, { $project: { _id: 0, score: 1, id: 1, name: 1, positiveAspects: 1 } }, { $limit: 10 } ], cursor: {} } keyUpdates:0 writeConflicts:0 numYields:226 reslen:7459 locks:{ Global: { acquireCount: { r: 906 } }, Database: { acquireCount: { r: 453 } }, Collection: { acquireCount: { r: 453 } } } protocol:op_query 38030ms

取出

$sort
，查询将在2秒内运行。

考虑到它可以使用索引，您能否解释一下为什么 

$sort
会导致如此大的性能下降？是否有我错过的索引 可以做什么来修复？

谢谢！

Mongodb 聚合 - 排序使得查询非常慢

mongodb aggregation-framework
3个回答
10
投票

这是因为

$sort
在聚合框架早期不使用时没有使用索引。要利用索引，必须使用 $sort 或 $match 作为第一阶段。

请参阅管道运算符和索引

4
投票

发生这种情况是因为 $unwind 阶段。您的查询性能很慢，因为查询不考虑 $unwind 阶段之后的索引。通过解释检查。您会知道。发生这种情况是因为在 $unwind 后整个文档将发生变化，并且它变得不同，存储在 RAM 中用于索引目的。

0
投票

正如 Kashyap Rathod 已经写的那样，这是由于 $unwind 操作造成的。您可以将 $unwind 操作移至聚合末尾。

{  
  aggregate:"product_trees",
  pipeline:[  
  {  
     $match:{  
        categoryId:"803",
        score:{  
           $gte:8.0
        }
     }
  },
  
  {  
     $match:{  
        positiveAspects.id:30030525,
        positiveAspects.score:{  
           $gte:9.0
        }
     }
  },
  {  
     $sort:{  
        positiveAspects.rank:-1
     }
  },
  {  
     $project:{  
        _id:0,
        score:1,
        id:1,
        name:1,
        positiveAspects:1
     }
  },
  {  
     $limit:10
  },
  {  
     $unwind:"$positiveAspects"
  },
 ]
}
最新问题
© www.soinside.com 2019 - 2024. All rights reserved.