MongoDB currentOp()和Cursor.explain()对于同一查询显示不同的结果

问题描述 投票:0回答:1

我已经使用MongoDB 3.2多年了,我发现currentOp()Cursor.explain()对于同一查询显示不同的结果。

我发现几个查询正在执行很长时间(20秒以上),我认为这是不可能的,因为我进行了很多测试并为其建立了索引。据我所知,查询通常是相同的。我认为它们正在导致数据库失锁,因为当某些查询变慢时,几乎有40-50个查询卡在currentOp()中。

但是当我在shell中执行相同的读取操作时,它按照我的预期运行得非常快。我从currentOp进行了相同的查询并执行。

[当数据库锁定(我认为它已锁定)时,CPU利用率持续数小时达到100%,而我的应用程序将非常缓慢。我每1分钟监视一次currentOp,当它几秒钟没有结束时,我不得不重新启动应用程序,然后它才能正常运行。

这里是需要很长时间的查询之一。一旦发生,40-50其他但类似的查询也会卡在currentOp中。

{
    "desc"              : "conn32882",
    "threadId"          : "140677207643904",
    "connectionId"      : 32882,
    "client"            : "client",
    "active"            : true,
    "opid"              : 1374027609,
    "secs_running"      : 20,
    "microsecs_running" : NumberLong(20560351),
    "op"                : "query",
    "ns"                : "db.collection",
    "query"             : {
        "find"         : "collection",
        "filter"       : {
            "p"    : {
                "$gt" : 0
            },
            "type" : "canvas",
            "id"   : {
                "$in" : [
                    576391,
                    570391,
                    767422
                ]
            }
        },
        "sort"         : {
            "_id" : -1
        },
        "projection"   : {

        },
        "limit"        : 5000,
        "returnKey"    : false,
        "showRecordId" : false
    },
    "numYields"         : 2761,
    "locks"             : {
        "Global"     : "r",
        "Database"   : "r",
        "Collection" : "r"
    },
    "waitingForLock"    : false,
    "lockStats"         : {
        "Global"     : {
            "acquireCount"        : {
                "r" : NumberLong(5524)
            },
            "acquireWaitCount"    : {
                "r" : NumberLong(349)
            },
            "timeAcquiringMicros" : {
                "r" : NumberLong(6613952)
            }
        },
        "Database"   : {
            "acquireCount" : {
                "r" : NumberLong(2762)
            }
        },
        "Collection" : {
            "acquireCount" : {
                "r" : NumberLong(2762)
            }
        }
    }
}

这是带有executionStats选项的Shell中相同查询的输出。

命令:

db.canvasdatas.find({
        "p"   : {
            "$gt": 0
        },
        "type": "canvas",
        "id"  : {
            "$in": [
                576391,
                570391,
                767422
            ]
        }
    }, {}).sort({ _id: -1 }).limit(5000).explain('executionStats');

输出:

{
    "queryPlanner"   : {
        "plannerVersion" : 1,
        "namespace"      : "db.collection",
        "indexFilterSet" : false,
        "parsedQuery"    : {
            "$and" : [
                {
                    "type" : {
                        "$eq" : "canvas"
                    }
                },
                {
                    "p" : {
                        "$gt" : 0
                    }
                },
                {
                    "id" : {
                        "$in" : [
                            570391,
                            576391,
                            767422
                        ]
                    }
                }
            ]
        },
        "winningPlan"    : {
            "stage"       : "SORT",
            "sortPattern" : {
                "_id" : -1
            },
            "limitAmount" : 5000,
            "inputStage"  : {
                "stage"      : "SORT_KEY_GENERATOR",
                "inputStage" : {
                    "stage"      : "FETCH",
                    "filter"     : {
                        "$and" : [
                            {
                                "type" : {
                                    "$eq" : "canvas"
                                }
                            },
                            {
                                "p" : {
                                    "$gt" : 0
                                }
                            }
                        ]
                    },
                    "inputStage" : {
                        "stage"        : "IXSCAN",
                        "keyPattern"   : {
                            "id" : 1
                        },
                        "indexName"    : "id_1",
                        "isMultiKey"   : false,
                        "isUnique"     : false,
                        "isSparse"     : false,
                        "isPartial"    : false,
                        "indexVersion" : 1,
                        "direction"    : "forward",
                        "indexBounds"  : {
                            "id" : [
                                "[570391.0, 570391.0]",
                                "[576391.0, 576391.0]",
                                "[767422.0, 767422.0]"
                            ]
                        }
                    }
                }
            }
        },
        "rejectedPlans"  : [
            {
                "stage"       : "SORT",
                "sortPattern" : {
                    "_id" : -1
                },
                "limitAmount" : 5000,
                "inputStage"  : {
                    "stage"      : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage"      : "FETCH",
                        "filter"     : {
                            "id" : {
                                "$in" : [
                                    570391,
                                    576391,
                                    767422
                                ]
                            }
                        },
                        "inputStage" : {
                            "stage"        : "IXSCAN",
                            "keyPattern"   : {
                                "p"    : 1,
                                "type" : 1
                            },
                            "indexName"    : "p_1_type_1",
                            "isMultiKey"   : false,
                            "isUnique"     : false,
                            "isSparse"     : false,
                            "isPartial"    : false,
                            "indexVersion" : 1,
                            "direction"    : "forward",
                            "indexBounds"  : {
                                "p"    : [
                                    "(0.0, inf.0]"
                                ],
                                "type" : [
                                    "[\"canvas\", \"canvas\"]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage"       : "SORT",
                "sortPattern" : {
                    "_id" : -1
                },
                "limitAmount" : 5000,
                "inputStage"  : {
                    "stage"      : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage"      : "FETCH",
                        "filter"     : {
                            "$and" : [
                                {
                                    "type" : {
                                        "$eq" : "canvas"
                                    }
                                },
                                {
                                    "id" : {
                                        "$in" : [
                                            570391,
                                            576391,
                                            767422
                                        ]
                                    }
                                }
                            ]
                        },
                        "inputStage" : {
                            "stage"        : "IXSCAN",
                            "keyPattern"   : {
                                "p" : 1
                            },
                            "indexName"    : "p_1",
                            "isMultiKey"   : false,
                            "isUnique"     : false,
                            "isSparse"     : false,
                            "isPartial"    : false,
                            "indexVersion" : 1,
                            "direction"    : "forward",
                            "indexBounds"  : {
                                "p" : [
                                    "(0.0, inf.0]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage"       : "SORT",
                "sortPattern" : {
                    "_id" : -1
                },
                "limitAmount" : 5000,
                "inputStage"  : {
                    "stage"      : "SORT_KEY_GENERATOR",
                    "inputStage" : {
                        "stage"      : "FETCH",
                        "filter"     : {
                            "$and" : [
                                {
                                    "p" : {
                                        "$gt" : 0
                                    }
                                },
                                {
                                    "id" : {
                                        "$in" : [
                                            570391,
                                            576391,
                                            767422
                                        ]
                                    }
                                }
                            ]
                        },
                        "inputStage" : {
                            "stage"        : "IXSCAN",
                            "keyPattern"   : {
                                "type" : 1
                            },
                            "indexName"    : "type_1",
                            "isMultiKey"   : false,
                            "isUnique"     : false,
                            "isSparse"     : false,
                            "isPartial"    : false,
                            "indexVersion" : 1,
                            "direction"    : "forward",
                            "indexBounds"  : {
                                "type" : [
                                    "[\"canvas\", \"canvas\"]"
                                ]
                            }
                        }
                    }
                }
            },
            {
                "stage"       : "LIMIT",
                "limitAmount" : 5000,
                "inputStage"  : {
                    "stage"      : "FETCH",
                    "filter"     : {
                        "$and" : [
                            {
                                "type" : {
                                    "$eq" : "canvas"
                                }
                            },
                            {
                                "p" : {
                                    "$gt" : 0
                                }
                            },
                            {
                                "id" : {
                                    "$in" : [
                                        570391,
                                        576391,
                                        767422
                                    ]
                                }
                            }
                        ]
                    },
                    "inputStage" : {
                        "stage"        : "IXSCAN",
                        "keyPattern"   : {
                            "_id" : 1
                        },
                        "indexName"    : "_id_",
                        "isMultiKey"   : false,
                        "isUnique"     : true,
                        "isSparse"     : false,
                        "isPartial"    : false,
                        "indexVersion" : 1,
                        "direction"    : "backward",
                        "indexBounds"  : {
                            "_id" : [
                                "[MaxKey, MinKey]"
                            ]
                        }
                    }
                }
            }
        ]
    },
    "executionStats" : {
        "executionSuccess"    : true,
        "nReturned"           : 2,
        "executionTimeMillis" : 0,
        "totalKeysExamined"   : 5,
        "totalDocsExamined"   : 2,
        "executionStages"     : {
            "stage"                       : "SORT",
            "nReturned"                   : 2,
            "executionTimeMillisEstimate" : 0,
            "works"                       : 10,
            "advanced"                    : 2,
            "needTime"                    : 6,
            "needYield"                   : 0,
            "saveState"                   : 0,
            "restoreState"                : 0,
            "isEOF"                       : 1,
            "invalidates"                 : 0,
            "sortPattern"                 : {
                "_id" : -1
            },
            "memUsage"                    : 906,
            "memLimit"                    : 33554432,
            "limitAmount"                 : 5000,
            "inputStage"                  : {
                "stage"                       : "SORT_KEY_GENERATOR",
                "nReturned"                   : 0,
                "executionTimeMillisEstimate" : 0,
                "works"                       : 6,
                "advanced"                    : 0,
                "needTime"                    : 3,
                "needYield"                   : 0,
                "saveState"                   : 0,
                "restoreState"                : 0,
                "isEOF"                       : 1,
                "invalidates"                 : 0,
                "inputStage"                  : {
                    "stage"                       : "FETCH",
                    "filter"                      : {
                        "$and" : [
                            {
                                "type" : {
                                    "$eq" : "canvas"
                                }
                            },
                            {
                                "p" : {
                                    "$gt" : 0
                                }
                            }
                        ]
                    },
                    "nReturned"                   : 2,
                    "executionTimeMillisEstimate" : 0,
                    "works"                       : 5,
                    "advanced"                    : 2,
                    "needTime"                    : 2,
                    "needYield"                   : 0,
                    "saveState"                   : 0,
                    "restoreState"                : 0,
                    "isEOF"                       : 1,
                    "invalidates"                 : 0,
                    "docsExamined"                : 2,
                    "alreadyHasObj"               : 0,
                    "inputStage"                  : {
                        "stage"                       : "IXSCAN",
                        "nReturned"                   : 2,
                        "executionTimeMillisEstimate" : 0,
                        "works"                       : 5,
                        "advanced"                    : 2,
                        "needTime"                    : 2,
                        "needYield"                   : 0,
                        "saveState"                   : 0,
                        "restoreState"                : 0,
                        "isEOF"                       : 1,
                        "invalidates"                 : 0,
                        "keyPattern"                  : {
                            "id" : 1
                        },
                        "indexName"                   : "id_1",
                        "isMultiKey"                  : false,
                        "isUnique"                    : false,
                        "isSparse"                    : false,
                        "isPartial"                   : false,
                        "indexVersion"                : 1,
                        "direction"                   : "forward",
                        "indexBounds"                 : {
                            "id" : [
                                "[570391.0, 570391.0]",
                                "[576391.0, 576391.0]",
                                "[767422.0, 767422.0]"
                            ]
                        },
                        "keysExamined"                : 5,
                        "dupsTested"                  : 0,
                        "dupsDropped"                 : 0,
                        "seenInvalidated"             : 0
                    }
                }
            }
        }
    },
    "serverInfo"     : {
        "host"       : "host",
        "port"       : 27017,
        "version"    : "3.2.21",
        "gitVersion" : ""
    },
    "ok"             : 1
}

我用这种意外行为在Google上搜索,但没有找到任何解决方案。因此,当服务器挂起时,我不得不重新启动它。

为了帮助理解,这是我的情况:

  1. 我正在使用MongoDB Cloud Manager,并且数据库实例托管在AWS EC2上
  2. 我正在使用ReplicaSet,并且我的读取首选项是secondaryPreferred。因此,所有读取操作都将转到辅助节点。
  3. MongoDB版本为3.2
  4. 我为查询中使用的每个字段(每个字段)创建了索引
  5. 我在主节点和辅助节点(使用slaveOk)中都执行了相同的查询
  6. 该集合有20M个对象。
  7. 并非每次都发生相同的查询。我认为还有其他一些因素会影响性能(例如复制?)

但是我不知道如何调试这种情况。对于此问题或调试方法是否有更好的主意?

谢谢,

Edit

:我仍然不知道是什么原因,但是试图通过进行任何更改来解决它。我删除了$gt,它似乎起作用了。但是$gt在我之前的执行中没有问题,我认为这是因为目前用户很少。

Edit

:我有MongoDB云管理器,所以我可以做一些指标更改,我认为查询目标增加了,但我仍然不知道。通常,每1个文档大约100个文档,但是今天它的数量超过了2K。可能相关吗?

Query targeting for 2month

我已经使用MongoDB 3.2多年了,我发现currentOp()和Cursor.explain()对于同一查询显示不同的结果。我发现很长一段时间都在执行几个查询(20+ ...

mongodb mongodb-query mongodb-replica-set
1个回答
0
投票

我认为这可以解释您看到的内容:

© www.soinside.com 2019 - 2024. All rights reserved.