我的应用程序正在运行许多WMI查询,这些查询是通过打开与root\cimv2
的连接,执行查询,然后关闭该连接来完成的。
现在看来WmiPrvSE.exe
进程正在进入内存泄漏。
heap_stat.py内存泄漏调查(heap_stat.py,如this URL中所述),显示以下对象是此泄漏的来源:
Type name Count New count
fastprox!CClassPart 2305 2409
fastprox!CInstancePart 1719 1705
cimwin32!CRefPtrLite 1131 1205
fastprox!CWbemObject 1363 1303
fastprox!CWbemInstance 1347 1443
...
provthrd!WmiTreeNode 734 882
combase!CNdrStream 166 300
在互联网上,有很多关于WMI相关内存泄漏的修补程序(example 1,example 2,...),但那些似乎只与某些情况相关。我怎么知道我的情况是否被任何一个案件和哪个案件所涵盖? (我不能告诉我的客户只是在他的系统上应用修补程序,而没有解释为什么这个修补程序可以解决他的问题)
我的Windows版本(Winver.exe结果)是:
Windows Server 2016
Microsoft Windows Server
Version 1607 (OS Build 14393.1770)
Copyright 2016 Microsoft Corporation. All rights reserved.
...
与此同时,我也了解到我们可能会使用一些工具来监控WMI状态,例如described here。我们怎样才能知道发生了什么以及如何(哪个补丁/解决方案)可以解决它?
在MagicAndre1981评论后编辑
在上面提到的SuperUser帖子中,有人提到Windbg
的!Analyze -v
命令可能会显示一些信息,因此我发布了这个命令,结果(我看了一下,但我不知道这一切意味着什么)。为了您的信息:我在两个不同的WmiPrvSE.exe
转储上启动了此命令,结果类似:
0:000> !analyze -v
*******************************************************************************
* *
* Exception Analysis *
* *
*******************************************************************************
GetUrlPageData2 (WinHttp) failed: 12030.
DUMP_CLASS: 2
DUMP_QUALIFIER: 400
FAULTING_IP:
+0
00000000`00000000 ?? ???
EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 0000000000000000
ExceptionCode: 80000003 (Break instruction exception)
ExceptionFlags: 00000000
NumberParameters: 0
FAULTING_THREAD: 00005e64
BUGCHECK_STR: BREAKPOINT
DEFAULT_BUCKET_ID: BREAKPOINT
PROCESS_NAME: WmiPrvSE.exe
ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION} Breakpoint A breakpoint has been reached.
EXCEPTION_CODE: (HRESULT) 0x80000003 (2147483651) - One or more arguments are invalid
EXCEPTION_CODE_STR: 80000003
WATSON_BKT_PROCSTAMP: 57899ab2
WATSON_BKT_PROCVER: 6.2.14393.0
PROCESS_VER_PRODUCT: Microsoft® Windows® Operating System
WATSON_BKT_MODULE: unknown
WATSON_BKT_MODVER: 0.0.0.0
WATSON_BKT_MODOFFSET: 0
WATSON_BKT_MODSTAMP: bbbbbbb4
BUILD_VERSION_STRING: 10.0.14393.1198 (rs1_release_sec.170427-1353)
MODLIST_WITH_TSCHKSUM_HASH: 89fd758871dd996e76ac11caaaa9667af30618db
MODLIST_SHA1_HASH: f52f927737ff9b80664faa9d7561eb8997ba5a98
COMMENT:
*** procdump -ma 32476
*** Manual dump
NTGLOBALFLAG: 0
PROCESS_BAM_CURRENT_THROTTLED: 0
PROCESS_BAM_PREVIOUS_THROTTLED: 0
APPLICATION_VERIFIER_FLAGS: 0
PRODUCT_TYPE: 3
SUITE_MASK: 272
DUMP_FLAGS: 8000c07
DUMP_TYPE: 3
ANALYSIS_SESSION_HOST: DOMINIQUEDS
ANALYSIS_SESSION_TIME: 03-28-2019 16:54:42.0605
ANALYSIS_VERSION: 10.0.16299.15 x86fre
THREAD_ATTRIBUTES:
OS_LOCALE: FRB
PROBLEM_CLASSES:
ID: [0n309]
Type: [@APPLICATION_FAULT_STRING]
Class: Primary
Scope: DEFAULT_BUCKET_ID (Failure Bucket ID prefix)
BUCKET_ID
Name: Omit
Data: Add
String: [BREAKPOINT]
PID: [Unspecified]
TID: [Unspecified]
Frame: [0]
PRIMARY_PROBLEM_CLASS: BREAKPOINT
LAST_CONTROL_TRANSFER: from 00007ffba40b4856 to 00007ffba2fe1164
STACK_TEXT:
00000044`9451f6f8 00007ffb`a40b4856 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : win32u!NtUserGetMessage+0x14
00000044`9451f700 00007ff6`656c5d7e : 00000044`9451f780 00000000`00000000 00000000`00000000 00000000`00000000 : user32!GetMessageW+0x26
00000044`9451f730 00007ff6`656c192b : 00000000`00000000 000001b1`ffffffff 000001b1`ae01a6c0 000001b1`adff0c40 : WmiPrvSE!Process+0x4ee
00000044`9451f860 00007ff6`656d9257 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : WmiPrvSE!WinMain+0x21b
00000044`9451f8e0 00007ffb`a3fd8364 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : WmiPrvSE!WinMainCRTStartup+0x1b7
00000044`9451f9a0 00007ffb`a66a7091 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0x14
00000044`9451f9d0 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21
STACK_COMMAND: ~0s; .ecxr ; kb
THREAD_SHA1_HASH_MOD_FUNC: 69047869935fe9e3124f9ea8ff8b6da09a09db5f
THREAD_SHA1_HASH_MOD_FUNC_OFFSET: 23d47332d72414f95439acd0d8334dbbce9ac40b
THREAD_SHA1_HASH_MOD: 07201fdab54c758a75c51b7668701b4fab031f6d
FOLLOWUP_IP:
win32u!NtUserGetMessage+14
00007ffb`a2fe1164 c3 ret
FAULT_INSTR_CODE: c32ecdc3
SYMBOL_STACK_INDEX: 0
SYMBOL_NAME: win32u!NtUserGetMessage+14
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: win32u
IMAGE_NAME: win32u.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 57a05800
BUCKET_ID: BREAKPOINT_win32u!NtUserGetMessage+14
FAILURE_EXCEPTION_CODE: 80000003
FAILURE_IMAGE_NAME: win32u.dll
BUCKET_ID_IMAGE_STR: win32u.dll
FAILURE_MODULE_NAME: win32u
BUCKET_ID_MODULE_STR: win32u
FAILURE_FUNCTION_NAME: NtUserGetMessage
BUCKET_ID_FUNCTION_STR: NtUserGetMessage
BUCKET_ID_OFFSET: 14
BUCKET_ID_MODTIMEDATESTAMP: 57a05800
BUCKET_ID_MODCHECKSUM: 22f84
BUCKET_ID_MODVER_STR: 6.2.14393.51
BUCKET_ID_PREFIX_STR: BREAKPOINT_
FAILURE_PROBLEM_CLASS: BREAKPOINT
FAILURE_SYMBOL_NAME: win32u.dll!NtUserGetMessage
FAILURE_BUCKET_ID: BREAKPOINT_80000003_win32u.dll!NtUserGetMessage
WATSON_STAGEONE_URL: http://watson.microsoft.com/StageOne/WmiPrvSE.exe/6.2.14393.0/57899ab2/unknown/0.0.0.0/bbbbbbb4/80000003/00000000.htm?Retriage=1
TARGET_TIME: 2019-03-20T13:39:18.000Z
OSBUILD: 14393
OSSERVICEPACK: 1198
SERVICEPACK_NUMBER: 0
OS_REVISION: 0
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
OSEDITION: Windows 10 Server TerminalServer SingleUserTS
USER_LCID: 0
OSBUILD_TIMESTAMP: 2017-04-28 01:48:56
BUILDDATESTAMP_STR: 170427-1353
BUILDLAB_STR: rs1_release_sec
BUILDOSVER_STR: 10.0.14393.1198
ANALYSIS_SESSION_ELAPSED_TIME: 378d
ANALYSIS_SOURCE: UM
FAILURE_ID_HASH_STRING: um:breakpoint_80000003_win32u.dll!ntusergetmessage
FAILURE_ID_HASH: {3112b5eb-303b-e877-0655-90bdfa336126}
Followup: MachineOwner
---------
在更深入地了解问题后进行编辑
在此期间,我查看了有关WMI活动的事件日志:
Application and Services Logs, Microsoft, Windows, WMI-Activity.
这包含许多错误(事件ID 5858),包含以下信息(出于安全原因,客户信息,计算机名称和GUID被混淆):
Log Name: Microsoft-Windows-WMI-Activity/Operational
Source: Microsoft-Windows-WMI-Activity
Date: 29/03/2019 11:44:54
Event ID: 5858
Task Category: None
Level: Error
Keywords:
User: SYSTEM
Computer: Computer_Name.customer_name.intra
Description:
Id = {00000000-0000-0000-0000-000000000000}; ClientMachine = Computer_Name; User = NT AUTHORITY\SYSTEM; ClientProcessId = 1220; Component = Unknown; Operation = Start IWbemServices::DeleteInstance - Root\Rsop\Computer : RSOP_ExtensionStatus.extensionGuid="{........-....-....-....-............}"; ResultCode = 0x80041002; PossibleCause = Unknown
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-WMI-Activity" Guid="{1418ef04-b0b4-4623-bf7e-d74ab47bbdaa}" />
<EventID>5858</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x4000000000000000</Keywords>
<TimeCreated SystemTime="2019-03-29T10:44:54.842915300Z" />
<EventRecordID>564437</EventRecordID>
<Correlation ActivityID="{........-....-....-....-............}" />
<Execution ProcessID="1736" ThreadID="3860" />
<Channel>Microsoft-Windows-WMI-Activity/Operational</Channel>
<Computer>Computer_Name.customer_name.intra</Computer>
<Security UserID="S-1-5-18" />
</System>
<UserData>
<Operation_ClientFailure xmlns="http://manifests.microsoft.com/win/2006/windows/WMI">
<Id>{00000000-0000-0000-0000-000000000000}</Id>
<ClientMachine>Computer_Name</ClientMachine>
<User>NT AUTHORITY\SYSTEM</User>
<ClientProcessId>1220</ClientProcessId>
<Component>Unknown</Component>
<Operation>Start IWbemServices::DeleteInstance - Root\Rsop\Computer : RSOP_ExtensionStatus.extensionGuid="{........-....-....-....-............}"</Operation>
<ResultCode>0x80041002</ResultCode>
<PossibleCause>Unknown</PossibleCause>
</Operation_ClientFailure>
</UserData>
</Event>
这些事件是否澄清了系统WMI处理可能出现的问题?
与所提到的PID(1220)相对应的服务如下(它是完整列表):
Windows Push Notifications System Service
User Profile Service
User Manager
Themes
Task Scheduler
System Event Notificiation Service
Shell hardware Detection
Remote Desktop Configuration
Network Setup Service
IP Helper
IKE and AuthIP IPsec Keying Modules
Group Policy Client
Geolocation Service
Certificate Propagation
Application Information
要跟踪WmiPrvSE.exe,您需要使用ETW或通过Eventviewer捕获Microsoft-Windows-WMI-Activity事件(单击Show Analytic and Debug Logs。在Applications and Service Logs下找到WMI的跟踪通道日志| Microsoft | Windows | WMI Activity)
我更喜欢xperf/ETW way,因为您可以将跟踪复制到不同的系统并仍然获取所有数据。
xperf -on PROC_THREAD+LOADER+PROFILE+INTERRUPT+DPC+DISPATCHER -stackwalk profile -BufferSize 1024 -MaxFile 256 -FileMode Circular -f Kernel.etl
xperf -start WMILogger -on Microsoft-Windows-WMI-Activity::0xff -BufferSize 1024 -f WMI.etl
echo Please capture about 30s of the WMI activity.
pause
xperf -stop
xperf -stop WMILogger
xperf -merge WMI.etl kernel.etl WMItracing.etl
del WMI.etl
del kernel.etl
在WPA.exe中打开生成的WMItracing.etl,并将“Generic Events”图表从左侧放到分析窗格中。
现在仅过滤到Microsoft-Windows-WMI-Activity事件,并查找WMI操作和ClientProcessId。
此客户端进程标识显示执行WMI操作的进程。
在我的示例中,此ClientProcessId属于名为Veeam ONE Monitor Server的工具。
检查系统中的WMI调用以及哪些ClientProcessId属于WMI调用。