在 Azure Web App 上调试内存泄漏:内存分析失败

问题描述 投票:0回答:2

我正在尝试诊断 Azure Web 应用程序上的内存泄漏。

我使用“诊断和解决问题”>“诊断工具”>“收集内存转储”(工具参考此处)。

这会收集 dmp 文件并生成分析报告。我可以在崩溃挂起分析中看到线程和其他信息,但 DotNetMemoryAnaysis 总是失败并出现错误

Type:  System.OutOfMemoryException

Message:  Exception of type 'System.OutOfMemoryException' was thrown.

Stack Trace:
DebugDiag.DotNet.NetDbgObj.d__73.MoveNext()
System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
System.Linq.Lookup`2.Create[TSource](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer)
System.Linq.GroupedEnumerable`3.GetEnumerator()
System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
System.Linq.Buffer`1..ctor(IEnumerable`1 source)
System.Linq.OrderedEnumerable`1.d__1.MoveNext()
System.Linq.Enumerable.d__25`1.MoveNext()
System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source, Func`2 predicate)
DebugDiag.AnalysisRules.DotNetMemoryAnalysis.GCRootWalker.ShowRoots(NetScriptManager manager, NetDbgObj debugger, NetProgress progress, IEnumerable`1 top40Query) in C:\src\DebugDiag\Development\src\DebugDiag.AnalysisRules\DotNetMemoryAnalysis.cs:line 1875
DebugDiag.AnalysisRules.DotNetMemoryAnalysis.DoDotNetMemoryAnalysis() in C:\src\DebugDiag\Development\src\DebugDiag.AnalysisRules\DotNetMemoryAnalysis.cs:line 222
DebugDiag.AnalysisRules.DotNetMemoryAnalysis.RunAnalysisRule(NetScriptManager manager, NetProgress progress) in C:\src\DebugDiag\Development\src\DebugDiag.AnalysisRules\DotNetMemoryAnalysis.cs:line 182
DebugDiag.DotNet.NetAnalyzer.RunAnalysisRulesInternal(DumpFileType bitness, NetProgress progress, String symbolPath, String imagePath, String reportFileFullPath, Boolean twoTabs, AnalysisModes analysisMode)

我尝试使用

dotnet-dump
cli 工具分析文件,但任何分析操作都会出错
SOS does not support the current target architecture 0x0000014c
。 在 Visual Studio 中打开 dmp 似乎也没有提供任何分析选项,只是调试。

有没有办法可以从另一台机器上运行 dmp 分析?我应该以其他方式收集垃圾吗?

更新

windows上的azure web apps使用的分析工具可以在https://www.microsoft.com/en-us/download/confirmation.aspx?id=58210

下载

但这并没有解决我的问题。我仍然遇到内存不足的异常。

我监控了系统内存使用情况,但它从未接近顶峰。

增加了

Program Files\DebugDiag\AnalysisRules\DebugDiag.AnalysisRules.dll.config
中的 GCRootTimeout。

我还在我能找到的每个配置文件中设置了 gcAllowVeryLargeObjects

.net memory-leaks azure-web-app-service .net-4.8
2个回答
2
投票

感谢您分享您的转储文件@farlee2121。我使用 WinDbg 打开了您的转储文件并开始了

!analyze -v
。这会将可用符号拉到您的本地缓存中。

SYMSRV:  BYINDEX: 0x1D
         https://msdl.microsoft.com/download/symbols
         SOS_x86_x86_4.8.4180.00.dll
         5E7D1ED77b0000
SYMSRV:  PATH: C:\debug\sym\SOS_x86_x86_4.8.4180.00.dll\5E7D1ED77b0000\SOS_x86_x86_4.8.4180.00.dll
SYMSRV:  RESULT: 0x00000000
DBGHELP: C:\debug\sym\SOS_x86_x86_4.8.4180.00.dll\5E7D1ED77b0000\SOS_x86_x86_4.8.4180.00.dll - OK

当尝试查看堆时,

!dumpheap -stat
,导致...

Object <exec cmd="!ListNearObj /d 5cdcf038">5cdcf038</exec> has an invalid method table.
0:000> !ListNearObj /d 5cdcf038
Before:  5cdcf014           36 (0x24)   System.Collections.Hashtable+HashtableEnumerator
After:  couldn't find any object between 0x5cdcf038 and 0x5cdd00cc
Heap local consistency not confirmed.

...这可能表明 GC 在收集转储时可能正在运行。此时我们可以做两种选择。一种是使用 mex 扩展并运行

!mex.dumpheap2
或使用 PerfView 来分析堆。

Mex 显示了大量的 Automapper 对象

1,014,591 20,291,820 System.Linq.Expressions.FullConditionalExpression
2,873     24,391,820 System.Char[]
1,541,328 24,661,248 System.Linq.Expressions.AssignBinaryExpression
1,680,035 26,880,560 AutoMapper.Mappers.ConvertMapper+<>c__DisplayClass1_1
1,571,447 31,428,940 System.Linq.Expressions.LogicalBinaryExpression
1,680,036 33,600,720 System.Lazy<System.Linq.Expressions.LambdaExpression>
1,194,017 33,941,972 System.Linq.Expressions.Expression[]
3,161,253 37,935,036 System.Linq.Expressions.ConstantExpression
9,941     39,286,832 System.Collections.Generic.Dictionary+Entry<AutoMapper.TypePair,System.Lazy<System.Linq.Expressions.LambdaExpression>>[]
844,692   39,384,092 System.Reflection.MemberInfo[]
665,549   47,919,528 AutoMapper.PropertyMap
1,680,036 53,761,152 System.Func<System.Linq.Expressions.LambdaExpression>
253,050   58,765,632 System.Int32[]
339,941   61,674,100 System.String
25,206    70,703,012 System.Byte[]
Total 37,374,335 Object(s), Total Size: 1.03 GB, Free Objects 812(352.21 KB)

如果你使用 Perfview 打开转储和转储 GC 堆,我们可以得到更好的图片

Name                                                                                                                                        Inc %              Inc
 LIB <<System.Core!Linq.Expressions.Expression>>>                                                                                            21.8      214,783,296
+ AutoMapper!AutoMapper.TypeMap                                                                                                              21.8      214,783,296
 + LIB <<mscorlib!Dictionary>>                                                                                                               21.8      214,783,296
 |+ AutoMapper!AutoMapper.MapperConfiguration                                                                                                21.8      214,783,296
 ||+ AutoMapper!AutoMapper.Mapper                                                                                                            21.8      214,783,296
 |||+ Fourstarzz.Accessors!Fourstarzz.Accessors.EntityFramework.DtoMapper                                                                    21.8      214,783,296
 ||||+ Fourstarzz.Accessors!Fourstarzz.Accessors.IntegrationAccessInfoAccessor                                                               21.8      214,783,296
 |||||+ Fourstarzz.Managers!Fourstarzz.Managers.Identity.AccountConnectionManager                                                            10.9      108,058,320
 ||||||+ Fourstarzz.Managers.Adapters!Fourstarzz.Managers.Adapters.Identity.OkanjoRegistrationHandler                                        10.9      108,058,320
 |||||| + Fourstarzz.Managers.Adapters!Fourstarzz.Managers.Adapters.Identity.CompositeIdentityEventHandler                                   10.9      108,058,320
 ||||||  + Fourstarzz.Managers!Fourstarzz.Managers.Identity.UserIdentityManager                                                              10.9      108,058,320
 ||||||   + Fourstarzz.Clients.Website!Fourstarzz.Clients.Website.IdentityWrapper                                                            10.9      108,058,320
 ||||||   |+ LIB <<System!Stack<Object>>>                                                                                                    10.9      108,058,320
 ||||||   | + Autofac!Autofac.Core.Disposer                                                                                                  10.9      108,058,320
 ||||||   |  + Autofac!Autofac.Core.Lifetime.LifetimeScope                                                                                   10.9      108,058,320
 ||||||   |  |+ LIB <<mscorlib!Func>>                                                                                                        10.9      108,058,320
 ||||||   |  | + Microsoft.AspNet.Identity.Owin!Owin.AppBuilderExtensions+<>c__DisplayClass1                                                 10.9      108,058,320
 ||||||   |  | |+ LIB <<mscorlib!Func,Microsoft.Owin.IOwinContext,Fourstarzz.Shared.FourstarzzIdentity.FourstarzzUserManager>>>              10.9      108,058,320
 ||||||   |  | ||+ Microsoft.AspNet.Identity.Owin!Microsoft.AspNet.Identity.Owin.IdentityFactoryProvider                                     10.9      108,058,320
 ||||||   |  | || + Microsoft.AspNet.Identity.Owin!Microsoft.AspNet.Identity.Owin.IdentityFactoryOptions                                     10.9      108,058,320
 ||||||   |  | ||  + Microsoft.AspNet.Identity.Owin!Microsoft.AspNet.Identity.Owin.IdentityFactoryMiddleware>                                10.9      108,058,320
 ||||||   |  | ||   + Microsoft.AspNet.Identity.Owin!Microsoft.AspNet.Identity.Owin.IdentityFactoryMiddleware>                               10.9      108,058,320
 ||||||   |  | ||    + Microsoft.Owin!Microsoft.Owin.Infrastructure.OwinMiddlewareTransition                                                 10.9      108,058,320
 ||||||   |  | ||     + LIB <<Microsoft.Owin.Host.SystemWeb!Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineBlueprint>>   10.9      108,058,320
 ||||||   |  | ||      + [static var Microsoft.Owin.Host.SystemWeb.OwinHttpModule._blueprint]                                                10.9      108,058,320
 ||||||   |  | ||       + [static vars]                                                                                                      10.9      108,058,320

这里有一些需要进一步进行的事情。首先,在 Application Settings 边栏选项卡上将您的 Web 应用程序从 x86 更改为 x64,这至少会给您更多的喘息空间。重新启动应用程序后,从“诊断和解决问题”刀片收集内存转储以获得基线。然后配置 AutoHeal 以在内存达到上限时收集转储文件,以解决您遇到的 OOM。此外,Perfview 允许您与转储堆进行比较,以便您可以看到哪些对象在分配中增长;查看 Perfview 中的“启动分析”帮助以获取更多信息。 根据我个人的经验,如果配置不当,AutoMapper 可能会导致性能问题。我曾经在每次处理数据时在不需要的时候创建一个映射器。它还看起来像您将映射器添加到 AutoFac IoC 容器中,并且该映射器引用了 EventHandler。事件处理程序可以将对象固定到堆,防止 GC 收集它。


0
投票

Type: System.OutOfMemoryException Message: Array dimensions exceeded supported range. Stack Trace: System.Collections.Generic.Dictionary`2.Resize(Int32 newSize, Boolean forceNewHashCodes) System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add) DebugDiag.AnalysisRules.DotNetMemoryAnalysis.GCRootWalker.<>c__DisplayClass5_0.b__0(UInt64 child, Int32 offset) Microsoft.Diagnostics.Runtime.GCDesc.WalkObject(UInt64 addr, UInt64 size, MemoryReader cache, Action`2 refCallback) Microsoft.Diagnostics.Runtime.Desktop.DesktopHeapType.EnumerateRefsOfObject(UInt64 objRef, Action`2 action) DebugDiag.AnalysisRules.DotNetMemoryAnalysis.GCRootWalker.FindRootChains(ClrHeap heap, HashSet`1 typesOfInterest, ClrRootChain rootChain, HashSet`1 allObjects, Action`2 onRootVisit) DebugDiag.AnalysisRules.DotNetMemoryAnalysis.GCRootWalker.FindRootChains(ClrHeap heap, HashSet`1 typesOfInterest, Action`1 preEnumerate, Action`2 onRootVisit) DebugDiag.AnalysisRules.DotNetMemoryAnalysis.GCRootWalker.ShowRoots(NetScriptManager manager, NetDbgObj debugger, NetProgress progress, IEnumerable`1 top40Query) DebugDiag.AnalysisRules.DotNetMemoryAnalysis.DoDotNetMemoryAnalysis() DebugDiag.AnalysisRules.DotNetMemoryAnalysis.RunAnalysisRule(NetScriptManager manager, NetProgress progress) DebugDiag.DotNet.NetAnalyzer.RunAnalysisRulesInternal(DumpFileType bitness, NetProgress progress, String symbolPath, String imagePath, String reportFileFullPath, Boolean twoTabs, AnalysisModes analysisMode)

在这种情况下,将 
gcAllowVeryLargeObjects

添加到这两个配置文件中确实有效:

C:\Program Files\DebugDiag\DebugDiag.Analysis.exe.config
  • C:\ Program Files \ DebugDiag \ AnalysisRules \ DebugDiag.AnalysisRules.dll.config
© www.soinside.com 2019 - 2024. All rights reserved.