In IIS 6 we used to do Url Rewriting with two third party components: Helicon (for handling extensionless url's) and UrlRewriting.net. Some time ago we migrated to IIS 7 - Classic Mode, still using these two components.
在IIS 6中，我们曾经使用两个第三方组件进行Url Rewriting：Helicon（用于处理无扩展网址）和UrlRewriting.net。前段时间我们迁移到IIS 7 - 经典模式，仍然使用这两个组件。
Now we are trying to switch to Integrated Mode without the third party components, by using native .Net Routing. The routing is working, but our web application is behaving entirely different. Our webservers used to stay wel below 10% CPU usage, but now are easily using 50% and beyond.
现在我们尝试使用本机.Net路由切换到没有第三方组件的集成模式。路由工作正常，但我们的Web应用程序表现完全不同。我们的网络服务器曾经保持低于10％的CPU使用率，但现在很容易使用50％甚至更高。
We've started analysing a memory dump, but do not seem to get to the root of the problems. It seems like the .Net caching mechanism is blocking the garbage collector? What has this got to do with using "Integrated Mode"?
我们已经开始分析内存转储，但似乎没有找到问题的根源。似乎.Net缓存机制正在阻止垃圾收集器？这与使用“集成模式”有什么关系？
Below you'll find an excerpt of our analysis. Any suggestions for where to look next would be greatly appreciated.
您可以在下面找到我们分析的摘录。任何关于下一步的建议都将不胜感激。
> !analyze -v -hang
*****************************
*
* Exception Analysis
*
*****************************
GetPageUrlData failed, server returned HTTP status 404
URL requested: http://watson.microsoft.com/00000000.htm?Retriage=1
FAULTING_IP:
+aceb6a0
00000000`00000000 ?? ???
EXCEPTION_RECORD: ffffffffffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 0000000000000000
ExceptionCode: 80000003 (Break instruction exception)
ExceptionFlags: 00000000
NumberParameters: 0
FAULTING_THREAD: 0000000000000022
BUGCHECK_STR: HANG
PROCESS_NAME: w3wp.exe
ERROR_CODE: (NTSTATUS) 0xcfffffff - <Unable to get error code text>
EXCEPTION_CODE: (NTSTATUS) 0xcfffffff - <Unable to get error code text>
MOD_LIST: <ANALYSIS/>
NTGLOBALFLAG: 0
APPLICATION_VERIFIER_FLAGS: 0
MANAGED_STACK: !dumpstack -EE
OS Thread Id: 0x42ec (221)
Current frame:
Child-SP RetAddr Caller, Callee
0000000016aee3f0 000007feecbae407 (MethodDesc 000007feec936b88 +0x77 DomainNeutralILStubClass.IL_STUB_PInvoke(IntPtr, Int32, Boolean, IntPtr ByRef))
0000000016aee4f0 000007fef8408d0a (MethodDesc 000007fef81650a8 +0x4a System.Globalization.TextInfo.GetHashCodeOrdinalIgnoreCase(System.String))
0000000016aee510 000007feecbaea82 (MethodDesc 000007feec936d80 +0x42 DomainNeutralILStubClass.IL_STUB_PInvoke(IntPtr))
0000000016aee9e8 000007fef49b6f57 (MethodDesc 000007fef4993988 +0x47 System.Configuration.Internal.InternalConfigRoot.AcquireHierarchyLockForWrite())
0000000016aeeab0 000007fef49b6f57 (MethodDesc 000007fef4993988 +0x47 System.Configuration.Internal.InternalConfigRoot.AcquireHierarchyLockForWrite())
0000000016aeeae0 000007fef49c601e (MethodDesc 000007fef49870e0 +0x5e System.Configuration.Internal.InternalConfigRoot.RemoveConfigImpl(System.String, System.Configuration.BaseConfigurationRecord))
0000000016aeeaf0 000007feecb18d36 (MethodDesc 000007feec99bc38 +0x86 System.Web.ApplicationImpersonationContext..ctor())
0000000016aeeb50 000007feecbab930 (MethodDesc 000007feec93acc8 +0x160 System.Web.Caching.CacheEntry.CallCacheItemRemovedCallback(System.Web.Caching.CacheItemRemovedCallback, System.Web.Caching.CacheItemRemovedReason))
0000000016aeeb70 000007feecbaae55 (MethodDesc 000007feec99e5a8 +0x75 System.Web.Caching.CacheDependency.DisposeInternal())
0000000016aeeba0 000007feecbac11d (MethodDesc 000007feec93cb88 +0x9d System.Web.Caching.CacheDependency.NotifyDependencyChanged(System.Object, System.EventArgs))
0000000016aeebf0 000007feecbaad3b (MethodDesc 000007feec93ace8 +0x16b System.Web.Caching.CacheEntry.Close(System.Web.Caching.CacheItemRemovedReason))
0000000016aeec70 000007feecb1538e (MethodDesc 000007feec99a9b0 +0x6fe System.Web.Caching.CacheSingle.UpdateCache(System.Web.Caching.CacheKey, System.Web.Caching.CacheEntry, Boolean, System.Web.Caching.CacheItemRemovedReason, System.Object ByRef))
0000000016aeed40 000007feecb14bdd (MethodDesc 000007feec99a248 +0x2d System.Web.Caching.CacheInternal.DoRemove(System.Web.Caching.CacheKey, System.Web.Caching.CacheItemRemovedReason))
0000000016aeed90 000007feecb4361d (MethodDesc 000007feec99ac80 +0x42d System.Web.Caching.ExpiresBucket.FlushExpiredItems(System.DateTime, Boolean))
0000000016aeee90 000007feecb43196 (MethodDesc 000007feec99aa68 +0x146 System.Web.Caching.CacheExpires.FlushExpiredItems(Boolean, Boolean))
0000000016aeeef0 000007fef8371839 (MethodDesc 000007fef807cd28 +0x9 System.Threading.Thread.get_CurrentThread())
0000000016aeef20 000007fef83717ec (MethodDesc 000007fef8090a70 +0xdc System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean))
0000000016aeef40 000007fef83ec8ec (MethodDesc 000007fef8090ac8 +0x7c System.Threading.ExecutionContext.CreateCopy())
0000000016aeef80 000007fef83ecfa7 (MethodDesc 000007fef8098b30 +0x97 System.Threading._TimerCallback.PerformTimerCallback(System.Object))
0000000016aef230 000007feed2ab8f4 (MethodDesc 000007feec9ec4c0 +0x6f4 System.Web.Hosting.PipelineRuntime.ProcessRequestNotificationHelper(IntPtr, IntPtr, IntPtr, Int32))
0000000016aef4a0 000007feed2a9ef1 (MethodDesc 000007feec9d2dc8 +0x51 DomainNeutralILStubClass.IL_STUB_ReversePInvoke(Int64, Int64, Int64, Int32))
DERIVED_WAIT_CHAIN:
Dl Eid Cid WaitType
-- --- ------- --------------------------
0 1414.168c Speculated (Triage) -->
34 1414.1ca4 Unknown
WAIT_CHAIN_COMMAND: ~0s;k;;~34s;k;;
BLOCKING_THREAD: 0000000000001ca4
DEFAULT_BUCKET_ID: APPLICATION_HANG_BusyHang
PRIMARY_PROBLEM_CLASS: APPLICATION_HANG_BusyHang
LAST_CONTROL_TRANSFER: from 000007fef951579e to 000007fef95155e1
STACK_TEXT:
00000000`0850f4c0 000007fe`f951579e : 00000000`03735c20 00000002`6f7f8e08 00000001`7f3ce748 00000000`107bdf08 : clr!SVR::gc_heap::mark_object_simple1+0x458
00000000`0850f560 000007fe`f9513745 : 00000000`03735c20 000007fe`ecc2d808 00000000`00000002 000007fe`ecc2d808 : clr!SVR::gc_heap::mark_object_simple+0x4d7
00000000`0850f5f0 000007fe`f9418987 : 00000001`7f3ce748 00000000`03735c20 00000000`201617f8 0000e5b3`06a4c486 : clr!SVR::GCHeap::Promote+0x161
00000000`0850f670 000007fe`f9418c77 : 00000000`201617f8 000007fe`f9418940 ffffffff`fffffe00 00000000`20161800 : clr!CalculateSizedRefSize+0x47
00000000`0850f6a0 000007fe`f9418be1 : 00000000`000000c0 00000000`00000001 00000000`00000003 00000000`0850f728 : clr!ScanConsecutiveHandlesWithUserData+0x67
00000000`0850f6e0 000007fe`f950dcb2 : 00000000`20160000 000007fe`f9418b88 00000000`00000003 00000000`0370d0b0 : clr!BlockScanBlocksWithUserData+0x59
00000000`0850f720 000007fe`f950d275 : 00000000`0850f8a0 00000000`0850f910 000007fe`f9418b88 00000000`0850f910 : clr!TableScanHandles+0x219
00000000`0850f7e0 000007fe`f9418ac8 : 00000000`00000002 00000000`00000008 00000000`00000008 00000007`ff316000 : clr!HndScanHandlesForGC+0x1ad
00000000`0850f890 000007fe`f95934c6 : 00000000`03735c20 000007fe`f95134b0 00000000`00000002 00000000`00000008 : clr!ScanSizedRefByAD+0xf8
00000000`0850f930 000007fe`f9511a43 : 00000000`03735c20 00000000`00000002 00000000`03735c20 000007fe`00000001 : clr!SVR::gc_heap::mark_phase+0x19c
00000000`0850f9c0 000007fe`f9512632 : 00000005`24fe3bc1 00000000`00000000 00000000`037363f8 00000000`03735c20 : clr!SVR::gc_heap::gc1+0x54
00000000`0850fa30 000007fe`f9511758 : 00000000`00000000 00000000`01120f30 00000000`01680000 00000000`00000000 : clr!SVR::gc_heap::garbage_collect+0x372
00000000`0850fac0 000007fe`f958a0cb : 00000000`0000406b 00000000`03735c20 00000000`0850fd40 00000000`00000000 : clr!SVR::gc_heap::gc_thread_function+0x78
00000000`0850fb10 00000000`7728652d : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : clr!SVR::gc_heap::gc_thread_stub+0x82
00000000`0850fd60 00000000`7771c521 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : kernel32!BaseThreadInitThunk+0xd
00000000`0850fd90 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x1d
FOLLOWUP_IP:
clr!SVR::gc_heap::gc_thread_stub+82
000007fe`f958a0cb cc int 3
SYMBOL_STACK_INDEX: d
SYMBOL_NAME: clr!SVR::gc_heap::gc_thread_stub+82
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: clr
IMAGE_NAME: clr.dll
DEBUG_FLR_IMAGE_TIMESTAMP: 4e1822f4
STACK_COMMAND: ~34s ; kb
BUCKET_ID: X64_HANG_clr!SVR::gc_heap::gc_thread_stub+82
FAILURE_BUCKET_ID: APPLICATION_HANG_BusyHang_cfffffff_clr.dll!SVR::gc_heap::gc_thread_stub
WATSON_STAGEONE_URL: http://watson.microsoft.com/00000000.htm?Retriage=1
Followup: MachineOwner
---------
> !clrstack
OS Thread Id: 0x42ec (221)
Child SP IP Call Site
0000000016aee998 00000000777418ca [HelperMethodFrame_1OBJ: 0000000016aee998] System.Threading.ReaderWriterLock.AcquireWriterLockInternal(Int32)
0000000016aeeac0 000007fef49b6f57 System.Configuration.Internal.InternalConfigRoot.AcquireHierarchyLockForWrite()
0000000016aeeaf0 000007fef49c601e System.Configuration.Internal.InternalConfigRoot.RemoveConfigImpl(System.String, System.Configuration.BaseConfigurationRecord)
0000000016aeeb60 000007feecbab930 System.Web.Caching.CacheEntry.CallCacheItemRemovedCallback(System.Web.Caching.CacheItemRemovedCallback, System.Web.Caching.CacheItemRemovedReason)
0000000016aeec00 000007feecbaad3b System.Web.Caching.CacheEntry.Close(System.Web.Caching.CacheItemRemovedReason)
0000000016aeec80 000007feecb1538e System.Web.Caching.CacheSingle.UpdateCache(System.Web.Caching.CacheKey, System.Web.Caching.CacheEntry, Boolean, System.Web.Caching.CacheItemRemovedReason, System.Object ByRef)
0000000016aeed50 000007feecb14bdd System.Web.Caching.CacheInternal.DoRemove(System.Web.Caching.CacheKey, System.Web.Caching.CacheItemRemovedReason)
0000000016aeeda0 000007feecb4361d System.Web.Caching.ExpiresBucket.FlushExpiredItems(System.DateTime, Boolean)
0000000016aeeea0 000007feecb43196 System.Web.Caching.CacheExpires.FlushExpiredItems(Boolean, Boolean)
0000000016aeef30 000007fef83717ec System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
0000000016aeef90 000007fef83ecfa7 System.Threading._TimerCallback.PerformTimerCallback(System.Object)
0000000016aef208 000007fef9399714 [GCFrame: 0000000016aef208]
0000000016aef3e0 000007fef9399714 [DebuggerU2MCatchHandlerFrame: 0000000016aef3e0]
0000000016aef5b8 000007fef9399714 [ContextTransitionFrame: 0000000016aef5b8]
0000000016aef7a0 000007fef9399714 [DebuggerU2MCatchHandlerFrame: 0000000016aef7a0]
> !syncblk
Index SyncBlock MonitorHeld Recursion Owning Thread Info SyncBlock Owner
7906 0000000015d167c8 1 1 000000000fdbee90 405c 270 000000010f37d138 System.Dynamic.Utils.CacheDict`2[[System.Type, mscorlib],[System.Reflection.MethodInfo, mscorlib]]
8144 00000000153c3038 3 1 0000000015e0f1f0 4338 228 000000021f79e8b0 System.Web.CachedPathData
26403 00000000154b5e58 5 1 0000000010bdb640 42b8 119 00000001cf83a158 System.Web.CachedPathData
26600 000000000fd0e0d8 3 1 000000001d006e60 403c 311 000000025f712b40 System.Web.CachedPathData
26677 000000001519c558 3 1 000000001eff0540 36e8 327 000000020f664fb8 System.Web.CachedPathData
27056 000000001519f428 5 1 000000001c862340 34e0 294 000000018f5fd5b8 System.Object
27174 000000000ce4b628 1 1 000000000fe5c830 464c 212 00000001ff6ed2a8 System.Web.DirectoryMonitor
27227 00000000107207d8 3 1 000000001f9b0cb0 47b4 260 000000016f85dcb8 System.Web.CachedPathData
27426 000000000c9ef2e8 3 1 000000000cbd3ff0 4058 131 00000001ff5a2ce8 System.Web.CachedPathData
27677 000000000ccee958 3 1 0000000015fa5d10 3b78 236 000000015f681dc8 System.Web.CachedPathData
27680 000000000ccd9588 3 1 00000000159aa750 4200 180 000000019fa2b008 System.Web.CachedPathData
27728 000000000c804c98 3 1 00000000159aae60 4568 211 00000001ff576860 System.Web.CachedPathData
27831 000000000a2d2a58 3 1 000000000cba4df0 43cc 127 00000000ff6f45a0 System.Web.CachedPathData
28088 00000000159b9688 5 1 000000001d139d30 3c40 177 000000023f887808 System.Web.CachedPathData
28491 00000000102d2fb8 7 1 000000000cacd460 411c 150 000000010f697cc8 System.Web.CachedPathData
28637 000000000a2d5f88 9 1 000000001ca9e8e0 41a8 130 000000024f6be1e8 System.Web.CachedPathData
28672 000000000c8675f8 3 1 000000000cbc38e0 40c4 151 000000018f481828 System.Web.CachedPathData
-----------------------------
Total 29135
CCW 2
RCW 2
ComClassFactory 0
Free 22330
> !threadpool
CPU utilization: 74%
Worker Thread: Total: 196 Running: 196 Idle: 0 MaxLimit: 2400 MinLimit: 24
Work Request in Queue: 32
Unknown Function: 000007feefe898e0 Context: 00000000109e2d10
Unknown Function: 000007feefe898e0 Context: 000000001d346680
AsyncTimerCallbackCompletion TimerInfo@0000000010a41d90
AsyncTimerCallbackCompletion TimerInfo@000000001cdeeb60
AsyncTimerCallbackCompletion TimerInfo@000000001f2c9860
Unknown Function: 000007feefe898e0 Context: 000000001ed65ad0
Unknown Function: 000007feefe898e0 Context: 000000000a1bc1c0
Unknown Function: 000007feefe898e0 Context: 000000001ef443e0
Unknown Function: 000007feefe898e0 Context: 0000000015410af0
Unknown Function: 000007feefe898e0 Context: 000000000fd96330
Unknown Function: 000007feefe898e0 Context: 000000001d624860
Unknown Function: 000007feefe898e0 Context: 000000000fc6e500
Unknown Function: 000007feefe898e0 Context: 0000000015b8e110
Unknown Function: 000007feefe898e0 Context: 00000000158dc6b0
Unknown Function: 000007feefe898e0 Context: 000000001f1ab600
Unknown Function: 000007feefe898e0 Context: 000000000cf05e38
Unknown Function: 000007feefe898e0 Context: 000000001d0b1c50
AsyncTimerCallbackCompletion TimerInfo@0000000003849290
Unknown Function: 000007feefe898e0 Context: 000000001c926f10
Unknown Function: 000007feefe898e0 Context: 000000001d472c80
Unknown Function: 000007feefe898e0 Context: 000000000cf15af0
Unknown Function: 000007feefe898e0 Context: 000000001f5b00c0
Unknown Function: 000007feefe898e0 Context: 000000000a022fa0
Unknown Function: 000007feefe898e0 Context: 000000001049d2e0
Unknown Function: 000007feefe898e0 Context: 00000000105e9370
AsyncTimerCallbackCompletion TimerInfo@000000000c985cd0
Unknown Function: 000007feefe898e0 Context: 000000001d376960
Unknown Function: 000007feefe898e0 Context: 000000001ee3f0d0
Unknown Function: 000007feefe898e0 Context: 000000001ee325b0
Unknown Function: 000007feefe898e0 Context: 0000000015f726b0
Unknown Function: 000007feefe898e0 Context: 0000000015598480
Unknown Function: 000007feefe898e0 Context: 000000000ce2a8a0
--------------------------------------
Number of Timers: 80
--------------------------------------
Completion Port Thread:Total: 5 Free: 3 MaxFree: 48 CurrentLimit: 5 MaxLimit: 2400 MinLimit: 24
2 个解决方案
#1
3
Okay, problem found and solved. This weekend someone pointed me at this Server Fault question. That seemed plausible, sending all requests through the routing module would definitely result in a performance issue. After going through some other posts, I realized it was a configuration issue. We were sending all requests through all managed modules:
好的，发现并解决了问题。本周末有人指出我这个服务器故障问题。这似乎是合理的，通过路由模块发送所有请求肯定会导致性能问题。经过其他一些帖子后，我意识到这是一个配置问题。我们通过所有托管模块发送所有请求：
<system.webServer>
<modules runAllManagedModulesForAllRequests="true" />
</system.webServer>
This is set to false and our web application is now behaving as expected.
这设置为false，我们的Web应用程序现在正在按预期运行。
@Aristos: even though I did not get to follow up on your suggestion, thanks for thinking along.
@Aristos：尽管我没有跟进你的建议，但感谢你的思考。
#2
0
This is a temporary solution. If you are gonna change your application in the future and use some managed modules (which has made web developing easy) you won't. You have disabled all managed modules for all requests which means you cannot benefit from managed modules in your App anymore.
这是一个临时解决方案。如果您将来要更改您的应用程序并使用一些托管模块（这使得Web开发变得简单），您将不会。您已为所有请求禁用了所有托管模块，这意味着您无法再从应用程序中的托管模块中受益。
Since changing the config has solved your problem without any drawbacks (App is working as before and no feature is turned off), I guess you have a lot of unnecessary modules loaded into your App which are consuming a lot of resources per request. IIS doesn't know if a module has to do something with a request or not, so if you have loaded the module for a request type, IIS will pass the request to managed code and these unnecessary transitions eats some of your resources (when the modules have nothing to do). Try removing any unnecessary modules from your web.config and change back the configuration to runAllManagedModulesForAllRequests="true" unless you sure no managed module is needed in the future.
由于更改配置已解决您的问题没有任何缺点（应用程序像以前一样工作，没有关闭任何功能），我猜你有很多不必要的模块加载到您的应用程序中，每个请求消耗大量资源。 IIS不知道模块是否必须对请求执行某些操作，因此如果您已为请求类型加载模块，IIS将把请求传递给托管代码，这些不必要的转换会占用您的一些资源（当模块无关）。尝试从web.config中删除任何不必要的模块，并将配置更改回runAllManagedModulesForAllRequests =“true”，除非您确定将来不再需要托管模块。
NOTE: You may not added this modules personally. Many of them are added by third party solutions (i.e using a CAPTCHA module) but they cannot find out you have changed your plan and removed the third party solution so the module remains in Bin folder (or GAC) and the line would not be removed from your config file.
注意：您可能未亲自添加此模块。其中许多是通过第三方解决方案添加的（即使用CAPTCHA模块），但他们无法找到您已更改计划并删除第三方解决方案，因此模块仍保留在Bin文件夹（或GAC）中，并且不会删除该行从您的配置文件。
#1
3
Okay, problem found and solved. This weekend someone pointed me at this Server Fault question. That seemed plausible, sending all requests through the routing module would definitely result in a performance issue. After going through some other posts, I realized it was a configuration issue. We were sending all requests through all managed modules:
好的，发现并解决了问题。本周末有人指出我这个服务器故障问题。这似乎是合理的，通过路由模块发送所有请求肯定会导致性能问题。经过其他一些帖子后，我意识到这是一个配置问题。我们通过所有托管模块发送所有请求：
<system.webServer>
<modules runAllManagedModulesForAllRequests="true" />
</system.webServer>
This is set to false and our web application is now behaving as expected.
这设置为false，我们的Web应用程序现在正在按预期运行。
@Aristos: even though I did not get to follow up on your suggestion, thanks for thinking along.
@Aristos：尽管我没有跟进你的建议，但感谢你的思考。
#2
0
This is a temporary solution. If you are gonna change your application in the future and use some managed modules (which has made web developing easy) you won't. You have disabled all managed modules for all requests which means you cannot benefit from managed modules in your App anymore.
这是一个临时解决方案。如果您将来要更改您的应用程序并使用一些托管模块（这使得Web开发变得简单），您将不会。您已为所有请求禁用了所有托管模块，这意味着您无法再从应用程序中的托管模块中受益。
Since changing the config has solved your problem without any drawbacks (App is working as before and no feature is turned off), I guess you have a lot of unnecessary modules loaded into your App which are consuming a lot of resources per request. IIS doesn't know if a module has to do something with a request or not, so if you have loaded the module for a request type, IIS will pass the request to managed code and these unnecessary transitions eats some of your resources (when the modules have nothing to do). Try removing any unnecessary modules from your web.config and change back the configuration to runAllManagedModulesForAllRequests="true" unless you sure no managed module is needed in the future.
由于更改配置已解决您的问题没有任何缺点（应用程序像以前一样工作，没有关闭任何功能），我猜你有很多不必要的模块加载到您的应用程序中，每个请求消耗大量资源。 IIS不知道模块是否必须对请求执行某些操作，因此如果您已为请求类型加载模块，IIS将把请求传递给托管代码，这些不必要的转换会占用您的一些资源（当模块无关）。尝试从web.config中删除任何不必要的模块，并将配置更改回runAllManagedModulesForAllRequests =“true”，除非您确定将来不再需要托管模块。
NOTE: You may not added this modules personally. Many of them are added by third party solutions (i.e using a CAPTCHA module) but they cannot find out you have changed your plan and removed the third party solution so the module remains in Bin folder (or GAC) and the line would not be removed from your config file.
注意：您可能未亲自添加此模块。其中许多是通过第三方解决方案添加的（即使用CAPTCHA模块），但他们无法找到您已更改计划并删除第三方解决方案，因此模块仍保留在Bin文件夹（或GAC）中，并且不会删除该行从您的配置文件。