ScreenCaptureforHWND sample main thread hang

[ollie-dawes]

Background:

I am working on a C++ screen capture application and have been running into a few issues. One issue is occasionally our application hangs when we are tearing down our display capture logic. When I pause our application in the debugger I get the following trace for the main thread (cannot share full trace as it is a closed source product):

win32u.dll!00007ffc6ebca104() (Unknown Source:0)
user32.dll!00007ffc707a0dee() (Unknown Source:0)
combase.dll!00007ffc70ca2f99() (Unknown Source:0)
combase.dll!00007ffc70ca2dcb() (Unknown Source:0)
combase.dll!00007ffc70cacdb6() (Unknown Source:0)
combase.dll!00007ffc70cf9c03() (Unknown Source:0)
combase.dll!00007ffc70cab71b() (Unknown Source:0)
combase.dll!00007ffc70ca2a24() (Unknown Source:0)
combase.dll!00007ffc70cf937e() (Unknown Source:0)
rpcrt4.dll!00007ffc6fc4dc24() (Unknown Source:0)
combase.dll!00007ffc70cf71e8() (Unknown Source:0)
combase.dll!00007ffc70d6f0b2() (Unknown Source:0)
GraphicsCapture.dll!00007ffc3f012ff5() (Unknown Source:0)
GraphicsCapture.dll!00007ffc3f00afa0() (Unknown Source:0)
GraphicsCapture.dll!00007ffc3f00ac14() (Unknown Source:0)
GraphicsCapture.dll!00007ffc3f00acb4() (Unknown Source:0)
GraphicsCapture.dll!00007ffc3f00b5d8() (Unknown Source:0)
AppName.exe!Microsoft::WRL::ComPtr<ABI::Windows::Graphics::Capture::IGraphicsCaptureSession>::InternalRelease() Line 235 (c:\Users\<user>\redacted_path\winsdk-x64-min\Windows Kits\10\Include\10.0.22621.0\winrt\wrl\client.h:235)
AppName.exe!Microsoft::WRL::ComPtr<ABI::Windows::Graphics::Capture::IGraphicsCaptureSession>::~ComPtr() Line 291 (c:\Users\<user>\redacted_path\winsdk-x64-min\Windows Kits\10\Include\10.0.22621.0\winrt\wrl\client.h:291)

It looks like our IGraphicsCaptureSession is hanging when we delete the owning ComPtr.

I decided to pull down the ScreenCaptureforHWND sample from this repo to see if I could reproduce the issue and I was able to reproduce the problem. Pausing the sample in Visual studio after getting into the hung state gives the following trace on the main thread:

 	[External Code]	
->	ScreenCaptureforHWND.exe!winrt::impl::consume_Windows_Foundation_IClosable<winrt::Windows::Graphics::Capture::GraphicsCaptureSession>::Close() Line 121	C++
 	ScreenCaptureforHWND.exe!SimpleCapture::Close() Line 85	C++
 	ScreenCaptureforHWND.exe!App::StartCapture(HWND__ * hwnd) Line 62	C++
 	ScreenCaptureforHWND.exe!WndProc(HWND__ * hwnd, unsigned int msg, unsigned __int64 wParam, __int64 lParam) Line 177	C++
 	[External Code]	
 	ScreenCaptureforHWND.exe!WinMain(HINSTANCE__ * instance, HINSTANCE__ * previousInstance, char * cmdLine, int cmdShow) Line 156	C++
 	[External Code]	

Not quite the same as what we are seeing but the same problem and same reproduction steps.

How to reproduce:

This is a tricky one to reproduce in both our app and in the ScreenCaptureforHWND sample so I suspect there is a timing issue at play.

To reproduce the problem:

1. Build the ScreenCaptureforHWND sample.
2. Focus the window selection combo box and press up and down on the keyboard. The rate seems to matter here, you need to change sources quickly but not too quickly or the sample will crash for another unrelated reason. Can take a few minutes of source switching to happen on my end.
3. Notice that the GUI is no longer responding to events as the main thread is hung. Sometimes an indicator that the problem is about to occur is the main thread starts to get bogged down and the GUI feels sluggish just before a full on deadlock is reached.

System info:

Edition	Windows 10 Pro
Version	22H2
Installed on	‎13/‎02/‎2021
OS build	19045.4170
Experience	Windows Feature Experience Pack 1000.19054.1000.0

[robmikh]

Thanks for the report! I'll see if I can repro. There have been a number of fixes in Windows 11, are you able to reproduce there?

[ollie-dawes]

I'm afraid I don't have access to Win 11, if some of my colleagues have a few spare cycles they may be able to give it a test and give you an update here. So far it has only been me that has been able to reproduce this deadlock issue though within our code base, though others have seen the handle issue.