Hey @cgutman, should this be revised in the doc too?
https://github.com/LizardByte/Sunshine/blob/master/docs/source/gamestream/gamestream.rst#limitations

Also, a random question, does this implementation of HDR for Windows requires the host to be plugged in an HDR display, like GameStream ?

Hey @cgutman, should this be revised in the doc too? https://github.com/LizardByte/Sunshine/blob/master/docs/source/gamestream/gamestream.rst#limitations

Yep, I missed removing that.

I added an HDR section in the Usage page of the docs: https://docs.lizardbyte.dev/projects/sunshine/en/nightly/about/usage.html#hdr-support

Also, a random question, does this implementation of HDR for Windows requires the host to be plugged in an HDR display, like GameStream ?

Yes it does. It's an OS limitation on a few fronts. Windows will not allow HDR to be turned on at the OS-level without an HDR-capable display or EDID emulator dongle connected. Without HDR enabled, DWM will not composite the desktop in HDR, so any HDR content rendered in games or applications would be clamped to SDR range before we could capture it using the Desktop Duplication API. Most HDR games won't even allow HDR to be enabled without an HDR display connected due to that limitation.

I have an idea of how to work around these OS limitations, but it requires injecting a DLL into games to hook some DirectX APIs. It's doable, but there's a ton of complexity in that approach (not to mention potential anti-cheat and game stability issues). I will probably come back to this approach because capturing frames straight from the game has a number of other benefits, including being able to encode on the render GPU rather than the display GPU on multi-GPU systems, but that's why it's not supported for v1 (and AFAIK no other Windows software can do HDR capture on SDR display today).

Great work! On the HDR support.

I wish the indirect display driver would support HDR. You can load a HDR EDID - similar to the NVIDIA data center driver, but nothing happens.

Some have suspected it is, because the wrong color space R8B8G8A8 is registered, but it’s unclear if Microsoft ever fixed that limitation.

It might be worth checking if the IDD driver can be hot-patched, before going down the route of intercepting the D3D calls - though as you said it might have other advantages.

FWIW, GameStream works fine with the IDD in SDR - did not have a chance to try SunShine, yet.

Great work! On the HDR support.

I wish the indirect display driver would support HDR. You can load a HDR EDID - similar to the NVIDIA data center driver, but nothing happens.

Some have suspected it is, because the wrong color space R8B8G8A8 is registered, but it’s unclear if Microsoft ever fixed that limitation.

My understanding (mostly from here) is that the DWM basically has 2 different composition pipelines that it can choose to use based on the connected displays and HDR setting.

One is the standard SDR/8-bit composition pipeline where DWM converts/clamps the presented frames into DXGI_FORMAT_B8G8R8A8_UNORM sRGB colorspace at the time they are received from the application. In this mode, the DWM composition pipeline is B8G8R8A8 all the way to the final presentation that gets picked up by the display driver. Any HDR or wide color gamut content (like DXGI_FORMAT_R10G10B10A2_UNORM or DXGI_FORMAT_R16G16B16A16_FLOAT) would be clipped by DWM prior to compositing the final image and unavailable for capture via the Desktop Duplication API.

The second mode is the "Advanced Color" mode which is used when HDR is enabled or an SDR display is connected that has Advanced Color capabilities. In this mode, DWM will take the application's frames in whatever format they are provided and feed that into the composition process. When DWM composites in this mode, it will use scRGB FP16 as the format of its composition buffer which allows it to keep the extra color information available in WCG/HDR content as it passes the buffer to the driver for presentation.

The standard composition pipeline makes sense for the vast majority of systems, since FP16 is twice the size of B8G8R8A8. Compositing in FP16 all the time would waste power and GPU bandwidth on systems where the output is going to be clamped to B8G8R8A8 anyway. For this reason, DWM only enables Advanced Color pipeline in scenarios where the extra color data will actually make it to the display.

There would need to be a way to convince DWM to use the Advanced Color composition pipeline to even have a shot at capturing HDR/WCG content. Emulating an SDR WCG display may work, but I suspect DWM is still going to clamp the output to [0.0f-1.0f] because the luminance behavior for SDR and HDR displays differs (1.0f is 80 nits on HDR displays and max display brightness on SDR displays). Unless the IDD also reports itself as an HDR10 display, games will probably not show the HDR option even if DWM is compositing to scRGB FP16.

It might be worth checking if the IDD driver can be hot-patched, before going down the route of intercepting the D3D calls - though as you said it might have other advantages.

Unless it can be done purely with registry changes, patching the driver is probably more trouble than doing the D3D/DXGI hooks because Windows kernel-mode code integrity rules are very strict. Drivers cannot (without an exploit) be patched in memory by a user-mode process.

FWIW, GameStream works fine with the IDD in SDR - did not have a chance to try SunShine, yet.

I think some people have reported that it works fine.