Skip to content

Conversation

psyke83
Copy link
Contributor

@psyke83 psyke83 commented Dec 1, 2022

Description

Implements IDXGIOutput5 with Per Monitor V2 DPI awareness support and additional fixes to fix mouse cursor rendering for hw encoding, and handle other screen formats that are used by exclusive fullscreen Vulkan and DX titles with "fullscreen optimizations" disabled.

Issues Fixed or Closed

Fixes: #553
May resolve error 0x887A0004 observed in DuplicateOutput on certain configurations and/or improve general capture performance.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

@psyke83 psyke83 marked this pull request as draft December 2, 2022 04:49
DXGI_FORMAT_R8G8B8A8_UNORM,
DXGI_FORMAT_B8G8R8A8_UNORM,
DXGI_FORMAT_R16G16B16A16_FLOAT,
DXGI_FORMAT_R10G10B10A2_UNORM,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can fix #376 here by specifying the appropriate formats (DXGI_FORMAT_B8G8R8A8_UNORM for SDR and DXGI_FORMAT_R10G10B10A2_UNORM for HDR, when that is implemented).

As I understand the docs, DWM should convert the format to match what we've asked for (for example, doing HDR -> SDR conversion if requested).

Copy link
Contributor Author

@psyke83 psyke83 Dec 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately I don't have any HDR capable monitors to test HDR or HDR -> SDR conversion; I even tried a virtual monitor driver for Windows, but it refuses to enable HDR in the virtual monitor.

Specifying the expanded list of formats doesn't seem to automatically convert formats, but it has allowed me to fix some other issues (added as separate commits).

Since my hands are tied in diagnosing/fixing HDR support due to lack of hardware, I would be more than happy to let you take over this PR (doesn't matter if you recommend this to be merged and then you continue, you cherry-pick some changes, or start over entirely, I don't mind).

In its present form, this PR will at minimum fix: #553

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifying the expanded list of formats doesn't seem to automatically convert formats, but it has allowed me to fix some other issues (added as separate commits).

It actually does, but we just made an mistake in our interpretation of the MSDN docs on it. DXGI_OUTDUPL_DESC will always give us the actual desktop mode, not the format that our frames are being converted to. See #609

For IDXGIOutput5, I think we have to actually acquire a frame and call GetDesc() on it to determine the actual format that was chosen from our supported list (or maybe just provide a list with only one SDR or HDR format to ensure we'll get that one).

… format

Resolves Sunshine disconnect due to incompatible format type
(DXGI_FORMAT_R8G8B8A8_UNORM) used by Vulkan applications in exclusive fullscreen,
or standard DX11/DX12 applications in exclusive fullscreen with "fullscreen
optimizations" disabled.

Also resolves missing cursor issue in certain UI elements such
as Notepad's main window.
…ive fullscreen

When screen format is set to DXGI_FORMAT_R8G8B8A8_UNORM, red and blue need to
be swapped during software colour conversion.

This format is selected in Vulkan or DX11/DX12 exclusive fullscreen modes
(the latter two occurring only if "disable fullscreen optimizations" flag is
also enabled).
@psyke83 psyke83 marked this pull request as ready for review December 22, 2022 03:13
@@ -291,27 +291,51 @@ int display_base_t::init(int framerate, const std::string &display_name) {
}

//FIXME: Duplicate output on RX580 in combination with DOOM (2016) --> BSOD
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed this comment from loki. My host GPU is a newer AMD GPU (RX 6600), but perhaps this is related to my problem; DOOM 2016 runs in Vulkan, so it could be using DXGI_FORMAT_R8G8B8A8_UNORM just like DX11 titles in true fullscreen exclusive mode.

I've been testing a game (Dying Light) with "disable fullscreen optimizations" checked. Before this PR, it wasn't possible to run in true exclusive fullscreen; now it works. However, if you toggle between resolutions in the settings menu, eventually it will either cause a driver timeout or complete BSOD. I was able to reproduce the crash on both libx264 and amdvce, and it looks like #605 doesn't solve my issue (but I realize that it wasn't the focus).

@@ -434,6 +434,7 @@ class hwdevice_t : public platf::hwdevice_t {

this->device_ctx_p = device_ctx_p;

DXGI_FORMAT src_format = format;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cgutman

Related to #609: I actually made an additional mistake here: I assumed that format was initialized with the correct value passed from display_base_t::init, but in fact it is just an uninitialized variable that resolves to DXGI_FORMAT_UNKNOWN, which coincidentally seems to handle both DXGI_FORMAT_R8G8B8A8_UNORM and DXGI_FORMAT_B8G8R8A8_UNORM. This is obviously not ideal and is supposed to select the proper format based on a similar img->is_bgr check as done here:

img->is_bgr = (format == DXGI_FORMAT_B8G8R8A8_UNORM ? true : false);

However, looking at #609, instead of fixing this, it looks like I should just remove both the vram and ram commits entirely in place of #609? I will not be able to edit the PR and verify in depth for a few hours, but I will be happy to back out those commits ASAP if you can verify my suspicion.

Copy link
Collaborator

@cgutman cgutman Dec 22, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think DXGI_FORMAT_UNKNOWN tells D3D11 to use the actual format of the texture (which is what we want).

The base IDXGIOutput5 part needs more work to handle the proper surface format selection. I think we want to avoid the HDR formats if we're not in HDR mode (and the SDR formats in HDR mode). I'll take a stab at it after the holidays.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants