Skip to content

Fix a reference leak of hw_frames_ctx and prepare for QSV #736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jan 10, 2023
Merged

Fix a reference leak of hw_frames_ctx and prepare for QSV #736

merged 1 commit into from
Jan 10, 2023

Conversation

cgutman
Copy link
Collaborator

@cgutman cgutman commented Jan 9, 2023

Description

This is some plumbing work required for QuickSync. During the QuickSync work, I discovered a reference leak caused by av_hwframe_get_buffer() clobbering the reference that we already stored there in frame->hw_frames_ctx. To avoid this, we need to pass the hw_frames_ctx pointer into set_frame() rather than set it in the frame itself.

Since QuickSync will need a fixed size frame pool, I changed display_vram_t to use av_hwframe_get_buffer() rather than allocating a texture itself.

No behavior changes expected (other than fixing the reference leak with CUDA and VAAPI on Linux).

Screenshot

Issues Fixed or Closed

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Dependency update (updates to dependencies)
  • Documentation update (changes to documentation)
  • Repository update (changes to repository files, e.g. .github/...)

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

  • I want maintainers to keep my branch updated

@github-actions github-actions bot changed the base branch from master to nightly January 9, 2023 03:36
@github-actions
Copy link

github-actions bot commented Jan 9, 2023

Your PR was set to master, PRs should be sent to nightly.
The base branch of this PR has been automatically changed to nightly.
Please check that there are no merge conflicts

@cgutman cgutman force-pushed the hw_frames_ctx_plumbing branch from 8563311 to 30a40d6 Compare January 10, 2023 05:13
@ReenigneArcher ReenigneArcher merged commit 44ad28e into LizardByte:nightly Jan 10, 2023
@cgutman cgutman mentioned this pull request Jan 12, 2023
Merged
11 tasks
@KarlVogel
Copy link
Contributor

After updating my sunshine, had a crash at startup, which I git bisected to this issue.

$ git bisect good
44ad28ebf4a17779e483971e80c260c74e972244 is the first bad commit
commit 44ad28ebf4a17779e483971e80c260c74e972244
Author: Cameron Gutman <aicommander@gmail.com>
Date:   Tue Jan 10 13:52:15 2023 -0600

    Fix a reference leak of hw_frames_ctx and prepare for QSV (#736)

 src/platform/common.h                   |  9 +++-
 src/platform/linux/cuda.cpp             | 17 ++++---
 src/platform/linux/vaapi.cpp            | 11 +++--
 src/platform/macos/nv12_zero_device.cpp |  2 +-
 src/platform/macos/nv12_zero_device.h   |  2 +-
 src/platform/windows/display_vram.cpp   | 87 ++++++++++++++++++++-------------
 src/video.cpp                           | 33 ++++++-------
 7 files changed, 92 insertions(+), 69 deletions(-)

The crash happens on a fedora 36 machine with an AMD Radeon RX6900XT, using VA-API and only when using KMS grabbing (ie. with setcap done).

Crash output:

./sunshine-0.17.0
[vt_software] -- [auto]
[amd_quality] -- [balanced]
[origin_web_ui_allowed] -- [lan]
[dwmflush] -- [enabled]
[gamepad] -- [x360]
[origin_pin_allowed] -- [pc]
[resolutions] -- [[
    352x240,
    480x360,
    858x480,
    1280x720,
    1920x1080,
    2560x1080,
    3440x1440,
    1920x1200,
    3860x2160,
    3840x1600
]]
[key_rightalt_to_key_win] -- [disabled]
[vt_realtime] -- [enabled]
[fec_percentage] -- [10]
[nv_coder] -- [auto]
[amd_coder] -- [auto]
[upnp] -- [disabled]
[nv_preset] -- [p4]
[nv_tune] -- [ull]
[vt_coder] -- [auto]
[amd_rc] -- [vbr_latency]
[hevc_mode] -- [0]
[nv_rc] -- [cbr]
[fps] -- [[10,30,60,90,120]]
[output_name] -- [0]
[min_log_level] -- [2]
[2023:01:18:15:10:38]: Error: Environment variable WAYLAND_DISPLAY has not been defined
[2023:01:18:15:10:38]: Info: Detecting connected monitors
[2023:01:18:15:10:38]: Info: // Testing for available encoders, this may generate errors. You can safely ignore those errors. //
[2023:01:18:15:10:38]: Info: Trying encoder [nvenc]
[2023:01:18:15:10:38]: Info: Screencasting with KMS
[2023:01:18:15:10:38]: Info: Found monitor for DRM screencasting
[2023:01:18:15:10:38]: Info: Color coding [Rec. 601]
[2023:01:18:15:10:38]: Info: Color range: [JPEG]
[2023:01:18:15:10:38]: Error: Failed to create a CUDA device: Operation not permitted
[2023:01:18:15:10:38]: Info: Screencasting with KMS
[2023:01:18:15:10:38]: Info: Found monitor for DRM screencasting
[2023:01:18:15:10:38]: Info: Color coding [Rec. 601]
[2023:01:18:15:10:38]: Info: Color range: [JPEG]
[2023:01:18:15:10:38]: Error: Failed to create a CUDA device: Operation not permitted
[2023:01:18:15:10:38]: Info: Screencasting with KMS
[2023:01:18:15:10:38]: Info: Found monitor for DRM screencasting
[2023:01:18:15:10:38]: Info: Color coding [Rec. 601]
[2023:01:18:15:10:38]: Info: Color range: [JPEG]
[2023:01:18:15:10:38]: Error: Failed to create a CUDA device: Operation not permitted
[2023:01:18:15:10:38]: Info: Screencasting with KMS
[2023:01:18:15:10:38]: Info: Found monitor for DRM screencasting
[2023:01:18:15:10:38]: Info: Color coding [Rec. 601]
[2023:01:18:15:10:38]: Info: Color range: [JPEG]
[2023:01:18:15:10:38]: Error: Failed to create a CUDA device: Operation not permitted
[2023:01:18:15:10:38]: Info: Encoder [nvenc] failed
[2023:01:18:15:10:38]: Info: Trying encoder [vaapi]
[2023:01:18:15:10:38]: Info: Screencasting with KMS
[2023:01:18:15:10:38]: Info: Found monitor for DRM screencasting
libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
libva info: VA-API version 1.14.0
[2023:01:18:15:10:38]: Info: Color coding [Rec. 601]
[2023:01:18:15:10:38]: Info: Color range: [JPEG]
libva info: Trying to open /usr/lib64/dri/radeonsi_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
zsh: segmentation fault (core dumped)  ./sunshine-0.17.0

Backtrace:

(gdb) bt
#0  0x00007fffe67c3b70 in ?? ()
#1  0x00007fffef26cae7 in vlVaDestroySurfaces () from /usr/lib64/dri/radeonsi_drv_video.so
#2  0x00007ffff7f1a02b in vaDestroySurfaces () from /lib64/libva.so.2
#3  0x0000000000e0124b in vaapi_buffer_free (opaque=0x26fdf80, data=<optimized out>) at libavutil/hwcontext_vaapi.c:463
#4  0x0000000000df5c97 in buffer_pool_flush (pool=0x26fe480) at libavutil/buffer.c:302
#5  av_buffer_pool_uninit (ppool=<optimized out>) at libavutil/buffer.c:332
#6  0x0000000000dfbeb3 in hwframe_ctx_free (opaque=<optimized out>, data=<optimized out>) at libavutil/hwcontext.c:230
#7  0x0000000000df57e9 in buffer_replace (src=0x0, dst=<optimized out>) at libavutil/buffer.c:133
#8  av_buffer_unref (buf=<optimized out>) at libavutil/buffer.c:144
#9  0x0000000000dfa11c in av_frame_unref (frame=<optimized out>) at libavutil/frame.c:493
#10 av_frame_unref (frame=0x26ed580) at libavutil/frame.c:477
#11 0x0000000000dfa23a in av_frame_free (frame=0x7fffffffcf98) at libavutil/frame.c:116
#12 0x0000000000d64670 in free_frame (frame=0x26ed580) at /home/kvo/source/Sunshine/src/platform/linux/graphics.cpp:868
#13 0x0000000000d5cb66 in util::Destroy<AVFrame*, void, &(free_frame(AVFrame*))>::operator() (this=0x2573220, p=0x26ed580) at /home/kvo/source/Sunshine/src/utility.h:800
#14 0x0000000000d5c492 in util::uniq_ptr<AVFrame, util::Destroy<AVFrame*, void, &(free_frame(AVFrame*))> >::reset (this=0x2573218, p=0x0) at /home/kvo/source/Sunshine/src/utility.h:501
#15 0x0000000000d5e949 in util::uniq_ptr<AVFrame, util::Destroy<AVFrame*, void, &(free_frame(AVFrame*))> >::~uniq_ptr (this=0x2573218, __in_chrg=<optimized out>) at /home/kvo/source/Sunshine/src/utility.h:496
#16 0x0000000000d5e6b4 in va::va_t::~va_t (this=0x25731f0, __in_chrg=<optimized out>) at /home/kvo/source/Sunshine/src/platform/linux/vaapi.cpp:278
#17 0x0000000000d5eaee in va::va_vram_t::~va_vram_t (this=0x25731f0, __in_chrg=<optimized out>) at /home/kvo/source/Sunshine/src/platform/linux/vaapi.cpp:411
#18 0x0000000000d5eee6 in std::_Destroy<va::va_vram_t> (__pointer=0x25731f0) at /usr/include/c++/12/bits/stl_construct.h:151
#19 0x0000000000d5ee9a in std::allocator_traits<std::allocator<void> >::destroy<va::va_vram_t> (__p=0x25731f0) at /usr/include/c++/12/bits/alloc_traits.h:648
#20 0x0000000000d5ed59 in std::_Sp_counted_ptr_inplace<va::va_vram_t, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose (this=0x25731e0) at /usr/include/c++/12/bits/shared_ptr_base.h:613
#21 0x0000000000b660c1 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x25731e0) at /usr/include/c++/12/bits/shared_ptr_base.h:346
#22 0x0000000000b6a479 in std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (this=0x7fffffffd188, __in_chrg=<optimized out>) at /usr/include/c++/12/bits/shared_ptr_base.h:1071
#23 0x0000000000cc8226 in std::__shared_ptr<platf::hwdevice_t, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0x7fffffffd180, __in_chrg=<optimized out>) at /usr/include/c++/12/bits/shared_ptr_base.h:1524
#24 0x0000000000cca741 in std::__shared_ptr<platf::hwdevice_t, (__gnu_cxx::_Lock_policy)2>::reset (this=0x7fffffffd2f0) at /usr/include/c++/12/bits/shared_ptr_base.h:1642
#25 0x0000000000cc925b in video::session_t::~session_t (this=0x7fffffffd2e0, __in_chrg=<optimized out>) at /home/kvo/source/Sunshine/src/video.cpp:326
#26 0x0000000000cd7d51 in std::_Optional_payload_base<video::session_t>::_M_destroy (this=0x7fffffffd2e0) at /usr/include/c++/12/optional:287
#27 0x0000000000cd2425 in std::_Optional_payload_base<video::session_t>::_M_reset (this=0x7fffffffd2e0) at /usr/include/c++/12/optional:318
#28 0x0000000000ccc4b8 in std::_Optional_payload<video::session_t, false, false, false>::~_Optional_payload (this=0x7fffffffd2e0, __in_chrg=<optimized out>) at /usr/include/c++/12/optional:439
#29 0x0000000000cc9b2c in std::_Optional_base<video::session_t, false, false>::~_Optional_base (this=0x7fffffffd2e0, __in_chrg=<optimized out>) at /usr/include/c++/12/optional:510
#30 0x0000000000cc9b48 in std::optional<video::session_t>::~optional (this=0x7fffffffd2e0, __in_chrg=<optimized out>) at /usr/include/c++/12/optional:705
#31 0x0000000000cc0754 in video::validate_config (disp=std::shared_ptr<platf::display_t> (use count 1, weak count 0) = {...}, encoder=..., config=...) at /home/kvo/source/Sunshine/src/video.cpp:1562
#32 0x0000000000cc0c3b in video::validate_encoder (encoder=...) at /home/kvo/source/Sunshine/src/video.cpp:1585
#33 0x0000000000cc28fb in video::init () at /home/kvo/source/Sunshine/src/video.cpp:1760
#34 0x0000000000b5b528 in main (argc=1, argv=0x7fffffffe578) at /home/kvo/source/Sunshine/src/main.cpp:322

@cgutman
Copy link
Collaborator Author

cgutman commented Jan 19, 2023

@KarlVogel can you file a bug for this so we don't lose track of it?

@KarlVogel KarlVogel mentioned this pull request Jan 19, 2023
Closed
3 tasks
@cgutman cgutman mentioned this pull request Jan 27, 2023
Merged
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@cgutman @KarlVogel @ReenigneArcher