I always dislike posting a half-done rushed analysis, but I ran out of time looking into this today. So this reply has some useful information, but more investigation is needed.

Executive Summary

The quick answer...

FFmpeg is embedded in the IINA application and can not be upgraded by the user
The problem extends beyond the version of FFmpeg being used

More details than you probably care to know follows...

IINA & mpv & FFmpeg

The IINA application embeds code from many open source projects. You can view the code that is embedded in the application, but you must not alter these files in any way whatsoever. Any changes will invalidate Code Signing causing macOS to terminate IINA at startup with a Code Signature Invalid error.

Listing the 3rd party code embedded in IINA using Terminal:

Embedded:

low-batt@gag ~$ ls /Applications/IINA.app/Contents/Frameworks/
Sparkle.framework		libfontconfig.1.dylib		liblept.5.dylib			libsodium.23.dylib		libswiftFoundation.dylib	libuchardet.0.dylib
libXau.6.dylib			libfreetype.6.dylib		libluajit-5.1.2.dylib		libsoxr.0.dylib			libswiftIOKit.dylib		libunistring.2.dylib
libXdmcp.6.dylib		libfribidi.0.dylib		liblz4.1.dylib			libspeex.1.dylib		libswiftMetal.dylib		libvidstab.1.1.dylib
libarchive.13.dylib		libgif.dylib			liblzma.5.dylib			libswiftAVFoundation.dylib	libswiftObjectiveC.dylib	libwebp.7.dylib
libass.9.dylib			libglib-2.0.0.dylib		libmpv.1.dylib			libswiftAppKit.dylib		libswiftQuartzCore.dylib	libwebpmux.3.dylib
libavcodec.58.dylib		libgmp.10.dylib			libmujs.dylib			libswiftCore.dylib		libswiftSafariServices.dylib	libxcb-shape.0.dylib
libavdevice.58.dylib		libgnutls.30.dylib		libnettle.8.dylib		libswiftCoreAudio.dylib		libswiftXPC.dylib		libxcb-shm.0.dylib
libavfilter.7.dylib		libgraphite2.3.dylib		libopenjp2.7.dylib		libswiftCoreData.dylib		libswiftos.dylib		libxcb-xfixes.0.dylib
libavformat.58.dylib		libharfbuzz.0.dylib		libp11-kit.0.dylib		libswiftCoreFoundation.dylib	libswiftsimd.dylib		libxcb.1.dylib
libavutil.56.dylib		libhogweed.6.dylib		libpcre.1.dylib			libswiftCoreGraphics.dylib	libswresample.3.dylib		libzimg.2.dylib
libb2.1.dylib			libidn2.0.dylib			libpng16.16.dylib		libswiftCoreImage.dylib		libswscale.5.dylib		libzmq.5.dylib
libbluray.2.dylib		libintl.8.dylib			libpostproc.55.dylib		libswiftCoreMedia.dylib		libtasn1.6.dylib		libzstd.1.dylib
libdav1d.5.dylib		libjpeg.9.dylib			librubberband.2.dylib		libswiftDarwin.dylib		libtesseract.5.dylib
libffi.8.dylib			liblcms2.2.dylib		libsnappy.1.dylib		libswiftDispatch.dylib		libtiff.5.dylib
low-batt@gag ~$

Many open source project use Autoconf configure scripts that provide many options as to how the source is built. In the case FFmpeg the configure script determines what features the resulting binary supports. An example from the FFmpeg configure script:

  --enable-libx265         enable HEVC encoding via x265 [no]

The [no] at the end is the default. Unless you enable x265 the resulting binary will not support HEVC encoding.

Most of the embedded code is built from sources by the IINA project specifically for embedding into IINA. Since IINA does not need FFmpeg encoding support the IINA build does not enable this feature.

And of course for IINA these must be universal binaries:

low-batt@gag Frameworks$ otool -L  libavcodec.58.dylib | grep architecture
libavcodec.58.dylib (architecture x86_64):
libavcodec.58.dylib (architecture arm64):
low-batt@gag Frameworks$

And support running under macOS 10.11+.

So upgrading/building dependencies is "interesting".

Testing FFmpeg & mpv

I am using this ProRes video file to test with:

mediainfo:

low-batt@gag issue-3874$ mediainfo DJI_0023.MOV 
General
Complete name                            : DJI_0023.MOV
Format                                   : MPEG-4
Format profile                           : QuickTime
Codec ID                                 : qt   0000.02 (qt  )
File size                                : 8.34 GiB
Duration                                 : 50 s 752 ms
Overall bit rate mode                    : Variable
Overall bit rate                         : 1 411 Mb/s
Encoded date                             : UTC 2021-11-03 14:15:51
Tagged date                              : UTC 2021-11-03 14:15:51
Writing application                      : DJIMavic3Cine
Cover                                    : Yes
snal                                     : (Binary)

Video
ID                                       : 1
Format                                   : ProRes
Format version                           : Version 0
Format profile                           : 422 HQ
Codec ID                                 : apch
Duration                                 : 50 s 751 ms
Bit rate mode                            : Variable
Bit rate                                 : 1 409 Mb/s
Width                                    : 5 120 pixels
Height                                   : 2 700 pixels
Display aspect ratio                     : 1.896
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 3.401
Stream size                              : 8.33 GiB (100%)
Writing library                          : DJI0
Language                                 : English
Encoded date                             : UTC 2021-11-03 14:15:51
Tagged date                              : UTC 2021-11-03 14:15:51
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Other #1
Type                                     : meta
Duration                                 : 50 s 752 ms
Bit rate mode                            : Variable
Default                                  : No

Other #2
Type                                     : dbgi
Duration                                 : 50 s 751 ms
Bit rate mode                            : Variable
Default                                  : No


low-batt@gag issue-3874$

The tests are being run on a MacBookPro18,2 with the M1 Max chip under macOS 12.4. The FFmpeg binary is 5.0.1 installed using Homebrew, so native code.

Hardware decoding is disabled by default, so I'm passing -hwaccel videotoolbox to enable it:

low-batt@gag issue-3874$ ffmpeg -hwaccel videotoolbox -i DJI_0023.MOV -map 0:a:0? -f null -

Montoring CPU usage showed hardware decoding is being used.

The mpv binary I'm using for testing is the 0.34.1 build from stolendata, running under Rosetta 2 using FFmpeg 4.4.1.

Hardware decoding is disabled by default, so I'm passing --hwdec=auto to enable it:

low-batt@gag issue-3874$ /Applications/mpv.app/Contents/MacOS/mpv --hwdec=auto --log-file=mpv.log DJI_0023.MOV

And the mpv.log shows hardware decoding was not used:

[   0.047][v][vd] Codec list:
[   0.047][v][vd]     prores - Apple ProRes (iCodec Pro)
[   0.047][v][vd] Opening decoder prores
[   0.047][v][vd] Not trying to use hardware decoding: codec prores is not on whitelist.
[   0.047][v][vd] Using software decoding.

I'm confused.

From section options-hwdec of the mpv manual:

auto: forcibly enable any hw decoder found (see below)
auto-safe: enable any whitelisted hw decoder (see below)

Why is mpv enforcing using of the whitelist when auto is used? Shouldn't that only happen when auto-safe is specified?

From section options-hwdec-codecs of the mpv manual:

--hwdec-codecs=<codec1,codec2,...|all>

Allow hardware decoding for a given list of codecs only. The special value all always allows all codecs.

You can get the list of allowed codecs with mpv --vd=help. Remove the prefix, e.g. instead of lavc:h264 use h264.

By default, this is set to h264,vc1,hevc,vp8,vp9,av1. Note that the hardware acceleration special codecs like h264_vdpau are not relevant anymore, and in fact have been removed from Libav in this form.

Looks like this PR mpv-player/mpv#9537 that adds prores to the whitelist was merged a bit after 0.34.0 was released. They missed the need to update the manual. Still I don't understand why auto is enforcing the whitelist.

Trying again with --hwdec-codecs=all:

low-batt@gag issue-3874$ /Applications/mpv.app/Contents/MacOS/mpv --hwdec=auto --hwdec-codecs=all --log-file=mpv.log DJI_0023.MOV

And the mpv.log shows hardware decoding was not used:

[   0.049][v][vd] Container reported FPS: 29.970030
[   0.049][v][vd] Codec list:
[   0.049][v][vd]     prores - Apple ProRes (iCodec Pro)
[   0.049][v][vd] Opening decoder prores
[   0.049][v][vd] No hardware decoding available for this codec.
[   0.049][v][vd] Using software decoding.

Adding --hwdec-codecs=all eliminated the whitelist issue, but for some reason mpv thinks hardware decoding is not available. Testing FFmpeg 4.4.1 downloaded from More ffmpeg releases and monitoring CPU usage showed hardware decoding was not used. Possibly that is why this build if mpv thinks hardware decoding is not available?

Conclusions

The main conclusion I have is some more investigation and testing is needed. It does seem like upgrading FFmpeg would provide hardware support for decoding ProRes. Apparently IINA will also need to be changed to add prores to the hwdec-codecs mpv configuration with the current release of mpv.

Uh oh!

ProRes Hardware Acceleration #3874

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions