Skip to content

Conversation

grabofus
Copy link
Collaborator

@grabofus grabofus commented May 1, 2025

This PR will...

Request key system access based on the eme configuration (with config.requireKeySystemAccessOnStart enabled) and block loading clear content until the media keys are set.

This resolves an issue on Chrome where if clear content is encrypted into the SourceBuffer followed by setting media keys then transitioning to encrypted content, Chrome will die with PIPELINE_DECODE_ERROR:
image

Why is this Pull Request needed?

  • If the HLS stream starts with clear content only in the manifest, and later transitions to encrypted content
  • If the HLS stream starts with mixed content in the manifest, but the first buffered segment is unencrypted. This creates a race-condition today, where the first clear segment might load before the media keys could be set. This has been noticed on MediaTailor live stream with prerolls, where the prerolls were loaded from disk cache, loading faster than it took the media keys to be set.

Are there any points in the code the reviewer needs to double check?

Resolves issues:

#4230
#5753

Checklist

  • changes have been done against master branch, and PR does not conflict
  • new unit / functional tests have been added (whenever applicable)
  • API or design changes are documented in API.md

@robwalch
Copy link
Collaborator

robwalch commented May 5, 2025

@grabofus I'm seeing very mixed results with https://sample-videos-zyrkp2nj.s3-eu-west-1.amazonaws.com/test-20250422/master_clear_to_cbcs_preset_media_keys_vod.html

Occasionally, Chrome successfully plays through and allows seeking between the discontinuities. At other times, it still fails with a pipeline error:

Media error in dev console:

$ video.error
MediaError {code: 3, message: 'PIPELINE_ERROR_DECODE: video decoder reinitialization failed'}

Media tab logs:

Cannot select DecryptingVideoDecoder for video decoding

Selected VideoToolboxVideoDecoder for video decoding, config: codec: h264, profile: h264 baseline, level: not available, alpha_mode: is_opaque, coded size: [640,360], visible rect: [0,0,640,360], natural size: [640,360], has extra data: false, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}

video decoder config changed midstream, new config: codec: h264, profile: h264 baseline, level: not available, alpha_mode: is_opaque, coded size: [640,360], visible rect: [0,0,640,360], natural size: [640,360], has extra data: false, encryption scheme: CBCS, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}

Cannot select DecryptingVideoDecoder for video decoding

video decoder reinitialization failed

  Error Group: DecoderStatus
  Error Code: 202
  Stacktrace: media/filters/decoder_selector.cc:213
  
  Error Group: PipelineStatus
  Error Code: 3
  Stacktrace: media/renderers/video_renderer_impl.cc:596
  
  Caused by:
    Error Group: DecoderStatus
    Error Code: 108
    Stacktrace: media/filters/decoder_stream.cc:196

On the occasion that I observed it play through without error, it detected the audio decoder config change first:

Cannot select DecryptingVideoDecoder for video decoding

Selected VideoToolboxVideoDecoder for video decoding, config: codec: h264, profile: h264 baseline, level: not available, alpha_mode: is_opaque, coded size: [640,360], visible rect: [0,0,640,360], natural size: [640,360], has extra data: false, encryption scheme: Unencrypted, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}

audio decoder config changed midstream, new config: codec: aac, profile: unknown, bytes_per_channel: 2, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Signed 16-bit, bytes_per_frame: 4, seek_preroll: 0us, codec_delay: 0, has extra data: false, encryption scheme: CBCS, discard decoder delay: false, target_output_channel_layout: STEREO, target_output_sample_format: Unknown sample format, has aac extra data: true

Selected DecryptingDemuxerStream for audio decryption, config: codec: aac, profile: unknown, bytes_per_channel: 2, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Signed 16-bit, bytes_per_frame: 4, seek_preroll: 0us, codec_delay: 0, has extra data: false, encryption scheme: CBCS, discard decoder delay: false, target_output_channel_layout: STEREO, target_output_sample_format: Unknown sample format, has aac extra data: true

Selected FFmpegAudioDecoder for audio decoding, config: codec: aac, profile: unknown, bytes_per_channel: 2, channel_layout: STEREO, channels: 2, samples_per_second: 48000, sample_format: Signed 16-bit, bytes_per_frame: 4, seek_preroll: 0us, codec_delay: 0, has extra data: false, encryption scheme: Unencrypted, discard decoder delay: false, target_output_channel_layout: STEREO, target_output_sample_format: Unknown sample format, has aac extra data: true

video decoder config changed midstream, new config: codec: h264, profile: h264 baseline, level: not available, alpha_mode: is_opaque, coded size: [640,360], visible rect: [0,0,640,360], natural size: [640,360], has extra data: false, encryption scheme: CBCS, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}

Selected DecryptingVideoDecoder for video decoding, config: codec: h264, profile: h264 baseline, level: not available, alpha_mode: is_opaque, coded size: [640,360], visible rect: [0,0,640,360], natural size: [640,360], has extra data: false, encryption scheme: CBCS, rotation: 0°, flipped: 0, color space: {primaries:BT709, transfer:BT709, matrix:BT709, range:LIMITED}

@grabofus
Copy link
Collaborator Author

grabofus commented May 6, 2025

@robwalch this branch was missing an earlier commit from our fork that attempts to set media keys as soon as they are created, instead of waiting for the encrypted event. I've added that commit now, and republished the test pages.

https://sample-videos-zyrkp2nj.s3-eu-west-1.amazonaws.com/test-20250422/master_clear_to_cbcs_preset_media_keys.html
https://sample-videos-zyrkp2nj.s3-eu-west-1.amazonaws.com/test-20250422/master_clear_to_cbcs_preset_media_keys_vod.html

Copy link
Collaborator

@robwalch robwalch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update. It works quite nicely.

Can you look at why this unit test is failing?

https://github.com/video-dev/hls.js/actions/runs/14861488418/job/41727400833?pr=7216

AssertionError: expected setMediaKeys to have been called exactly 'once', but it was called twice

expect(media.setMediaKeys).callCount(1);

I'm taking a look now. To troubleshoot tests, add only to the test (it.only('should request keysystem...) and then start the test runner in Chrome in debuggable mode:

$ npm run test:unit:debug

@robwalch robwalch added this to the 1.6.3 milestone May 6, 2025
@@ -292,6 +294,7 @@ class EMEController extends Logger implements ComponentAPI {
certificate,
);
}
this.attemptSetMediaKeys(keySystem, mediaKeys);
Copy link
Collaborator

@robwalch robwalch May 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is resulting in a second call to setMediaKeys (failing unit test assertion that setMediaKeys is only called once).

Is it necessary? (perhaps so based on your last comment)

If there is a certificate, however, they will not be set. getMediaKeysPromise is only supposed to retrieve the media keys. They shouldn't be set here.

I would recommend this be done in selectKeySystem but that is now used in two places. We only want to do the attempt from key-loader, so a new public method or key-loader would be approapriate:

  public getKeySystemAccess(keySystemsToAttempt: KeySystems[]) {
    return this.getKeySystemSelectionPromise(keySystemsToAttempt).then(
      ({ keySystem, mediaKeys }) => {
        return this.attemptSetMediaKeys(keySystem, mediaKeys);
      },
    );
  }

  private selectKeySystem(

}
}
}
return null;
Copy link
Collaborator

@robwalch robwalch May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loadClear should more or less do the same thing whether or not the playlist has encrypted segments and requireKeySystemAccessOnStart is enabled.

Here I've modified it to only follow either path when there is not a selected format (this.emeController.getSelectedKeySystemFormats().length) and to call getKeySystemAccess, defined above, in either path when requireKeySystemAccessOnStart is enabled.

  loadClear(
    loadingFrag: Fragment,
    encryptedFragments: Fragment[],
  ): null | Promise<void> {
    if (
      this.emeController &&
      this.config.emeEnabled &&
      !this.emeController.getSelectedKeySystemFormats().length
    ) {
      // access key-system with nearest key on start (loading frag is unencrypted)
      if (encryptedFragments.length) {
        const { sn, cc } = loadingFrag;
        for (let i = 0; i < encryptedFragments.length; i++) {
          const frag = encryptedFragments[i];
          if (
            cc <= frag.cc &&
            (sn === 'initSegment' || frag.sn === 'initSegment' || sn < frag.sn)
          ) {
            return this.emeController
              .selectKeySystemFormat(frag)
              .then((keySystemFormat) => {
                frag.setKeyFormat(keySystemFormat);
                if (
                  this.emeController &&
                  this.config.requireKeySystemAccessOnStart
                ) {
                  const keySystem =
                    keySystemFormatToKeySystemDomain(keySystemFormat);
                  if (keySystem) {
                    return this.emeController.getKeySystemAccess([keySystem]);
                  }
                }
              });
          }
        }
      } else if (this.config.requireKeySystemAccessOnStart) {
        const keySystemsInConfig = getKeySystemsForConfig(this.config);
        if (keySystemsInConfig.length) {
          return this.emeController.getKeySystemAccess(keySystemsInConfig);
        }
      }
    }
    return null;
  }

I'm still not 100% on how useful getSelectedKeySystemFormats() is and whether or not there is a simpler way to test if a format needs to be selected and set when we have encrypted fragments.

@grabofus
Copy link
Collaborator Author

grabofus commented May 9, 2025

@robwalch Thanks for the review! I've incorporated the changes you've suggested and updated the test pages.
All unit tests are passing now. However, I did notice that some logs are duplicated now:

[...]
[log] > [eme]: Access for key-system "com.widevine.alpha" obtained
[log] > [eme]: Create media-keys for "com.widevine.alpha"
[log] > [eme]: Media-keys created for "com.widevine.alpha"
[log] > [eme]: Setting media-keys for "com.widevine.alpha"
[log] > [eme]: Setting media-keys for "com.widevine.alpha"
[log] > [eme]: Media-keys set for "com.widevine.alpha"
[log] > [eme]: Media-keys set for "com.widevine.alpha"
[log] > [stream-controller]: FRAG_LOADING->IDLE
[log] > [stream-controller]: Loading main sn: 0 of level 0 (frag:[0.000-6.000]) cc: 0 [0-2], target: 0
[...]

On a separate note, should the requireKeySystemAccessOnStart config option stay disabled by default?

@robwalch
Copy link
Collaborator

Hi @grabofus. Thanks for making those changes.

All unit tests are passing now. However, I did notice that some logs are duplicated now:

Any thoughts on deduping media-keys setting?

On a separate note, should the requireKeySystemAccessOnStart config option stay disabled by default?

I think it should for v1.6.3. File an issue for defaulting it to enabling it in v1.7.

@robwalch robwalch self-requested a review May 28, 2025 19:58
@robwalch robwalch changed the title fix: PIPELINE_DECODE_ERROR during clear to encrypted transition Add requireKeySystemAccessOnStart (fixes Chrome PIPELINE_DECODE_ERROR on clear to encrypted transition) May 28, 2025
@robwalch robwalch merged commit 681c31f into video-dev:master May 28, 2025
12 checks passed
@robwalch
Copy link
Collaborator

robwalch commented May 28, 2025

@grabofus I merged and followed up with #7284 to address mediaKeys being set multiple times. Please let me know if you spot anything else.

@grabofus
Copy link
Collaborator Author

@robwalch thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

2 participants