Skip to content

[extractor/opencast] Add ltitools interface to OpencastPlaylistIE and fix Tests #6371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 9, 2023

Conversation

C0D3D3V
Copy link
Contributor

@C0D3D3V C0D3D3V commented Feb 27, 2023

Description

I'm the original author of the Opencast Extractor (bwildenhain then took care of it further so that it would be merged into yt-dlp. Thanks again for that), and I want to make an update to the extractor. Opencast now has a new "interface" to access playlists, and that is now used more often than the old one. For that reason I extended the valid URLs of the OpencastPlaylist Extractor

Also I fixed the Tests. oc-video.ruhr-uni-bochum.de seems to be down currently so I replaced it with an other example. I also added the missing keys to the first test.

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

@C0D3D3V C0D3D3V changed the title [Opencast] Add ltitools interface to OpencastPlaylistIE and fix Tests [extractor/opencast] Add ltitools interface to OpencastPlaylistIE and fix Tests Feb 27, 2023
Copy link
Member

@Grub4K Grub4K left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not remove the old test. If it is failing then mark it as skip with an appropriate message.

@C0D3D3V
Copy link
Contributor Author

C0D3D3V commented Feb 27, 2023

@Grub4K there is not even a DNS record for oc-video.ruhr-uni-bochum.de anymore I am not sure if it will ever come back online. We could also just remove it from the hosts list.

@Grub4K
Copy link
Member

Grub4K commented Feb 27, 2023

I am always for removing stuff but tests are usually kept for reference. In the past @pukkandan wanted to keep stuff around. I think its good to go since its always in the git history and removing this wont hurt since we can know its unused and we have equivalent tests. We'll see

@pukkandan
Copy link
Member

When new test covers the same URL pattern and code path, it's fine to remove old one

@C0D3D3V
Copy link
Contributor Author

C0D3D3V commented Feb 28, 2023

I replaced https://oc-video.ruhr-uni-bochum.de/engage/ui/index.html?e=1&p=1&epFrom=b1a54262-3684-403f-9731-8e77c3766f9a
with https://electures.uni-muenster.de/engage/ui/index.html?e=1&p=1&epFrom=39391d10-a711-4d23-b21d-afd2ed7d758c
so the same pattern is still covered.

And for the new pattern I added https://oc-video1.ruhr-uni-bochum.de/ltitools/index.html?subtool=series&series=cf68a4a1-36b1-4a53-a6ba-61af5705a0d0&lng=de. That is the same playlist as in the first playlist test, but a diffrent pattern.

So the code coverage is the same as before (+ the new pattern).

It could only be that the old playlist I choosed had videos with diffrent formats than the new one, so that a little bit more code was covered. But I remember that the tests I added back then did not cover all format paths.
However expanding the code coverage was not meant to be part of this PR. But if it is requested I can add it or make a second PR in future.

@C0D3D3V
Copy link
Contributor Author

C0D3D3V commented Mar 3, 2023

So please give me instructions what I should do that you are fine with it.

@pukkandan
Copy link
Member

Pls be patient. Will review it when I can

@pukkandan pukkandan added the site-bug Issue with a specific website label Mar 4, 2023
@C0D3D3V
Copy link
Contributor Author

C0D3D3V commented Mar 9, 2023

Also sorry for the delay
PS. In the last rebase I only renamed the commit message to match the pattern.

@C0D3D3V C0D3D3V force-pushed the update_opencast branch 2 times, most recently from 2b8510a to cfdd7c3 Compare March 9, 2023 15:36
@pukkandan
Copy link
Member

Pls avoid force pushing in middle of review. Commit messages don't matter

@pukkandan pukkandan merged commit 3588be5 into yt-dlp:master Mar 9, 2023
elyse0 pushed a commit to elyse0/yt-dlp that referenced this pull request Mar 12, 2023
qa4FKm3mUr added a commit to qa4FKm3mUr/yt-dlp that referenced this pull request Jul 30, 2023
* [utils] Add hackish 'now' support for --download-sections

* [utils] Add microseconds to unified_timestamp

* [common] Extract start and end keys for Dash fragments

* [utils] Allow using local timezone for 'now' timestamps

* Use local timezone for download sections

* Add fixme in modified parse_chapters function

A range like '*(now-1hour)-(now-30minutes)' doesn't work

* [youtube] Support --download-sections for YT Livestream from start

* Create last_segment_url only if necessary

* Improve parse_chapters comments

* Fix linter

* [extractor/iq] Set more language codes (yt-dlp#6476)

Authored by: D0LLYNH0

* [extractor/opencast] Add ltitools to `_VALID_URL` (yt-dlp#6371)

Authored by: C0D3D3V

* [downloader/curl] Fix progress reporting

Bug in 8c53322
Closes yt-dlp#6490

* [extractor/youtube] Bypass throttling for `-f17`

and related cleanup

Thanks @AudricV for the finding

* [extractor/twitch] Fix `is_live` (yt-dlp#6500)

Closes yt-dlp#6494
Authored by: elyse0

* [extractor/cbc:gem] Update `_VALID_URL` (yt-dlp#6499)

Authored by: makeworld-the-better-one
Closes yt-dlp#6395

* Support loading info.json with a list at it's root

* [extractor/hidive] Fix login

Fixes yt-dlp#6493 (comment)

* [extractor/opencast] Fix format bug (yt-dlp#6512)

Authored by: C0D3D3V

* [extractor/rokfin] Re-construct manifest url (yt-dlp#6507)

Authored by: vampirefrog

* [extractor/youtube] Add client name to `format_note` when `-v` (yt-dlp#6254)

Authored by: Lesmiscore, pukkandan

* [extractor/youtube] Add extractor-arg `include_duplicate_formats`

* [extractor/youtube] Construct fragment list lazily

Building fragment list for all formats take significant time for large videos

* Support negative durations

* Revert "[utils] Allow using local timezone for 'now' timestamps"

This reverts commit 1799a6a.

* Add fragment count

* Fix unified_timestamp

* Remove tz_aware date code

* Add debug for selected section

* Add initial documentation

* Fix linter

* Fix linter

* Allow days in parse_duration

* Improve option documentation

* Add some documentation

* Lock less agressively

This gives a speed performance of about 30%

* Fix return values of _extract_sequence_from_mpd

* Always compute last_seq

* Support for epoch timestamps

* Update options docs

* Restore README.md

I think this is auto-generated by some script

* Add warning about --download-sections without --live-from-start

* Fix bug after merge

* Update yt_dlp/options.py

* Cleanup

---------

Co-authored-by: Elyse <26639800+elyse0@users.noreply.github.com>
Co-authored-by: Sophire <115919609+sophie0x@users.noreply.github.com>
Co-authored-by: D0LLYNH0 <67797325+D0LLYNH0@users.noreply.github.com>
Co-authored-by: Daniel Vogt <daniel-vogt@mail.de>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
Co-authored-by: makeworld <25111343+makeworld-the-better-one@users.noreply.github.com>
Co-authored-by: Daniel Vogt <c0d3d3v@mag-keinen-spam.de>
Co-authored-by: vampirefrog <vampirefrog@users.noreply.github.com>
Co-authored-by: Lesmiscore <nao20010128@gmail.com>
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
Co-authored-by: bashonly <bashonly@bashonly.com>
aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this pull request Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-bug Issue with a specific website
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants